Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

parallel netcdf-4 for WRF: Is there interest?

This post was from a previous version of the WRF&MPAS-A Support Forum. New replies have been disabled and if you have follow up questions related to this post, then please start a new thread from the forum home page.

microted

New member
Howdy,
I recently adapted the standard interface for WRF 3.9 to use the capability in netcdf-4.7.4 to do parallel file IO with compression enabled (as a separate IO option). It seems to work pretty well. If there is interest to have this in WRF 4, it should be fairly easy to port forward. File sizes are nearly the same as serial netcdf-4, and faster to write. One thing is that the write efficiency can be a little finicky depending on the domain decomposition, so suggestions on handling the chunking are welcome. (It seems to do better with tiles that are wider in x and shorter in y.)

Is there interest in adding this option to WRF-4.x? If so, I can work on a PR.

-- Ted

P.S. I tried to use the PIO interface that was in a pull request (but never accepted), but could not get it to run efficiently (using compressed parallel IO).
 
Ted,
Thanks for the kind information. I will forward your message to our software engineer and he will get back to you.
I know that some researchers do need this capability, especially those who work in the field of climate study.
Ming
 
Ted,
Would you please put a PR in GitHub? Then we can talk about the issue and proceed. Thanks!
Ming
 
Ming,
Sorry for the lag -- I must not have notifications set correctly. I'd be happy to submit a PR when I can make the time to transfer it to the latest WRF version.
-- Ted
 
Ted,
Please don't worry for the lag, --- anytime will be fine for us. We are almost ready to release WRFv4.3. After the release, we will be able to move on to incorporate new features either for minor release or for major release next year. Dave Gill is our chief software engineer for WRF and I am sure he will be happy to see the parallel netcdf-4 feature in WRF. Thanks and have a good day.
Ming
 
Top