Slow model writes

This post was from a previous version of the WRF&MPAS-A Support Forum. New replies have been disabled and if you have follow up questions related to this post, then please start a new thread from the forum home page.

N

neel14

Member
Hi,
I am getting slow model writes with Netcdf4 compression.I am using Netcdf-4.5.0, netcdf-fortran-4.4.4 and hdf5-1.8.18. Also all my restart files are of different sizes. Sharing from the rsl file for a few timestamps:

Code: 
Timing for Writing /scratch/sharadbg/OUTPUT/wrfout_d01_2018-04-07_00:00:00 for domain        1:   28.26053 elapsed seconds
Timing for Writing /scratch/sharadbg/OUTPUT/wrf_trad_fields_d01_2018-04-07_00:00:00 for domain        1:    2.43434 elapsed seconds
......................
.....................
Timing for Writing /scratch/sharadbg/OUTPUT/wrfout_d01_2018-04-07_01:00:00 for domain        1:   55.84825 elapsed seconds
Timing for Writing /scratch/sharadbg/OUTPUT/wrf_trad_fields_d01_2018-04-07_01:00:00 for domain        1:    2.31206 elapsed seconds
.....................
.....................
Timing for Writing /scratch/sharadbg/OUTPUT/wrfout_d01_2018-04-07_02:00:00 for domain        1:   56.50365 elapsed seconds
Timing for Writing /scratch/sharadbg/OUTPUT/wrf_trad_fields_d01_2018-04-07_02:00:00 for domain        1:    2.31537 elapsed seconds
....................
....................

Kindly suggest ways to improve the model write speed. Any suggestion is appreciated.
Thanks!
 
Hi,
With compression, we do expect the file sizes to be different, so that is okay. Are you still working with version 4.1.3 of the code? Will you attach your namelist.input file, along with your full runtime log (e.g., rsl.out.0000)? Thanks!
 
Hi,
Yes, I am using 4.1.3.
The wrfout writes take almost a minute for writing approximately 1.2 GB.
I just wanted to know if I can improve this through changing some settings(chunk cache size, # chunks, etc.) or any other changes that I should make. I don't know what are the optimum values of these parameters for WRF/WRF-Chem.

Thanks!
 

Attachments

Thanks for sending those. One reason your write times may be slow is because compression is being done. I believe it takes a bit longer to actually compress the data. While you end up with smaller files, the time to get those can take a bit longer. Unfortunately I'm not very familiar with WRF-Chem output and what a normal write time is. That may be something you'll need to post in the wrf-chem section of the forum.
 
You must log in or register to reply here.
Back
Top