Slow model writes

Topics specifically related to the wrf.exe program
Post Reply
neel14
Posts: 77
Joined: Fri Mar 15, 2019 6:52 am

Slow model writes

Post by neel14 » Sun Aug 30, 2020 4:12 am

Hi,
I am getting slow model writes with Netcdf4 compression.I am using Netcdf-4.5.0, netcdf-fortran-4.4.4 and hdf5-1.8.18. Also all my restart files are of different sizes. Sharing from the rsl file for a few timestamps:

Code: Select all

Timing for Writing /scratch/sharadbg/OUTPUT/wrfout_d01_2018-04-07_00:00:00 for domain        1:   28.26053 elapsed seconds
Timing for Writing /scratch/sharadbg/OUTPUT/wrf_trad_fields_d01_2018-04-07_00:00:00 for domain        1:    2.43434 elapsed seconds
......................
.....................
Timing for Writing /scratch/sharadbg/OUTPUT/wrfout_d01_2018-04-07_01:00:00 for domain        1:   55.84825 elapsed seconds
Timing for Writing /scratch/sharadbg/OUTPUT/wrf_trad_fields_d01_2018-04-07_01:00:00 for domain        1:    2.31206 elapsed seconds
.....................
.....................
Timing for Writing /scratch/sharadbg/OUTPUT/wrfout_d01_2018-04-07_02:00:00 for domain        1:   56.50365 elapsed seconds
Timing for Writing /scratch/sharadbg/OUTPUT/wrf_trad_fields_d01_2018-04-07_02:00:00 for domain        1:    2.31537 elapsed seconds
....................
....................
Kindly suggest ways to improve the model write speed. Any suggestion is appreciated.
Thanks!

kwerner
Posts: 1926
Joined: Wed Feb 14, 2018 9:21 pm

Re: Slow model writes

Post by kwerner » Tue Sep 01, 2020 6:48 pm

Hi,
With compression, we do expect the file sizes to be different, so that is okay. Are you still working with version 4.1.3 of the code? Will you attach your namelist.input file, along with your full runtime log (e.g., rsl.out.0000)? Thanks!
NCAR/MMM

neel14
Posts: 77
Joined: Fri Mar 15, 2019 6:52 am

Re: Slow model writes

Post by neel14 » Wed Sep 02, 2020 4:52 am

Hi,
Yes, I am using 4.1.3.
The wrfout writes take almost a minute for writing approximately 1.2 GB.
I just wanted to know if I can improve this through changing some settings(chunk cache size, # chunks, etc.) or any other changes that I should make. I don't know what are the optimum values of these parameters for WRF/WRF-Chem.

Thanks!
Attachments
namelist.input
(9.98 KiB) Downloaded 15 times
rsl.error.0000.txt
(435.06 KiB) Downloaded 14 times

kwerner
Posts: 1926
Joined: Wed Feb 14, 2018 9:21 pm

Re: Slow model writes

Post by kwerner » Fri Sep 04, 2020 7:56 pm

Thanks for sending those. One reason your write times may be slow is because compression is being done. I believe it takes a bit longer to actually compress the data. While you end up with smaller files, the time to get those can take a bit longer. Unfortunately I'm not very familiar with WRF-Chem output and what a normal write time is. That may be something you'll need to post in the wrf-chem section of the forum.
NCAR/MMM

Post Reply

Return to “wrf.exe”