About restart file generation

[samuel]

Dear wrfhelper,

I am using ARW-WRF v3.9.1 to run simulations but failed to write out rst files with io_form_restart=2 (the model just stopped). I compiled my wrf3.9.1 code with WRFIO_NCD_LARGE_FILE_SUPPORT=1 and set MP_STACK_SIZE and KMP_STACKSIZE as large as possible, although the WRF user guide mentions that the restriction of the only 2Gb basic netCDF file support is removed by default with WRF v3.9.
However, the WRF3.9.1 works well with io_form_restart=102 but with multi-pieces of rsl files. Since I am using a lot of nodes, there would be plenty of rst files (which I don’t like).
Also, I have tried WRF3.6 and it works with io_form_restart=2. Could you have some suggestions to solve this issue?

Thanks,

Sam

 

[kwerner]

Hi Sam,
Do you mind issuing the following from the directory where you are outputting files:
ls -ls >& ls.txt

Please send that ls.txt file, along with your namelist.input file and your rsl.out.0000/rsl.error.0000 files. Thanks!

 

[samuel]

Thank you kwerner.

Here I attached the four files you want to check. I put .txt extension for the two log files since their original format cannot be recognized when I uploaded to this forum.
BTW, I forgot to mention that I am actually using WRF-Chem (not WRF only). Thanks,

Sam

 

[kwerner]

Hi,
Thank you for sending those. In looking at your ls.txt file, I see that your wrfbdy file is about 3 GB, the wrfinput file is 11 GB and the first wrfout* file is 13 GB. The large-file support setting allows files to be written that are larger than 2 GB, but smaller than 4 GB. You have a fairly large domain with high-resolution, and you are outputting history every minute, which can all contribute to large file size, but perhaps since you are running WRF-chem, it adds a lot of extra content to the files, causing them to be as large as they are? If you are concerned about the file size, you can post a question to their group under the wrf-chem section of this forum. Unfortunately the only solution for files larger than 4 GB is to use the io_form_* setting of 102. As you mentioned, the downside is that it does create a lot of files (1 per processor), but we can provide a script that can help you to patch those files back together. You can obtain that here: http://www2.mmm.ucar.edu/wrf/users/special_code.html There is also a presentation you can download that gives instructions for using the program. Note: this script was provided to us by a WRF user so we did not develop it and do not officially support it. Hopefully this helps!

 

[samuel]

Thanks. I will try the code to combine all the rsl files. Thank you very much.

Sam

 

[kwerner]

Sam,
Just to be clear, the code will allow you to combine your wrf output files (not the rsl files).

 

[samuel]

Thanks for the remind. But actually I don't have any issue about history file (wrfout*) generation. They were always produced only one file every history interval, instead of multi-pieces (not like the rst files which were generated per node). This is why I am a little curious that even though the history file in my case is much larger than 4 GB but still generated in one piece. Unfortunately, this is not case for the rst file.

Sam

 

[kwerner]

Hi,
If you set io_form_restart = 102, then they will be output per processor. This is how the 102 format (which means "split" netcdf files) works. It splits the files up by processor so that each file will be smaller. I'm not sure how your wrfout file was able to be written out since it was so large. It's possible that it didn't actually complete or that it's not realistic, but I can't say for sure. I would recommend using the 102 option for all the output files from wrf.exe, and then using the joiner script to stitch them back together.

 

[samuel]

Yes, that's possible that wrfout file is not completely wrote out. I will try the joiner script. Thanks!

Sam