ndown for long period of time exceeding Cheyenne's Walltime

[mahdad]

Hello,

What would be the best way to restart ndown? I am running a LES simulation for a very long time and because I am running my smallest domain with some modifications to the wrf code I run the smallest domain separately using ndown to get initial and boundary conditions for my innermost domain from the top domain. However due to walltime limit on Cheyenne I won't be able to run ndown for the whole period of time. I have the following questions:

• Do you have any suggestions for how I can run it for the whole period or how to restart ndown making several wrfbdy files. I have been using wrfrst files to restart the ndown process but I am not sure if this would have any significant effect on my simulation.

• Since there is no overlap between wrfinput and wrfbdy variables, I thought using wrfrst files for restarting ndown has no effect on the wrfbdy files generated. Is that correct? or does wrfinput static fields affect the interpolation of the 3-D meteorological fields in wrfbfy file?

Thank you,

 

[kwerner]

Hi,
I apologize for the delay in response. We are trying to catch up with forum posts after the WRF tutorial.

Is it the actual ndown.exe program that you aren't able to complete in the wallclock time, or is it the wrf.exe run for the fine-resolution domain that you aren't able to complete?

 

[mahdad]

Hi Kelly,

Thank you for getting back to me. It is the ndown.exe for the smaller grid.

Thanks,
Mahdad

 

[kwerner]

Mahdad,
Okay, thanks. So this is one way you could do it. I'll try to explain this without making too complicated. To simplify it, I'm going to discuss a 2 day case, calling each day: day1 and day2. Assuming you already have all of the wrfout_d01* files,

1) Run real.exe for 2 domains for day1. Afterward rename wrfinput_d02 to wrfndi_d02.
2) Using the wrfout_d01* files and wrfndi_d02 file, run ndown for day1. Afterward rename wrfinput_d02 to wrfinput_d01, and wrfbdy_d02 to wrfbdy_d01_day1.
3) Store the above wrfinput and wrfbdy files somewhere else.
4) Run real.exe for 2 domains for day2. Afterward rename wrfinput_d02 to wrfndi_d02.
5) Using the wrfout_d01* files and wrfndi_d02 file, run ndown for day2. You can discard (or save elsewhere) wrfinput_d02, and then rename wrfbdy_d02 to wrfbdy_d01_day2.
6) Bring the previously stored wrfinput_d01 and wrfbdy_d01_day1 file back to the running directory.
7) Using NCO tools, concatenate the wrfbdy files into a single file.

 ncrcat wrfbdy_d01_day1 wrfbdy_d01_day2 wrfbdy_d01

Now the new wrfbdy_d01 file should have all the boundary times in it. You can check with (ncdump -v Times wrfbdy_d01)
8) You can then use the wrfbdy_d01 and wrfinput_d01 file to run wrf.exe for d02, as always. You can restart whenever you need.

Does that make sense? To be sure this works, you could try this with a very small time frame to verify it's all okay, before trying the full run.

 

[mahdad]

Kelly,

Thank you very much for your response. I have a question with regards to your answer:

Would it be fine to use wrfrst_d01 at the end of the first day simulation instead of wrfinput_d01 (generated by real.exe in step 4 for day 2)? I have been using this method for a couple of months now. I have copied and renamed wrfrst_d01 to wrfndi_d02. Basically wrfrst had all the parameters in wrfinput and I figured I can use it and it worked. However, I have been concerned as I am not familiar with the ndown.exe routines and I am not sure that extra field data that exist in wrfrst compared to wrfinput has any effect on the interpolation schemes in ndown.


Thank you,
mahdad

 

[kwerner]

Hi Mahmad,
I suppose that should likely be okay. It may be worth doing a quick test by running 2 very short ndown simulations - one starting with the wrfrst* file, and the other starting with the wrfinput* (or wrfndi) file, and then comparing the output to make sure they are identical. You can use the WRF/external/io_netcdf/diffwrf utility to compare the output of the 2 runs at the same corresponding times.

 

[mahdad]

Kelly,

Thank you very much. I will run the test and will update you as soon as I get the results.

Also, it is not necessary to combine the two wrfbdy files into one using ncrcat (as explained in one of the steps), because I can use them separately. Correct?

Thank you for your help
Mahdad

 

[kwerner]

Mahmad,
If you will be using them in 2 different runs, then yes, that should be okay. But if you are running a continuous WRF run for d02, then you'll need to combine them.