Using Split met_em Files With real.exe

[smurd]

I am trying to run WRFv4.4.1 using split met_em files (io_form_metgrid = 102 in namelist.wps), but the resulting split met_em files cannot be read by real.exe, even though io_form_input is set to 102. Instead, I get the following error:

FATAL CALLED FROM FILE: <stdin> LINE: 409
error opening met_em.d01.2021-07-24_23_00_00.nc for input; bad date in namelist or file not in directory

All of my met_em files are named met_em.d01.YYYY-mm-DD_HH_MM_SS.nc.XXXX, where XXXX is the processor number. Based on the error message, it sounds like real.exe is looking for a single met_em file for each input time when it really should be looking for multiple, one for each processor. For what it's worth, both metgrid.exe and real.exe are run using 8 processors.

I created a simple example showing this behavior for a 10-km CONUS grid with RAP LBCs/ICs that is run for 1 hour. I uploaded the met_em files, namelist.wps, run_metgrid.sh (script used to run metgrid.exe), namelist.input, run_real.sh (script used to run real.exe), and the rsl.out and rsl.error files to the WRF Forum Nextcloud (file name: smurd_wrf_20221107.tar). My software stack is as follows:

szip/2.1
intel/18.0.5.274
impi/2018.4.274
hdf5/1.8.9
netcdf/4.2.1.1
pnetcdf/1.6.1
nco/4.9.3
ncl/6.5.0

Sidenote: Split met_em files are not really needed for the example I provided, but the goal is to eventually run WRF with a massive domain (several thousand by several thousand gridpoints), and in that case, I don't think having a single met_em is practical because it can be 10+ GB large.

Thanks!
Shawn

 

[Ming Chen]

Shawn,
if a parallel io_form is chosen for the metgrid output files, the real program must be run using the same number of processors.
Please confirm that you run with the same number of processors.

 

[smurd]

Hi Ming,

Thanks for the response!

Yup, both metgrid.exe and real.exe are run using 8 processors.

 

[smurd]

I'm not sure how to link to the tarball in Nextcloud, so I attached my namelists, rsl files, and job submission scripts in a separate tarball here.

 

[Ming Chen]

Can you set

nocolons = .false.,
nio_tasks_per_group = 0,
nio_groups = 1,

then try again?

 

[Ming Chen]

I am not sure whether the quilting option can be used for this case. Please let me know whether REAL can work with this option off. Thanks.

 

[smurd]

Hi Ming,

Thanks for the suggestion! Unfortunately, changing nocolons, nio_tasks_per_group, and nio_groups results in the same error with the exception that the underscores between the hours, minutes, and seconds in the met_em files are switched to colons (which is what would be expected with changing nocolons to .false.). I changed the met_em file names to follow the same convention (i.e., switched the underscores between the hours, minutes, and seconds to colons), but still got the same error.

 

[smurd]

Update: I added io_form_auxinput1 to namelist.input and set it to 102, and now real.exe is able to read the split met_em files (and was able to successfully run wrf.exe afterwards). After examining main/real_em.f90, it looks like WRF doesn't actually use io_form_input to determine the format of the met_em files for real.exe. Instead, io_form_auxinput1 is used. I'm guessing this is not the desired WRF behavior.

As a follow-up question, is auxiliary stream 1 used exclusively for reading input files? In other words, will setting io_form_auxinput1 to 102 have unintended consequences?

Thanks!

 

[Ming Chen]

Hi Smurd,

Many thanks for the update about this issue. I didn't realize that we need to add io_form_auxinput1 for reading split met_em files.

To answer your question, auxiliary stream 1 is used for reading met-em data. I don't think it will affect anything else.