Issues with ingesting a high-resolution EROD dataset in geogrid file

[Chapacha]

I am trying to use a high resolution EROD dataset at ~500 m resolution in a WRF domain at 9 km resolution but somehow my WPS version 3.9.1.1 stops every time when it processes the EROD dataset without producing any useful error messages. My "index" file is as follows:

type=continuous
signed = no
projection=regular_ll
dx=0.004166666667
dy=0.004166666667
known_x=1.0
known_y=1.0
known_lat=0.28959111180130842
known_lon=-13.331237185512171
wordsize=1
missing_value = 111
tile_x=19000
tile_y=12000
tile_z=3
scale_factor=0.01
units="fraction"
description="high resolution erod."

And the modified EROD part of my "GEOGRID.TBL" file is as follows:

===============================
name = EROD
priority = 1
dest_type = continuous
masked=water
z_dim_name=dust_erosion_dimension
fill_missing=0.
interp_option = default:average_gcell(4.0)
abs_path = default:/glade/p/ral/nsap/zhangyx/MAGEN-DVIOC/NewEROD/erod_data/
===============================

The last few lines of my geogrid.log.0000 are as follows:

2019-09-02 11:21:58.962 --- Processing field 15 of 29 (VAR)
2019-09-02 11:21:58.982 --- Processing field 16 of 29 (OA1)
2019-09-02 11:21:58.995 --- Processing field 17 of 29 (OA2)
2019-09-02 11:21:59.008 --- Processing field 18 of 29 (OA3)
2019-09-02 11:21:59.021 --- Processing field 19 of 29 (OA4)
2019-09-02 11:21:59.033 --- Processing field 20 of 29 (OL1)
2019-09-02 11:21:59.044 --- Processing field 21 of 29 (OL2)
2019-09-02 11:21:59.064 --- Processing field 22 of 29 (OL3)
2019-09-02 11:21:59.076 --- Processing field 23 of 29 (OL4)
2019-09-02 11:21:59.087 --- Processing field 24 of 29 (EROD)

It looks like that geogrid.exe stops when processing EROD. Since there are no error messages, I am not sure if the new EROD data are not read or if the data sizes are too big or if there are any other issues.

Thanks in advance for your help.

Yongxin

 

[kwerner]

Hi Yongxin,
Can you attach the following so that I can take a look:

1) full namelist.wps
2) full index file
3) 1 (or a few, if they aren't too large) of your EROD data files
4) modified GEOGRID.TBL
5) full geogrid.log

Thanks!

 

[Chapacha]

Hi Karl,

Thank you very much for your help with this. I attached the following files to this post:

1) full namelist.wps file
2) full index file
3) modified GEOGRID.TBL file (now the EROD part is modified)
4) full geogrid.log.0000 file (I used select=2:ncpus=36:mpiprocs=36 so there are 72 geogrid.log files)

There is only one single EROD data file that is named 00001-19000.00001-12000 and the file is very big with a size of ~700 MB. I copied this file under /glade/scratch/zhangyx/testEROD/ and hopefully you can access this file.

Please let me know if you need any more information.

Thanks again,
Yongxin

 

[kwerner]

Hi,
I believe the problem is that you are not actually asking to use the EROD data in your namelist.wps file. In your GEOGRID.TBL, you have this line:

abs_path = default:/glade/p/ral/nsap/zhangyx/MAGEN-DVIOC/NewEROD/erod_data/

But in your namelist.wps file, you have this set:

 geog_data_res             = 'modis_30s+30s',

This is telling the geogrid program to first use the modis_30s data, and where that is not available, to use the 30s data (paths specified in GEOGRID.TBL), but you are not asking it to use 'default' data. You need to set:

 geog_data_res             = 'default+modis_30s+30s',

I'm not certain, but it's likely hanging because you don't have the setting "optional = yes" in your GEOGRID.TBL, which is fine, as long as you do have the EROD data, and you are telling it the correct path to look for it.

 

[Chapacha]

Thank you very much for all of the suggestions. I made all the changes including setting "geog_data_res = 'default+modis_30s+30s'" in my namelist.wps file and setting "optional = yes" in my GEOGRID.TBL file but geogrid.exe still stops when processing EROD. In one of the tries, the following error message even appeared:

MPT: shepherd terminated: r13i6n15.ib0.cheyenne.ucar.edu - job aborting

It seems that somehow WPS is not reading the new EROD data or I am not sure if the new EROD data size is just too large. I am wondering if you have any other suggestions.

Thanks!
Yongxin

 

[Chapacha]

Using a single node with less tasks has solved the problem of not reading the new EROD data. Now my geogrid file has been ingested with the high-resolution EROD dataset. Thank you so much for helping me fix this issue!

Yongxin

 

[kwerner]

Yongxin,

I'm glad to hear that you were able to get the problem resolved. Thanks for the update!