Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

Resolved issues with GEFS ICs causing WRF to crash in first model time step

J

jaredlee

New member
For the last few days I've been struggling trying to figure out why my WRF (v4.4.2) runs consistently crashed in the first d01 model time step with extremely vague segmentation faults, when initialized with GEFS 0.5° data (both the pgrb2a.0p50 and pgrb2b.0p50 files) from a date in 2022. I finally figured out a solution, so I thought I'd post it here in case other people encounter similar issues initializing WRF from GEFS.

I ran ungrib separately on the pgrb2a and pgrb2b files (which I assigned the prefixes GEFS_A and GEFS_B), then used metgrid to combine them in the met_em files. Real also ran successfully. However, inspection of the wrfinput files revealed potential problems with the soil fields. The top SMOIS layer had negative soil moisture (-0.033 m3/m3) at certain locations along the coastlines of BC, WA, OR, and CA (see first image—the bad wrfinput file is on the bottom/right). I wrote a Python script to force those negative soil moisture values to be a small positive value (0.01 m3/m3), but WRF still seg-faulted in the first d01 time step. Then I finally looked at the TSLB field, and was alarmed at what I found along the ocean coastlines and around Great Salt Lake (see second image). Going down to the bottom soil layer, these same areas had soil temperatures at or below 10 K (not shown). Similar issues initializing from GEFS data were documented in a 2019 thread (The 2019 GEFS data cannot drive WRF4.0).

That led me to look at the met_em files that metgrid produced. I found that in those files, the top three soil layers looked fine, but the bottom soil moisture & soil temperature fields were really wonky (again, soil temps around 10 K near coastlines or shorelines of large lakes). My namelist.wps had this entry in the &metgrid namelist, with the GEFS_A files provided first:

fg_name = '/path/to/GEFS_A','/path/to/GEFS_B',

On a lark, I tried changing the order of those to:

fg_name = '/path/to/GEFS_B','/path/to/GEFS_A',

Rerunning metgrid with the GEFS_B files provided first yielded met_em files with no artifacts in the soil moisture or soil temperature fields—no more negative soil moisture pixels, and no more <10 K soil temperature pixels. Real ran successfully, and produced wrfinput files with no soil artifacts either (see the top/left plots in both attached images). WRF also successfully ran, instead of crashing with a seg fault in the first model time step. I've never encountered this issue before, and I don't know why it matters, but apparently the order in which you provide multiple datasets to metgrid with the fg_name variable matters—at least when using GEFS for ICs/LBCs.

I hope this post and solution helps save time and headaches for others.

Jared
 

Attachments

  • SMOIS.png
    SMOIS.png
    1.3 MB · Views: 12
  • TSLB.png
    TSLB.png
    1.4 MB · Views: 13
Dear Jared Lee,

Could you please share your instruction that how did you run Ungrib with two different sets of initial files together and then how you merge them with Metgrid? Thank you in advance.
 
fthbhz said:
Dear Jared Lee,

Could you please share your instruction that how did you run Ungrib with two different sets of initial files together and then how you merge them with Metgrid? Thank you in advance.
Click to expand...
Hi fthbhz,

You need to run ungrib separately on the "a" and "b" files. When you run on the "a" files (be sure to link to the "a" files using ./link_grib.csh), set this in your namelist.wps:

Code: 
&ungrib
 out_format = 'WPS',
 prefix = '/path/to/GEFS_A',
/

And then you would link to the "b" files using ./link_grib.csh, and then change your namelist to this:

Code: 
&ungrib
 out_format = 'WPS',
 prefix = '/path/to/GEFS_B',
/

Then metgrid can combine the two datasets together by setting this option in the &metgrid section of namelist.wps, as I mentioned in my original post:

fg_name = '/path/to/GEFS_B','/path/to/GEFS_A',

Hope that helps.

Jared
 
Hi Jared,
Thank you so much for sharing this information, which will be of great help for many people who want to use GEFS data.
Would you please clarify:
(1) where did you downlaod pgrb2a.0p50 and pgrb2b.0p50 ?
(2) Are you able to ungrib pgrb2a.0p50 and pgrb2b.0p50 simultaneously instead of ungrib them separately?
(3) for the option fg_name = '/path/to/GEFS_B','/path/to/GEFS_A', metgrid will read data from GEFS_B first, and then read GEFS_A. If the same meteorological field is available from both files, data from GEFS_A will be used. This indicates that soil data in GEFS_A are correct, whereas those in GEFS_B are wrong, ---- it is thus important for us to know the source of pgrb2a.0p50 and pgrb2b.0p50.
Thanks again for the detailed explanation and kind information!
 
Ming Chen said:
Hi Jared,
Thank you so much for sharing this information, which will be of great help for many people who want to use GEFS data.
Would you please clarify:
(1) where did you downlaod pgrb2a.0p50 and pgrb2b.0p50 ?
(2) Are you able to ungrib pgrb2a.0p50 and pgrb2b.0p50 simultaneously instead of ungrib them separately?
(3) for the option fg_name = '/path/to/GEFS_B','/path/to/GEFS_A', metgrid will read data from GEFS_B first, and then read GEFS_A. If the same meteorological field is available from both files, data from GEFS_A will be used. This indicates that soil data in GEFS_A are correct, whereas those in GEFS_B are wrong, ---- it is thus important for us to know the source of pgrb2a.0p50 and pgrb2b.0p50.
Thanks again for the detailed explanation and kind information!
Click to expand...
You're welcome, Ming! And in answer to your questions:

1. I downloaded the files from AWS. At this point I forget exactly which date/cycle this was for, but I downloaded files like these using a Python script I wrote:


2. I ungribbed the "a" and "b" files separately and in separate directories so that intermediate or final output wouldn't get clobbered if I ran the different instances of ungrib simultaneously. I am unaware of a way to ungrib the "a" and "b" files simultaneously in a single execution of ungrib.exe, but if someone knows how to do that, that would save some complexity.

Jared
 
Last edited:
jaredlee said:
For the last few days I've been struggling trying to figure out why my WRF (v4.4.2) runs consistently crashed in the first d01 model time step with extremely vague segmentation faults, when initialized with GEFS 0.5° data (both the pgrb2a.0p50 and pgrb2b.0p50 files) from a date in 2022. I finally figured out a solution, so I thought I'd post it here in case other people encounter similar issues initializing WRF from GEFS.

I ran ungrib separately on the pgrb2a and pgrb2b files (which I assigned the prefixes GEFS_A and GEFS_B), then used metgrid to combine them in the met_em files. Real also ran successfully. However, inspection of the wrfinput files revealed potential problems with the soil fields. The top SMOIS layer had negative soil moisture (-0.033 m3/m3) at certain locations along the coastlines of BC, WA, OR, and CA (see first image—the bad wrfinput file is on the bottom/right). I wrote a Python script to force those negative soil moisture values to be a small positive value (0.01 m3/m3), but WRF still seg-faulted in the first d01 time step. Then I finally looked at the TSLB field, and was alarmed at what I found along the ocean coastlines and around Great Salt Lake (see second image). Going down to the bottom soil layer, these same areas had soil temperatures at or below 10 K (not shown). Similar issues initializing from GEFS data were documented in a 2019 thread (The 2019 GEFS data cannot drive WRF4.0).

That led me to look at the met_em files that metgrid produced. I found that in those files, the top three soil layers looked fine, but the bottom soil moisture & soil temperature fields were really wonky (again, soil temps around 10 K near coastlines or shorelines of large lakes). My namelist.wps had this entry in the &metgrid namelist, with the GEFS_A files provided first:

fg_name = '/path/to/GEFS_A','/path/to/GEFS_B',

On a lark, I tried changing the order of those to:

fg_name = '/path/to/GEFS_B','/path/to/GEFS_A',

Rerunning metgrid with the GEFS_B files provided first yielded met_em files with no artifacts in the soil moisture or soil temperature fields—no more negative soil moisture pixels, and no more <10 K soil temperature pixels. Real ran successfully, and produced wrfinput files with no soil artifacts either (see the top/left plots in both attached images). WRF also successfully ran, instead of crashing with a seg fault in the first model time step. I've never encountered this issue before, and I don't know why it matters, but apparently the order in which you provide multiple datasets to metgrid with the fg_name variable matters—at least when using GEFS for ICs/LBCs.

I hope this post and solution helps save time and headaches for others.

Jared
Click to expand...
Dear Jared

Thanks for helping the WRF community. I am also trying to run WRF with GEFS data however, I need some information. Did you use the invidual control members for the GEFS data? I want to use the ensemble mean data (geavg.) However, upon running the metgrid.exe I get following error.
ERROR: The mandatory field TT was not found in any input data.
 
saabokhari said:
Dear Jared

Thanks for helping the WRF community. I am also trying to run WRF with GEFS data however, I need some information. Did you use the invidual control members for the GEFS data? I want to use the ensemble mean data (geavg.) However, upon running the metgrid.exe I get following error.
ERROR: The mandatory field TT was not found in any input data.
Click to expand...
Hi saabokhari,

I have not tried using geavg files to drive WRF. I've only tried using the individual member files (gep01, gep02, etc.).

Jared
 
You must log in or register to reply here.
Top