Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

Questions on using the ECMWF IFS forecast data with WPS

This post was from a previous version of the WRF&MPAS-A Support Forum. New replies have been disabled and if you have follow up questions related to this post, then please start a new thread from the forum home page.

Chapacha

New member
Sorry to bother you again about using the ECMWF IFS (Integrated Forecasting System) data with WPS. These data are downloaded in near real time by our collaborators in Saudi Arabia on their machine. Thanks to the help I got from WRFHelp and this forum, I was able to run WPS/WRF using the ECMWF IFS data obtained by NCAR/RDA (dataset ds113.1) but I have run into many problems using the ECMWF IFS data downloaded in Saudi Arabia.

I attached one data file here (P1D12111200121618001: I needed to add "txt" at the end of the file name so the file could be attached). This data file (and all other data files that are downloaded) is in Grib format but it contains both Grib1 data and Grib2 data since I need to use wgrib to read some data and wgrib2 to read some other data. I used all tricks that I learned from using NCAR/RDA's ds113.1 datasets but none of them worked. Then I used wgrib to output the Grib1 data in a Grib1 file (command: wgrib ECMWF.grb -grib -o ECMWF.grb1 -d all) and wgrib2 to output the Grib2 data in a Grib2 file (command: wgrib2 ECMWF.grb -grib ECMWF.grb2). Once I ran g1print.exe to read the Grib1 file, I always got the following errors (if I ran g2print.exe to read the Grib2 file, I got "Successful completion of g2print"):

###################
WPS.ECMWF/gribdata> ./g1print.exe P1D12111200121618001.grb1
Copen: File = P1D12111200121618001.grb1
Fortran Unit = 29360191
UNIX File descriptor: 3

----------------------------------------------------
rec GRIB GRIB Lvl Lvl Lvl Time Fcst
Num Code name Code one two hour
----------------------------------------------------
End-of-record mark (7777) not found1610993754
Sec0(1) = 26214508 0
1 133 Q 100 850 0 2019-12-11_12:00 +126
*** stopping in findgrib in gribcode ***\n
\tI could not find the GRIB string in the input file
\tafter testing the first 100,000 bytes.
\tThe file may be corrupt or it is not a GRIB file.
\tPerhaps a gzipped GRIB file or netcdf?\n
findgrib
#################

Then I asked the system administrators to install ECMWF's "eccodes" package (this package is specifically designed for decoding and encoding the ECMWF IFS data) on their machine and used this package to output the required variables in individual files in Grib1 format; however, I always got the error messages of "End-of-record mark (7777) not found" and g1print.exe was never successful for reading those individual files. I also tried with "eccodes" on Cheyenne and got the same problems.

I am wondering if anyone could offer some help or suggestions as how to solve the problems.

Thanks!
Yongxin
 

Attachments

  • P1D12111200121618001.txt
    192.4 MB · Views: 96
Hi Yongxin

I would do the following to separate the file into two files:

wgrib -s P1D12111200121618001.grb > list.grib1
wgrib2 -s P1D12111200121618001.grb > list.grib2

wgrib -i P1D12111200121618001.grb -o P1D12111200121618001.grib1 -grib < list.grib1
wgrib2 -i P1D12111200121618001.grb -grib P1D12111200121618001.grib2 < list.grib2

You need to apply ungrib.exe separately to each file and then combine them in metgrid.exe.
However, it doesn't look like there is anything useful in the fields in grib2 format.

The other thing to consider is to make sure your Vtable matches with your input file.
There are different Vtables for different models.

I was able to make it work with one time in ungrib.exe and metgrid.exe for that single
time.
 
Hi, I have a query related to using ECMWF IFS operational data but different from the previous post.

I am attempting to use ECMWF meteorological data downloaded from the ECMWF archive resolution of 0.1° x 0.1° lat/long grid but I have encountered errors in the ungribbing process.

Pressure variables and surface variables were downloaded in two separate files ecmwf.pl and ecmwf.sfc respectively for a specified area. Both files are grib1. The variables have been downloaded in accordance with https://confluence.ecmwf.int/display/UDOC/ECMWF+WRF+Test+Data

In preparation for input into WPS. I split the dataset by timestep using cdo spilthour command.
I tried two ways but get errors in both.
Test #1 ungribbing the pl and sfc variable together. It appears the ungrib package is unable to decode the input grib data. I have attached the View attachment ungrib_1.log

Test#2 Ungrib separately.
When I ungrib separately, I get a successful run for the surface levels but the intermediate files are empty. I have attached View attachment ungrib2.log. When the pl files are run separately I get the same result as when they are combined. There are problems with the pressure levels for sure.

For the purpose of troubleshooting, I have included supporting files for three timesteps and for 1 domain.
View attachment namelist.wps
I am using the default ECMWF vtable provided.
I will attach the original grib files in subsequent message.

Additionally, I have reviewed the previous post on using ecmwf IFS data with no luck.
I have reviewed these previous post with no luck.

https://forum.mmm.ucar.edu/phpBB3/v...it=End+of+record+mark+7777+not+found+0#p15081

https://forum.mmm.ucar.edu/phpBB3/viewtopic.php?f=32&t=433&p=1279&hilit=ec_rec_len#p1279

I am using GNU fortran and gcc compiler , Bash shell

Thanks in advance,
Adele
 

Attachments

  • ecmwf.sfc.split12.txt
    1.2 MB · Views: 36
  • ecmwf.sfc.split06.txt
    1.2 MB · Views: 39
  • ecmwf.sfc.split00.txt
    1.2 MB · Views: 44
  • ecmwf.pl.split12.txt
    3.2 MB · Views: 37
  • ecmwf.pl.split06.txt
    3.2 MB · Views: 43
  • ecmwf.pl.split00.txt
    3.2 MB · Views: 43
As promised I am attached the original grib files.
I could not include any more files in the previous message.


Regards,
Adele
 

Attachments

  • ecmwf.pl.2019.txt
    25.5 MB · Views: 48
  • ecmwf.sfc.2019.txt
    10 MB · Views: 50
Hi Adele,
To run ungrib program for ECMWF IFS, a new namelist option is needed for namelist record &ungrib:
ec_rec_len = 26214508,
This is needed for both upper air and surface data (https://rda.ucar.edu/datasets/ds113.1/). However this fix does not work for all surface fields. In particular there are 13 of them that need to be removed before running the ugrib program, and these are *cvl* *cvh* *ci* *sdfor* *sdor* *isor* *anor* *slor* *lsm* *_al* *_slt* *_sd* *_sstk*.
As one may notice, we have to remove sstk from surface fields. The alternative is to find another SST data source.
I am not sure whether this can solve your problem. Please keep us updated if you find a solution to this problem. Thanks.
 
Top