Great, that is good news.
Mariusz
Change in soil variables from ECMWF Cycle 49r1
- Thread starter Gabriel Cassol
- Start date
Mariusz
mantovanijr
New member
Hi there! Is this solution able to manage ECMWF-IFS IC? I'm getting error when managing surface file for dates from 2023, the sd variable comes with different Edition (2) and packingType (grid_ccsds) than other variables. Take a look on the output of grid_ls file.nc:
edition centre typeOfLevel level dataDate stepRange dataType shortName packingType gridType
...
1 ecmf surface 0 20231027 0 an skt grid_simple regular_gg
2 ecmf snowLayer 0 20231027 0 an sd grid_ccsds regular_gg
edition centre typeOfLevel level dataDate stepRange dataType shortName packingType gridType
...
1 ecmf surface 0 20231027 0 an skt grid_simple regular_gg
2 ecmf snowLayer 0 20231027 0 an sd grid_ccsds regular_gg
Hi,
It's hard to say anything specific from the information you wrote. Are the data you're talking about pressure level or hybrid level data? Are these files in grib1 or grib2 format? The fix I suggested only supports ECMWF open data in grib2 format. ECMW in the IFS model in the second half of last year changed the coding of soil moisture and temperature fields in grib2. And my fix only works with such data. If you have a sample file from 2023 IFS, attach it.
M.
It's hard to say anything specific from the information you wrote. Are the data you're talking about pressure level or hybrid level data? Are these files in grib1 or grib2 format? The fix I suggested only supports ECMWF open data in grib2 format. ECMW in the IFS model in the second half of last year changed the coding of soil moisture and temperature fields in grib2. And my fix only works with such data. If you have a sample file from 2023 IFS, attach it.
M.
mantovanijr
New member
Hi,
I tried to attach the grb file, but it's too large for the server to handle. Can I send it to you via Google Drive? Let me know your email address.
Best,
Mantovani.
I do not use google drive. Please send the file via the website: WeTransfer | Send Large Files Fast
Problem solved
The matter is trivially simple. Your files are probably delivered directly from ECMWF. The Institute of Meteorology and Water Management, where I manage the Center for Numerical Weather Forecasts, also receives grib files operationally from ECMWF. Generally, ECMWF is moving to grib2 format and therefore some new fields are only in grib2 format, and older ones in grib1. If the contract states that files are to be delivered in grib1 format, they will be in this format, but other newer ones in grib2 format. Of course, everything is in one file and many programs cannot handle reading such a compiled grib1/grib2 file. See below, this is the content of your file:
od_oper_an_sfc_GF640_2023102712.grb
edition centre typeOfLevel level dataDate stepRange dataType shortName packingType gridType
1 ecmf surface 0 20231027 0 an ci grid_simple regular_gg
1 ecmf surface 0 20231027 0 an sst grid_simple regular_gg
1 ecmf depthBelowLandLayer 0 20231027 0 an swvl1 grid_simple regular_gg
1 ecmf depthBelowLandLayer 7 20231027 0 an swvl2 grid_simple regular_gg
1 ecmf depthBelowLandLayer 28 20231027 0 an swvl3 grid_simple regular_gg
1 ecmf depthBelowLandLayer 100 20231027 0 an swvl4 grid_simple regular_gg
1 ecmf surface 0 20231027 0 an slt grid_simple regular_gg
1 ecmf surface 0 20231027 0 an z grid_simple regular_gg
1 ecmf surface 0 20231027 0 an sp grid_simple regular_gg
1 ecmf depthBelowLandLayer 0 20231027 0 an stl1 grid_simple regular_gg
1 ecmf depthBelowLandLayer 7 20231027 0 an stl2 grid_simple regular_gg
1 ecmf depthBelowLandLayer 28 20231027 0 an stl3 grid_simple regular_gg
1 ecmf depthBelowLandLayer 100 20231027 0 an stl4 grid_simple regular_gg
1 ecmf surface 0 20231027 0 an sd grid_simple regular_gg
1 ecmf surface 0 20231027 0 an msl grid_simple regular_gg
1 ecmf surface 0 20231027 0 an 10u grid_simple regular_gg
1 ecmf surface 0 20231027 0 an 10v grid_simple regular_gg
1 ecmf surface 0 20231027 0 an 2t grid_simple regular_gg
1 ecmf surface 0 20231027 0 an 2d grid_simple regular_gg
1 ecmf surface 0 20231027 0 an lsm grid_simple regular_gg
1 ecmf surface 0 20231027 0 an skt grid_simple regular_gg
2 ecmf snowLayer 1 20231027 0 an sd grid_ccsds regular_gg
2 ecmf snowLayer 2 20231027 0 an sd grid_ccsds regular_gg
2 ecmf snowLayer 3 20231027 0 an sd grid_ccsds regular_gg
2 ecmf snowLayer 4 20231027 0 an sd grid_ccsds regular_gg
2 ecmf snowLayer 5 20231027 0 an sd grid_ccsds regular_gg
26 of 26 messages in od_oper_an_sfc_GF640_2023102712.grb
Notice the last five records in the surface fields file. The first column indicates the type of the grib file. You can clearly see that it is grib2, and all above it are grib1. The solution is to split the file. Save the grib1 fields to one file and grib2 to another. Only grib1 should be used for the WRF model. As far as I remember, no more corrections need to be made. You only need to check if the appropriate Vtable and metgrid.tbl files are prepared, but I think that should be it. Remember, we are working with grib1 files.
This is an example of how to save selected fields in grib1 format to a new grib1 file.
grib_copy -w shortName=msl/stl1/stl2/stl3/stl4/swvl1/swvl2/swvl3/swvl4 od_oper_an_sfc_GF640_2023102712.grb tmp_s.grib1
The second file is correct and contains pressure fields at 25 levels.
I hope I solved your problems. Good luck.
Mariusz
The matter is trivially simple. Your files are probably delivered directly from ECMWF. The Institute of Meteorology and Water Management, where I manage the Center for Numerical Weather Forecasts, also receives grib files operationally from ECMWF. Generally, ECMWF is moving to grib2 format and therefore some new fields are only in grib2 format, and older ones in grib1. If the contract states that files are to be delivered in grib1 format, they will be in this format, but other newer ones in grib2 format. Of course, everything is in one file and many programs cannot handle reading such a compiled grib1/grib2 file. See below, this is the content of your file:
od_oper_an_sfc_GF640_2023102712.grb
edition centre typeOfLevel level dataDate stepRange dataType shortName packingType gridType
1 ecmf surface 0 20231027 0 an ci grid_simple regular_gg
1 ecmf surface 0 20231027 0 an sst grid_simple regular_gg
1 ecmf depthBelowLandLayer 0 20231027 0 an swvl1 grid_simple regular_gg
1 ecmf depthBelowLandLayer 7 20231027 0 an swvl2 grid_simple regular_gg
1 ecmf depthBelowLandLayer 28 20231027 0 an swvl3 grid_simple regular_gg
1 ecmf depthBelowLandLayer 100 20231027 0 an swvl4 grid_simple regular_gg
1 ecmf surface 0 20231027 0 an slt grid_simple regular_gg
1 ecmf surface 0 20231027 0 an z grid_simple regular_gg
1 ecmf surface 0 20231027 0 an sp grid_simple regular_gg
1 ecmf depthBelowLandLayer 0 20231027 0 an stl1 grid_simple regular_gg
1 ecmf depthBelowLandLayer 7 20231027 0 an stl2 grid_simple regular_gg
1 ecmf depthBelowLandLayer 28 20231027 0 an stl3 grid_simple regular_gg
1 ecmf depthBelowLandLayer 100 20231027 0 an stl4 grid_simple regular_gg
1 ecmf surface 0 20231027 0 an sd grid_simple regular_gg
1 ecmf surface 0 20231027 0 an msl grid_simple regular_gg
1 ecmf surface 0 20231027 0 an 10u grid_simple regular_gg
1 ecmf surface 0 20231027 0 an 10v grid_simple regular_gg
1 ecmf surface 0 20231027 0 an 2t grid_simple regular_gg
1 ecmf surface 0 20231027 0 an 2d grid_simple regular_gg
1 ecmf surface 0 20231027 0 an lsm grid_simple regular_gg
1 ecmf surface 0 20231027 0 an skt grid_simple regular_gg
2 ecmf snowLayer 1 20231027 0 an sd grid_ccsds regular_gg
2 ecmf snowLayer 2 20231027 0 an sd grid_ccsds regular_gg
2 ecmf snowLayer 3 20231027 0 an sd grid_ccsds regular_gg
2 ecmf snowLayer 4 20231027 0 an sd grid_ccsds regular_gg
2 ecmf snowLayer 5 20231027 0 an sd grid_ccsds regular_gg
26 of 26 messages in od_oper_an_sfc_GF640_2023102712.grb
Notice the last five records in the surface fields file. The first column indicates the type of the grib file. You can clearly see that it is grib2, and all above it are grib1. The solution is to split the file. Save the grib1 fields to one file and grib2 to another. Only grib1 should be used for the WRF model. As far as I remember, no more corrections need to be made. You only need to check if the appropriate Vtable and metgrid.tbl files are prepared, but I think that should be it. Remember, we are working with grib1 files.
This is an example of how to save selected fields in grib1 format to a new grib1 file.
grib_copy -w shortName=msl/stl1/stl2/stl3/stl4/swvl1/swvl2/swvl3/swvl4 od_oper_an_sfc_GF640_2023102712.grb tmp_s.grib1
The second file is correct and contains pressure fields at 25 levels.
I hope I solved your problems. Good luck.
Mariusz
mantovanijr
New member
To successfully interpolate the ECMWF-IFS IC to the MPAS mesh the command is:
grib_copy -w shortName=msl/stl1/stl2/stl3/stl4/swvl1/swvl2/swvl3/swvl4/lsm/skt/ci/sst/slt/sp/z od_oper_an_sfc_GF640_2023102712.grb tmp_s.grib1
grib_copy -w shortName=msl/stl1/stl2/stl3/stl4/swvl1/swvl2/swvl3/swvl4/lsm/skt/ci/sst/slt/sp/z od_oper_an_sfc_GF640_2023102712.grb tmp_s.grib1
Super, I'm glad I could help
Mariusz
mantovanijr
New member
Thank you Mr Mariusz!
Hello Mariusz,
I am a new WRF user and I'm happy to find your answer. I trtied to use your rd_grib2.F and Vtable to run ungrib.exe for EC-IFS-0p25 data, but I failed. Here are the steps I followed:
1、I saved your Vtable.txt as /wrf/model/WPS-4.5/ungrib/Variable_Tables/Vtable.ECIFS. And I linked it using the command: ln -sf ungrib/Variable_Tables/Vtable.ECIFS Vtable
2、I replaced /wrf/model/WPS-4.5/ungrib/src/rd_grib2.F with your rd_grib2.F.
PS: My EC-IFS data is 2025032600z-3h-oper-fc.grib2 of 24h from 2025032600 to 2025032700. I download it by ecmwf-opendata python package.
However, when I ran ./ungrib.exe >& ungrib.log, I encountered some error. Could you please check if I missed or misunderstood anything?
Here is my ungrib.log for reference.
Thank you very much for your time and patience!
zhxx
@zhxx
NCAR RDA starts to provides ERA data in netCDF format. Please see
NCAR RDA Dataset d633000
We provide a python package to process the above ERA5 data:
GitHub - NCAR/era5_to_int: A simple Python script for converting ERA5 model-level netCDF files to the WPS intermediate format
This package is pretty easy to use and allow us to avoid troubles involved in processing ERA5 data.
Please try and let me know if you have any issues. Thanks.
NCAR RDA starts to provides ERA data in netCDF format. Please see
NCAR RDA Dataset d633000
We provide a python package to process the above ERA5 data:
GitHub - NCAR/era5_to_int: A simple Python script for converting ERA5 model-level netCDF files to the WPS intermediate format
This package is pretty easy to use and allow us to avoid troubles involved in processing ERA5 data.
Please try and let me know if you have any issues. Thanks.
Hi @figurski,
I am trying to use the he Vtable.ECMWF and your Vtable for the ECMWF IFS dataset for the WRF model v.4.6.0.
However, the use of Vtable and rd_grib2.F has not been successful, and the error is similar to @zhxx.
@Ming Chen What is the possibilities in use the ECMWF IFS files and running the model for forecasting?
I am trying to use the he Vtable.ECMWF and your Vtable for the ECMWF IFS dataset for the WRF model v.4.6.0.
However, the use of Vtable and rd_grib2.F has not been successful, and the error is similar to @zhxx.
@Ming Chen What is the possibilities in use the ECMWF IFS files and running the model for forecasting?
Hi Ana,
I checked the corrected rd_grib2.F file works in version 4.6. The error in the attached ungrib.log file indicates that the downloaded IFS grib2 file was not decompressed before being loaded by ungrib.
Mariusz
Hi Mariusz,
I created a script using eccodes to decompressed the IFS GRIB2. I was able to successfully run ungrib.
When I try to run metgrid using your Vtable, I can't determine if the process takes longer to execute because I kill the process.
However, with the Vtable.ECMWF from the model, there are indeed no soil variables, and during the real execution, it shows the message that num_soil_metgrid=0.
I will try again with your changes and provide feedback on the forum.
I created a script using eccodes to decompressed the IFS GRIB2. I was able to successfully run ungrib.
When I try to run metgrid using your Vtable, I can't determine if the process takes longer to execute because I kill the process.
However, with the Vtable.ECMWF from the model, there are indeed no soil variables, and during the real execution, it shows the message that num_soil_metgrid=0.
I will try again with your changes and provide feedback on the forum.
Hi Mariusz,
The link I used to download the data is:
ECMWF Data Store (ECPDS)
I used your Vtable and rd_grib2.F and compile WPS. The ungrib step was successful, but metgrid did not complete.
The only met_em.d01 file created shows num_metgrid_soil=4.
Attached are the log files for ungrib and metgrid.
Ana Teena
The link I used to download the data is:
ECMWF Data Store (ECPDS)
I used your Vtable and rd_grib2.F and compile WPS. The ungrib step was successful, but metgrid did not complete.
The only met_em.d01 file created shows num_metgrid_soil=4.
Attached are the log files for ungrib and metgrid.
Ana Teena
@Ana Teena
NCAR RDA archives NetCDF-4 structured version of the 0.25 degree atmospheric ECMWF Reanalysis 5 (ERA5), which is produced using high-resolution forecasts (HRES) at 31 kilometer resolution and a 62 kilometer resolution ten member 4D-Var ensemble of data assimilation (EDA) in CY41r2 of ECMWF's Integrated Forecast System (IFS) with 137 hybrid sigma-pressure (model) levels in the vertical. Please see
registry.opendata.aws
I wonder whether this data can meet your need ? if so, I guess you can avoid the troubles of running ungrib and move on to use the
python package to process the above ERA5 data. You can download the python package from:
GitHub - NCAR/era5_to_int: A simple Python script for converting ERA5 model-level netCDF files to the WPS intermediate format
NCAR RDA archives NetCDF-4 structured version of the 0.25 degree atmospheric ECMWF Reanalysis 5 (ERA5), which is produced using high-resolution forecasts (HRES) at 31 kilometer resolution and a 62 kilometer resolution ten member 4D-Var ensemble of data assimilation (EDA) in CY41r2 of ECMWF's Integrated Forecast System (IFS) with 137 hybrid sigma-pressure (model) levels in the vertical. Please see
NSF NCAR Curated ECMWF Reanalysis 5 (ERA5) - Registry of Open Data on AWS
registry.opendata.aws
I wonder whether this data can meet your need ? if so, I guess you can avoid the troubles of running ungrib and move on to use the
python package to process the above ERA5 data. You can download the python package from:
GitHub - NCAR/era5_to_int: A simple Python script for converting ERA5 model-level netCDF files to the WPS intermediate format
Hi @Ming Chen
The objective in use the ECMWF IFS dataset for the weather forecast for 3 days. The use the ERA5 isn't ideal for the analysis.
The idea is the new option for forecast for studies in rainfall for the Brazilian Northeast where the GFS hourly dataset used to forecasting.
@figurski the metgrid.tbl used in the test is the adopted the WPS file in github.
Ana Teena
The objective in use the ECMWF IFS dataset for the weather forecast for 3 days. The use the ERA5 isn't ideal for the analysis.
The idea is the new option for forecast for studies in rainfall for the Brazilian Northeast where the GFS hourly dataset used to forecasting.
@figurski the metgrid.tbl used in the test is the adopted the WPS file in github.
Ana Teena