William.Hatheway
Active member
The log files you requested @Josh
How to use ERA5 Data From Copernicus Database
- Thread starter William.Hatheway
- Start date
-
- Tags
- metgrid running wps ungrib wps
@Josh
Hi Josh,
We are aware of this issue, although we haven't fixed the ungrib codes yet.
On November 12, ECMWF released version 49r1 of the operational IFS. Despite all the improvements, they modified the soil variables (temperature and moisture) which broke the ECMWF Vtable that had been released in the 4.6.0 version of the WPS.
Briefly, it is necessary to make a change within Ungrib, more specifically in the rd_grib2.F code so that it works with the new IFS soil variables. With 49R1 cycle, encoding of soild parameters changed from stl1 , stl2, stl3 , and stl4 changed to sot. and from swvl1 , swvl2, swvl3, and swvl4 changed to vsw image.png (view on web).
We will look at this issue and hopefully we can fix it soon.
Hello WRF users. I have created a new Vtable for ERA5 data (specifically tailored to users of the Copernicus Climate Data Store). Right now, it is basically the same as the Vtable.ECMWF. I made some minor changes/comment additions, but that should not change what is extracted during the Ungrib process, or what is used by Metgrid. I made the new Vtable because of the changes to ECMWF mentioned by Ming in the comment above. Once the changes within Ungrib are made, the Vtable.ERA5 and Vtable.ECMWF will be different.
I will make a pull request on GitHub for this Vtable.ERA5 shortly, but I will post it here for now. It is posted here as Vtable.txt, but will actually be called "Vtable.ERA5".
Josh
I will make a pull request on GitHub for this Vtable.ERA5 shortly, but I will post it here for now. It is posted here as Vtable.txt, but will actually be called "Vtable.ERA5".
Josh
William.Hatheway
Active member
helpful,
But wouldn't it be called the EIFS since the changes are only affecting the IFS forecast system and not the other data?
or
Just modify the existing ECMWF table to use both variables?
Last edited:
The ERA5 data I think is really one of many subsets or products of the ECMWF IFS. I think ERA5 will generally not change, while the latest ECMWF operational product has recently changed as we know it. This table probably should be named Vtable.ERA5.pl, because that is really what it is, as I will mention on Github.
Eventually there will also be ERA6!
William.Hatheway
Active member
Since I only have access to the IFS free data, that's the only one that changed the soil correct? or did the paid version of ecmwf also change the soil?
If it's both I could understand changing the vtables to made something like ECMWF-old and ECMWF-new
I am not very familiar with ECMWF free/paid data but here is some info on the cycles of ECMWF IFS:
Changes to the forecasting system - Forecast User - ECMWF Confluence Wiki
So as you can see, there are many different versions (or cycles) of ECMWF. I think ECMWF-old vs ECMWF-new names could work, as long as there aren't new versions that change the Vtable again.
For example, ERA5 is based on Cycle 41r2. I assume all the cycles up to Cycle 49r1 can use my ERA5 Vtable if that makes sense.
Josh
William.Hatheway
Active member
operational IFSI am not very familiar with ECMWF free/paid data but here is some info on the cycles of ECMWF IFS:
Changes to the forecasting system - Forecast User - ECMWF Confluence Wiki
confluence.ecmwf.int
So as you can see, there are many different versions (or cycles) of ECMWF. I think ECMWF-old vs ECMWF-new names could work, as long as there aren't new versions that change the Vtable again.
For example, ERA5 is based on Cycle 41r2. I assume all the cycles up to Cycle 49r1 can use my ERA5 Vtable if that makes sense.
Josh
ECMWF Data Store (ECPDS)
data.ecmwf.int
When utilizing ERA5 as input data for the Weather Research and Forecasting (WRF) model, I initially downloaded the data from the National Center for Atmospheric Research (NCAR) website (NCAR RDA Dataset d633000). However, I encountered an issue as the data is provided exclusively in netCDF4 format, which is not directly compatible with WRF.
I subsequently discovered that the Copernicus Climate Data Store offers ERA5 data in GRIB format, which is suitable for WRF.
Could you provide guidance on the recommended approach for running WRF with ERA5 data? Specifically, how should one handle the data format and preprocessing steps to ensure compatibility?
I subsequently discovered that the Copernicus Climate Data Store offers ERA5 data in GRIB format, which is suitable for WRF.
Could you provide guidance on the recommended approach for running WRF with ERA5 data? Specifically, how should one handle the data format and preprocessing steps to ensure compatibility?
The Data Engineering and Curation Section (DECS) of NSF NCAR CISL is in the process of phasing out whole file download, subsetting, and campaign storage access to RDA ERA5 Reanalysis in GRIB format (d633000). We encourage users to transition to the NetCDF files produced by DECS for the RDA. For more details, including the NetCDF data inventory, please visit the ERA5 Reanalysis dataset website.
WRF and MPAS users who previously relied on ERA5 in GRIB format for model initialization should transition to using NetCDF-based initialization scripts developed by NSF NCAR MMM. This script can process ERA5 in netCDF format, and produces intermediate files used as input for metgrid.exe.
Please try and let us know if you have any issues.
WRF and MPAS users who previously relied on ERA5 in GRIB format for model initialization should transition to using NetCDF-based initialization scripts developed by NSF NCAR MMM. This script can process ERA5 in netCDF format, and produces intermediate files used as input for metgrid.exe.
Please try and let us know if you have any issues.
William.Hatheway
Active member
With the next update for WPS or WRF could this script be included in the utilities folder?
Hello, might be a bit of a necro-post, but I have recently started running simulations using the ERA5 data again. For some reason, met_em* files had UU, VV, TT and GHT variables in the 2D fields, when using the Vtable.ECMWF, which worked before.
However, when using your table, it worked properly. What made the difference? What changed in the ERA5 data? Many thanks for the table!
EDIT: I may have been premature, it did not work entirely. For some reason, some of the met_em files have all 34 vertical levels for the variables on the pressure levels, while some of them had only 1 vertical level for those variables (UU, VV, TT, GHT and moisture). I am confused as to what is happening here.
Last edited:
I have not looked at this in a while. But the Vtable.ECMWF and Vtable.ERA5 are effectively the same right now. Ultimately, I would expect them to diverge since there will be changes to ECMWF, but there won't be changes to ERA5 as far as I know.
If you are having weird issues, remember that you need to use the same Vtable twice. Use it once to ungrib the single level data, then use it again to ungrib the pressure level data. Then you need to use metgrid in a way so that it will incorporate the single level data, and pressure level data that was made from the ungrib process.
It is important to note that this Vtable has only been tested to work with ERA5 data on pressure levels, not native levels. Pressure level data is easier to obtain from Copernicus than native level data. But I think native level data would give you more levels which could help WRF.
My bad, I resolved the issue (by running ungrib twice, as you mention), but I did not put an edit here to notify that the issue was resolved.
I have a question however - you say that the numerical level data (eta-level, I suppose) is better than the pressure level for initialization purposes? I have to admit that I did not think about it thus far, but have you experience comparing the simulation quality for the two cases?
I have never tried running WRF with the ERA5 native level data.
The ERA5 pressure level data has under 40 vertical levels, while the native level data has over 100 vertical levels (at least this is what I read). The native levels also follow the terrain near the surface, and since there are more vertical levels, the vertical resolution is better, especially near the ground.
This means that WRF will be given way more information to help with the boundary layer and near-surface interactions. So while, I haven't tried using native level data, I would expect a better result with WRF. But I have not seen anyone verify this.
The 2 main drawbacks I can think of:
1) The native level data is more difficult and more complicated to download from the Copernicus website. You would need to know what variables to get, and how to download it properly, which I heard takes longer.
2) Once you get the data, you need to likely use a slightly modified Vtable, that can handle the native level data. And/or I think it is a requirement to convert this native level data to pressure levels (now you have over 100 pressure levels instead of under 40), before giving it to WRF. This would also be a bit complicated, but there are some tools to do it.
Good luck!! I want to try to use native ERA5 data in the future, but I just don't have the time right now to figure it all out.
The ERA5 pressure level data has under 40 vertical levels, while the native level data has over 100 vertical levels (at least this is what I read). The native levels also follow the terrain near the surface, and since there are more vertical levels, the vertical resolution is better, especially near the ground.
This means that WRF will be given way more information to help with the boundary layer and near-surface interactions. So while, I haven't tried using native level data, I would expect a better result with WRF. But I have not seen anyone verify this.
The 2 main drawbacks I can think of:
1) The native level data is more difficult and more complicated to download from the Copernicus website. You would need to know what variables to get, and how to download it properly, which I heard takes longer.
2) Once you get the data, you need to likely use a slightly modified Vtable, that can handle the native level data. And/or I think it is a requirement to convert this native level data to pressure levels (now you have over 100 pressure levels instead of under 40), before giving it to WRF. This would also be a bit complicated, but there are some tools to do it.
Good luck!! I want to try to use native ERA5 data in the future, but I just don't have the time right now to figure it all out.