Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

ERA5 surface data not found in WPS

mfe_et

New member
Hello,
I have a problem running WPS with ERA5 data. I have downloaded 2 grib files, one for pressure levels (all pressure levels) and the other one for single level data (all variables ecept precipitation type, K index and total totals index). But it seems like metgrid doesn't recognise surface level data, some examples of metgrid.log:
=====
Couldn't find PSFC at level 200100.000000 to fill level 200100.000000 of PRES
Couldn't find SOILT060 at level 200100.000000 to fill level 60.000000 of SOILT
Couldn't find SOILT100 at level 200100.000000 to fill level 100.000000 of SOILT
=====
However it says "Successful completion of metgrid."
But then I run WRF real.exe and there's an error because it needs surface data and is not available in metgrid output's nectcdf files:
=====
Missing surface temp, replaced with closest level, use_surface set to false.
Missing surface RH, replaced with closest level, use_surface set to false.
Missing surface u wind, replaced with closest level, use_surface set to false.
Missing surface v wind, replaced with closest level, use_surface set to false.
ERROR in psfc: flag_psfc = 0, flag_soilhgt = 0, flag_slp = 0, sfcp_to_sfcp = F
-------------- FATAL CALLED ---------------
FATAL CALLED FROM FILE: <stdin> LINE: 1403
not enough info for a p sfc computation
-------------------------------------------
=====
I'm using 4.4 version of WRF and Vtable.ECMWF for ungrib.
You can find the namelist.wps and namelis.input files I'm using.
I don't know if the problem is in ungrib step because surface data is not well processed from the grib file, or in metgrid step... but there must be something wrong with my namelist files, Vtable or grib files.
Please any help would be much much much appreciated.
Thank you!
 

Attachments

  • namelist.wps
    760 bytes · Views: 12
  • namelist.input
    3.8 KB · Views: 6

kwerner

Administrator
Staff member
Do you have soil data in your input files? You need at least 2 levels of soil temperature and soil moisture.
 

mfe_et

New member
Yes I have, at the single level data grib file there are some soil variables:
1676145657838.png
I discovered that I must run ungrib twice, one for pressure level data and a second time for single level data.
The problem is that there is an error running ungrib.exe, when I try to process the single level data grib file (I can't attach it because it's too big 43.5MB).
I have tryed using different Vtables: Vtable.ECMWF_sigma, Vtable.ECMWF, ERA-Interim.ml, ERA-Interim.pl
but all of them give the same output, you can find attached the ungrib.log file, just in case the output helps to identify the problem.
As ungrib can't process single level data, metgrid doesn't find surface data and finally real.exe can't run.
 

Attachments

  • ungrib.log
    2.4 KB · Views: 11
D

Deleted member 3607

Guest
ERA-Interim.ml, ERA-Interim.pl

Those are the vtables for ERA5 that you need to use.

But the log that you are showing says that the input file is tossing this error

ERROR: Data not found: 2022-09-20_09:00:00.0000

Did you download all the ERA5 data as a subset from the RDA database or just the single variable? The vtables most likely need all the data to run ungrib
 

mfe_et

New member
I downloaded ERA5 data using CDS API, I did 2 requests (attached file) because there are 2 types of ERA5 data: reanalysis-era5-pressure-levels and reanalysis-era5-single-levls. I can'ts attach those 2 grib files because they are too big:
+ pressure-levels.grib -> 38.7MB
+ single-level.grib -> 43.5MB
but they contain all the variables available in ERA5 (except precipitation type, k index and total totals index) as you can check in the requests.
I have read that ungrib must be run once for pressure level variables using Vtable.ECMWF and then again only for single level data. But when I run it the second time, using only single-level.grib I get the error you mention "ERROR: Data not found: 2022-09-20_09:00:00.0000".
Which Vtable should I use to run ungrib with single-level.grib? or am I doing any other mistake?
If I run ungrib with both grib files at the same run it only precesses pressure level data and then metgrid has not enough info (Couldn't find XXX at level 200100.000000) and then real.exe fails: "not enough info for a p sfc computation"
 

Attachments

  • cdsapi_ERA5_requests.txt
    12.3 KB · Views: 12

kwerner

Administrator
Staff member
@mfe_et
Can you use the g1print.exe (or g2print.exe, depending whether your data are grib1 or grib2 format) utility to issue the following. From the WPS directory, issue

Code:
./util/g1print.exe <path-to-your-input-data>/<input_data_file> >& g1print.log

Obviously entering your actual path to the data and the actual name of the data file for your surface fields. Then attach the g1print.log file so I can take a look. Thanks!
 

mfe_et

New member
Sure! I have attached two files, using g1print.exe with the pressure levels data file and with the single level data file.
Thank you very much!
 

Attachments

  • g1print_pressurelevels.log
    318.4 KB · Views: 16
  • g1print_singlelevel.log
    139.5 KB · Views: 12

kwerner

Administrator
Staff member
Thanks for sending those. I notice that the dates in your file are all out of order, which we've found to cause issues. Take a look at this post, which addresses this.
 
D

Deleted member 3607

Guest
Thanks for sending those. I notice that the dates in your file are all out of order, which we've found to cause issues. Take a look at this post, which addresses this.
@kwerner @mfe_et

This might be helpful too. Since I am working on ERA5 and finding the same problems.

 

mfe_et

New member
Thank you so much for your replies!!
At the same blog @Whatheway suggests I found this:
Initializing the WRF model with ERA5 (pressure level)
where ERA5 is used instead of ERA-Interim and pressure levels (same as me). So those are the steps I follow to run WRF.
I tried what @kwerner said to create a grib file with ordered data:
grib_copy -B 'dataDate:i asc, dataTime:i asc, validityDate:i asc, validityTime:i asc, shortName asc' download_sl.grib download_sl_reordered.grib
grib_copy -B 'dataDate:i asc, dataTime:i asc, validityDate:i asc, validityTime:i asc, shortName asc' download_pl.grib download_pl_reordered.grib
Then I run g1print.exe to check the results (you can find the files attached).
But I still have the same problem when I run ungib, metgrid and real.exe, the surface data is not processed. I attach ungrib.log, metgrid.log and the error output of real.exe files for you to check them.
Is the problem related to the Fcst hour of the data??
Example:
4 22 1 0 0 2022-09-22_06:00 + 02
is after
3 23 1 0 0 2022-09-22_06:00 + 03
I tried to order by Fcst hour but I didn't get it, and I don't know how to remove that data and take only Fcst hour = +00 data.
 

Attachments

  • rsl.error.0000
    1.5 KB · Views: 1
  • metgrid.log
    512.8 KB · Views: 2
  • ungrib.log
    222.5 KB · Views: 1
  • g1_sl_reordered.log
    139.5 KB · Views: 3
  • g1_pl_reordered.log
    318.4 KB · Views: 2

kwerner

Administrator
Staff member
Thanks for sending those, and I'm glad you were able to easily reorder the GRIB files. I believe the issue now is that your surface fields file is still not quite correctly ordered. The initial dates and times are ordered, but the forecast hour (the "+ x" in the far right column) is still out of order.

Code:
  1  49            1    0    0  2022-09-22_06:00 + 02
   2 145 BLD        1    0    0  2022-09-22_06:00 + 02
   3  23            1    0    0  2022-09-22_06:00 + 03
   4  22            1    0    0  2022-09-22_06:00 + 02
   5   1            1    0    0  2022-09-22_06:00 + 03
   6 143 CP         1    0    0  2022-09-22_06:00 + 02
   7 218            1    0    0  2022-09-22_06:00 + 03
   8 239 CSF        1    0    0  2022-09-22_06:00 + 02
   9 220            1    0    0  2022-09-22_06:00 + 03
  10  17            1    0    0  2022-09-22_06:00 + 03
  11  24            1    0    0  2022-09-22_06:00 + 03
  12  16            1    0    0  2022-09-22_06:00 + 03
  13  15            1    0    0  2022-09-22_06:00 + 03
  14 182 E          1    0    0  2022-09-22_06:00 + 02
  15  44 ES         1    0    0  2022-09-22_06:00 + 02
  16 180 EWSS       1    0    0  2022-09-22_06:00 + 02
  17  21            1    0    0  2022-09-22_06:00 + 02
 

mfe_et

New member
I am sorry for the delay.
I was also thinking that, but I am really not able to order the data correctly, taking into account forecast hour.
I have been searching in different blogs, wikis and users' guides and I can't find how to get it.
I tried typing commands like:
grib_copy -B 'dataDate:i asc, dataTime:i asc, Fcsthour:i asc, validityDate:i asc, validityTime:i asc, shortName asc' download_sl.grib dslr.grib
grib_copy -B 'dataDate:i asc, dataTime:i asc, dataFcsthour:i asc, validityDate:i asc, validityTime:i asc, shortName asc' download_sl.grib dslr.grib
grib_copy -B 'dataDate:i asc, dataTime:i asc, FcstTime:i asc, validityDate:i asc, validityTime:i asc, shortName asc' download_sl.grib dslr.grib
grib_copy -B 'dataDate:i asc, dataTime:i asc, forecastTime:i asc, validityDate:i asc, validityTime:i asc, shortName asc' download_sl.grib dslr.grib
etc.
But nothing works, the grib file is always not correctly ordered when I check it using g1print.exe (example):
----------------------------------------------------
rec GRIB GRIB Lvl Lvl Lvl Time Fcst
Num Code name Code one two hour
----------------------------------------------------
1 49 1 0 0 2022-09-22_06:00 + 06
2 39 1 0 0 2022-09-22_06:00 + 06
3 50 1 0 0 2022-09-22_06:00 + 06
4 40 1 0 0 2022-09-22_06:00 + 06
5 53 1 0 0 2022-09-22_06:00 + 06
6 49 1 0 0 2022-09-22_06:00 + 02
7 33 1 0 0 2022-09-22_06:00 + 06

Can someone help me please??
 

kwerner

Administrator
Staff member
Unfortunately I'm not sure how to manipulate the data either, but perhaps someone else will see this post and will be able to offer some assistance. Let us know if you figure it out.
 

mfe_et

New member
Thank you for your answers.
@Whatheway I downloaded data from cds API, you can find attached a txt file with the queries I did (I can't attach a .py file, but that's exactly the code I run).
This is the page for surface data (the data that I have problems to reorder, and that's why ungrib can't process it): Copernicus Climate Data Store |
And this is for data in pressure levels (the data that ungrib processes correctly): Copernicus Climate Data Store |
 

Attachments

  • cdsapi_ERA5_requests.txt
    12.3 KB · Views: 10
D

Deleted member 3607

Guest
Thank you for your answers.
@Whatheway I downloaded data from cds API, you can find attached a txt file with the queries I did (I can't attach a .py file, but that's exactly the code I run).
This is the page for surface data (the data that I have problems to reorder, and that's why ungrib can't process it): Copernicus Climate Data Store |
And this is for data in pressure levels (the data that ungrib processes correctly): Copernicus Climate Data Store |
@mfe_et @kwerner

Just for trial and error could you please try downloading the ERA5 data from the NCAR RDA database and see if it cannot create met files?


So the steps for using the ERA5 data from NCAR RDA.

Step 1.
link ERA-interim.pl to vtable
link_grib.csh to the pressure files. (make sure the pressure files are in a seperate folder from the surface files)
run ./ungrib.exe

Step 2.
Change wps.namelist
from
&ungrib
out_format = 'WPS',
prefix = 'FILE',
/
to
&ungrib
out_format = 'WPS',
prefix = 'SFILE',
/
then link
cp -f ungrib/Variable_Tables/Vtable.ECMWF ./Vtable
and link the surface files (needs to be in a seperate folder from pressure files) with link_grib.csh
run ungrib.exe

Step 3.
Change wps.namelist again for metgrid to this
&metgrid
fg_name = 'FILE', 'SFILE'
io_form_metgrid = 2,
/
run metgrid.exe

Should work. If not message me and i'll arrange time to help
 

llsalazardom

New member
Hi,
I am trying to run the wrf with the ERA5 data but when I run the ungrib it returns this error
Subroutine DATINT: Interpolating 3-d files to fill in any missing data...
Looking for data at time 2022-06-11_00
ERROR: Data not found: 2022-06-11_00:00:00.0000

Do you know what could be the error?
 

kwerner

Administrator
Staff member
@llsalazardom
Please post your question as a new topic, as your error is different than the one originally posted here. This helps with readability for other users. Thanks!
 

mfe_et

New member
Sorry for the long delay @Whatheway, I've been working on something different and I couldn't try what you suggested. Thank you for your help.
I downloaded ERA5 data from NCAR RDA Database and followed the steps you describe. Apparently it works well for ungrib and metgrib but it crashes when I run real.exe. This time the error message is different:
Using sfcprs to compute psfc
-------------- FATAL CALLED ---------------
FATAL CALLED FROM FILE: <stdin> LINE: 6241
troubles, could not find trapping x locations
-------------------------------------------
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0

You can find attached the output files from ungrib, metgrid and real.exe (real.exe output and error files are compressed because they were too big to attach them).
I tried to find that error in other users' questions at the forum and it seems that they corrected it updating WPS version. I am using WPS-4.4 so I don't know what I should do to solve it.
 

Attachments

  • ungrib.log
    6.6 MB · Views: 2
  • metgrid.log
    318.9 KB · Views: 2
  • rsl.out.zip
    3.8 MB · Views: 1
  • rsl.error.zip
    3.8 MB · Views: 2
Top