Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

How to process ERA5 invariant data? Issue with LANDSEA mask; and a metgrid error.

This post was from a previous version of the WRF&MPAS-A Support Forum. New replies have been disabled and if you have follow up questions related to this post, then please start a new thread from the forum home page.

tshepherd

New member
Hello,

There seems to be sparse information on formal step-by-step instructions for processing ERA5 data in WPS, which is a problem when issues are encountered. I have used ERA5 in WPS, so am aware of the steps required. I have not, however, used ERA5 without some form of external SST data - and this is where I am having some issues due to the masking from ERA5 being retained (as opposed to specific masking being used from SST datasets).

Without separate SST data, I was seeing problems with the output of the SST field in metgrid, identical to that described here: https://forum.mmm.ucar.edu/phpBB3/viewtopic.php?t=8964

I have since fixed the SST field by changing the METGRID.TBL file to read:

name=SST
interp_option=wt_average_16pt
fill_missing=0.
missing_value=-1.E30
flag_in_output=FLAG_SST
interp_mask = LANDSEA(1)
masked=land

But the issue remains for SKINTEMP in the metgrid files.
This is the resulting LANDSEA mask:
Screenshot 2020-12-20 18.44.50.png

And this is what SKINTEMP looks like as a result of that mask:
Screenshot 2020-12-20 18.44.42.png

My WRF input and output looks OK for the variables I am outputting (including T2 etc.). When I check the restart files, T2, TSK etc. all look fine, but I want to make sure I am following best practices, as I am not convinced metgrid is giving me the right result due to the masking.

I have attempted to follow the instructions given here for using the static data:
https://forum.mmm.ucar.edu/phpBB3/viewtopic.php?f=75&t=9714&p=19000&hilit=ERA5#p19000
http://climate-cms.wikis.unsw.edu.au/How_to_run_WRF_with_ERA-Interim_data

But, I am getting an error in metgrid (see below).

The ERA5 static data file I am using is:
e5.oper.invariant.128_172_lsm.ll025sc.1979010100_1979010100.grb

Steps:

1. I run ungrib for ERA5 *.pl and *.sl data (=FILE in namelist.wps) using Vtable.ERA-interim.pl (this is for the specific meteorological dates)

2. I then run ungrib for the e5.oper.invariant.* file above (=INV* namelist.wps) for the specific fixed date 1979/01/01 00:00:00, using the following Vtable information:

GRIB | Level| Level| Level| metgrid | metgrid | metgrid |
Code | Code | 1 | 2 | Name | Units | Description |
-----+------+------+------+----------+---------+-----------------------------------------+
172 | 1 | 0 | | LANDSEA | 0/1 Flag| Land/Sea flag |
-----+------+------+------+----------+---------+-----------------------------------------+

3. I then run metgrid for the specific meteorological dates using the following addition to &metgrid:

&metgrid
fg_name = 'FILE',
io_form_metgrid = 2,
constants_name = 'INV:1979-01-01_00'


But, as indicated, the following error occurs when running metgrid:

Processing domain 1 of 3
INV:1979-01-01_00
Processing 1988-07-03_18
FILE
ERROR: Cannot combine time-independent data with time-dependent data for field LANDSEA.mask

I would hope someone out there has encountered this and has a solution. Also, I hope someone can clarify the steps for running WPS with ERA5 static data (in case I am wrong in my approach), because the forum posts I have mentioned above don't outline the steps involved for using the static data.

Thank you!

Tristan
 
Hi Tristan,
It looks like the invariant file should also include SOILHGT information. I'm attaching the invariant file we use so that you can try skipping the ungrib process for that particular file. Just change the attached file name to ERA5_INVARIANT (removing the .txt suffix that was necessary to attach the file here) and set constants_name = 'ERA5_INVARIANT' in the &metgrid record of namelist.wps, and then run metgrid again. Let me know if this helps your problem at all. Thanks!
 

Attachments

  • ERA5_INVARIANT.txt
    7.9 MB · Views: 155
Hello,

Thanks very much for your reply, and for the file.

I ran through this again using the file and the namelist entry you suggested. Unfortunately the issue remains:

Processing domain 1 of 3
ERA5_INVARIANT
Processing 1985-11-27_18
FILE
ERROR: Cannot combine time-independent data with time-dependent data for field LANDSEA.mask

I see that this issue has been recently reported before for an unrelated issue, but no solution is currently found there either.
https://forum.mmm.ucar.edu/phpBB3/viewtopic.php?t=9049

A few questions:

1. Are there any pre-processing steps in the ungrib process that you need to undertake on the ERA5 surface data so that metgrid runs without this error (e.g. remove LANDSEA from the ERA5 data etc.). This seems unlikely to be the cause because it's a date issue, but you never know - there may be a conflict because of two masks?

2. Is the ERA5 data available at Copernicus e.g. https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-pressure-levels?tab=overview the same as that provided on the NCAR site https://rda.ucar.edu/datasets/ds633.0/? I have downloaded my ERA5 from the Copernicus climate data store.

3. Are there any changes required to METGRID.TBL for any of the mask fields? I have attached the table (in txt format) I am currently using. The only change I made was to the SST field.

Cheers,
Tristan
 

Attachments

  • METGRID.txt
    26.1 KB · Views: 75
OK, so after digging into this a little more, the error "ERROR: Cannot combine time-independent data with time-dependent data for field LANDSEA.mask" goes away when using ERA5 data obtained from NCAR. So there must be a compatibility issue with the ERA5 data from Copernicus CDS when used with the ERA5 invariant file sourced from NCAR.

Despite the error being resolved, this still doesn't resolve the LANDSEA mask issue.

In the met files, this is the resulting LANDSEA mask:

Screenshot 2021-01-05 11.36.10.png

And this is SKINTEMP (NB: the wrfinput file result is OK as previously reported, but this is still unsettling):

Screenshot 2021-01-05 11.36.24.png

As another test, I changed the interp option in METGRID.TBL for LANDSEA from nearest neighbor to 4pt weighted average, and this is the result:

Screenshot 2021-01-05 11.28.52.png

So, it's not clear to me how to obtain higher resolution coastline for the LANDSEA field when using ERA5, or whether it even matters. As I said in my first post, by updating the SST field in METGRID.TBL, I was able to correct the mask issue for the SST field. That is currently:

name=SST
interp_option=wt_average_16pt
fill_missing=0.
missing_value=-1.E30
flag_in_output=FLAG_SST
interp_mask = LANDSEA(1)
masked=land
 
Hi,
I ran a test with the invariant data, along with ERA5 surface and pressure level data, and tried to create a domain that was close to yours. My LANDSEA mask looks exactly like yours, and I think that's just due to the resolution of what's given to us from ERA. I'm not having any trouble with the SKINTEMP field. However, I'm also not using an external SST field, and I haven't modified my METGRID.TBL at all. Can you attach the following so that I can run a test with your information?
  • namelist.wps
    METGRID.TBL
    Vtable you are using for ERA and for SST
    a single time of your SST data

Please also let me know which version of WPS you are using. Thanks!
 
Hello,
So it's taken me a while to get onto this again as I was running simulations. Hopefully you're still able to take a look please.

Just to clarify, I'm not using an external SST dataset in this case. The SSTs come from ERA5.

I do think there is some issue with the input data. Even though I have successfully run my simulations on an HPC system using Intel, on another cloud-based system using WRF compiled with GNU, I am observing some strange behavior. This latter issue is not the same as what I have described in the previous posts, but I do think it's too much of a coincidence not to be related.

On the cloud system, GNU-compiled WRF, I cannot run this simulation with sea ice fraction turned on. It will only run if fractional_seaice =0. This is despite sea ice being present in the met files. There is no issue with running sea ice on the HPC intel-compiled WRF.
Running WRF in debug mode, I was able to trace the error back to the surface layer scheme sea ice module, specifcally the psim_unstable function:

Code:
function psim_unstable(zolf)
      integer :: nzol
      real    :: rzol
        nzol = int(-zolf*100.)
        rzol = -zolf*100. - nzol
        if(nzol+1 .le. 1000)then
           psim_unstable = psim_unstab(nzol) + rzol*(psim_unstab(nzol+1)-psim_unstab(nzol))
        else
           psim_unstable = psim_unstable_full(zolf)
        endif
      return
      end function


Code:
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0  0x7f16e1c26d01 in ???
#1  0x7f16e1c25ed5 in ???
#2  0x7f16e18cb20f in ???
#3  0x55a1c38ef955 in __module_sf_sfclayrev_MOD_psim_unstable
        at /wrf/WRFV3/phys/module_sf_sfclayrev.f90:1200
#4  0x55a1c38f42b1 in __module_sf_sfclayrev_MOD_sfclayrev1d
        at /wrf/WRFV3/phys/module_sf_sfclayrev.f90:666
#5  0x55a1c38faf55 in __module_sf_sfclayrev_MOD_sfclayrev
        at /wrf/WRFV3/phys/module_sf_sfclayrev.f90:262
#6  0x55a1c32714ac in __module_surface_driver_MOD_sfclayrev_seaice_wrapper
        at /wrf/WRFV3/phys/module_surface_driver.f90:5421
#7  0x55a1c32b8f43 in __module_surface_driver_MOD_surface_driver
        at /wrf/WRFV3/phys/module_surface_driver.f90:1765
#8  0x55a1c2c086a6 in __module_first_rk_step_part1_MOD_first_rk_step_part1
        at /wrf/WRFV3/dyn_em/module_first_rk_step_part1.f90:518
#9  0x55a1c276baa8 in solve_em_
        at /wrf/WRFV3/dyn_em/solve_em.f90:940
#10  0x55a1c269cf6f in solve_interface_
        at /wrf/WRFV3/share/solve_interface.f90:157
#11  0x55a1c1a9da71 in __module_integrate_MOD_integrate
        at /wrf/WRFV3/frame/module_integrate.f90:348
#12  0x55a1c1a3f395 in __module_wrf_top_MOD_wrf_run
        at ../main/module_wrf_top.f90:343
#13  0x55a1c1a3e3d0 in wrf
        at /wrf/WRFV3/main/wrf.f90:50
#14  0x55a1c1a3e427 in main
        at /wrf/WRFV3/main/wrf.f90:28


So, I think the two are related, and if the input data can be corrected, this new issue may resolve itself. I am certainly interested in the fact your SKINTEMP field looks OK. That may be a clue that something is wrong here.

I have attached my namelist.wps, METGRID.TBL, and the Vtable I used to generate these met files and subsequent WRF input files (NB: I have had to change the file extensions to upload them). Just to confirm again, there is no external SST data. I am using WPS v3.8.1 (and WRF 3.8.1) - our work is longstanding with this version and we maintain it for consistency. Also FYI, as I reported in a previous post, the SST field in my METGRID.TBL has been modified to 'fix' the masking issue with the SST field. That may have inadvertently introduced the memory reference problem...

Thanks!
 

Attachments

  • METGRID.TBL
    26.1 KB · Views: 76
  • Vtable.txt
    3.5 KB · Views: 71
  • namelist_wps.txt
    1,013 bytes · Views: 84
Thanks for sending those. Looking at your namelist.wps file, I see that you have set:
Code:
geog_data_res     = '5m+default', 'usgs30s+default', 'usgs30s+default',

The default setting is MODIS data, and therefore if it's having to use any of the 'default' fields, it's mixing USGS and MODIS data, which have a different number of landuse types, and different values for them. This can certainly cause problems if that's the case. Will you try this again, choosing either default (MODIS) or USGS landuse types and see if that makes any difference? Thanks!
 
Hello,

Thanks for the advice on removing the 'default' setting from the namelist. That will help to avoid any potential conflicts. The memory reference error I reported is a bit of a red herring, as it relates to a system environment problem. But, back to my original issue:

Even after removing the 'default' setting, I'm not seeing any observable improvement in the SKINTEMP field in the met files. So, I am still interested in the fact that your SKINTEMP field is OK, and why mine differs. Is there a setting in METGRID that differs, or a Vtable setting that is different? Could it be that because you are able to run the ERA5_INVARIANT file that SKINTEMP resolves OK?

I do think there is some conflicting mask issue occurring here, but I'm yet to find the root cause. It looks to me like the resolution from LANDMASK is being applied to the SKINTEMP field, but the resolution from LANDSEA (in ERA5) is then overlain or interfering with the resolved field, because the coarse resolution of this mask is penetrating into the SKINTEMP field. (See the images previously posted for this).

So, just to confirm, what are your settings in METGRID.TBL for SKINTEMP and LANDSEA - do they match mine:

Code:
name=SKINTEMP
        interp_option=sixteen_pt+four_pt+wt_average_4pt+wt_average_16pt+search
        masked=both
        interp_land_mask  = LANDSEA(1)
        interp_water_mask = LANDSEA(0)
        fill_missing=0.

Code:
name=LANDSEA
        interp_option=nearest_neighbor
        fill_missing=-1.
        fill_lev=200100:LANDMASK(1)

As I said previously, I was able to resolve the SST mask issue by making a modification to METGRID.TBL for the SST field. That fixed the coarse resolution for that field and applied the LANDMASK from geogrid. The data looks fine for SST, so I doubt it is a data problem in the SKINTEMP field, but the mask being applied.

Also, just to reiterate my Vtable settings for those fields:

Code:
 172 |  1   |   0  |      | LANDSEA  | 0/1 Flag | Land/Sea flag                            |
 235 |  1   |   0  |      | SKINTEMP | K        | Sea-Surface Temperature                  |
  34 |  1   |   0  |      | SST      | K        | Sea-Surface Temperature


Any suggestions on how to fix SKINTEMP?

Cheers,
Tristan
 
Hi Tristan,
I have not been able to find a way to repeat your problem with SKINTEMP. I will list everything I did, and include the files I can.
I'm using unmodified WPSV3.8.1.
  • I'm using the namelist.wps file you sent (removing 'default' from the geog_data_path, and adding the ERA_INVARIANT file to the metgrid section) - attached.
  • I run geogrid. I'm attaching the GEOGRID.TBL I'm using.
  • I'm using the ERA5 pressure level and surface analysis for 1979-10-25 to 1979-10-31. I'm attaching a screenshot, showing the 2 types of data I downloaded for this - the top 2 rows on the data page. The data come from NCAR, and they are packaged as a full month - several different files for different variables. I'm also attaching a screenshot showing the exact files I have.
  • I then link in all the files at one time (./link_grib.csh ../data/e5) and then link to the Vtable, which I'm attaching.
  • I run ungrib.exe and get FILE:* files.
  • I then ensure that the ERA5_INVARIANT file (attached) is in the WPS directory, and make sure it's also listed in the &metgrid section of the namelist (attached) and run metgrid. I've tried this by linking to your METGRID.TBL and to the default table. I'll attach the version of yours that I've used to make sure they are still the same.
No matter what I do, I cannot see any problems with SKINTEMP. I'm not sure where to go from here. I would recommend you following the exact same procedures I have, using the exact same files and data, and make sure your WPS is not modified in any sort of way. If you still get the problems, I'm at a loss and will have to ask around to see if anyone else has any clues.
 

Attachments

  • wrfhelp_files.tar
    8.6 MB · Views: 86
I'm having the same problem as you, Tristan, with the blocky SST along the coast. I'm also using ERA5 from Copernicus. First, I find it interesting that in the standard METGRID.TBL file (for ARW), there are many fields that get masked, and they all make sense based on whether they're defined over land or water. But, SKINTEMP is unique in that it does both maskings. This would imply that its values are considered physically meaningful, and are used, over both land and water and so both sides of the coastline need care. To be honest, I've never been quite sure what SKINTEMP is even used for, as it seems redundant with SST over water and top layer soil temperature over land. It's clear that it's not being substituted for SST in my case, because then I wouldn't be having the coastline problem. On the other hand, it is really interesting that SST, while seeing like an obvious candidate for land masking, is NOT MASKED in the standard METGRID.TBL. I think the reason for this may be (though haven't confirmed) that until now, all gridded data sources that WRF has used (GFS, ECMWF, various reanalyses) have included some sort of physically reasonable filled-in values of SST over land, so there was no compelling reason to mask it. Then came along ERA5, and it apparently has missing values over land that METGRID seems to be interpretting as physically real, so where ERA5 says "land" and WRF says "water", you get wacky results. And it's even possible that this state of affairs is unique to Copernicus-downloaded ERA5 data and not NCAR-downloaded ERA5 data, which perhaps explains why only Copernicus ERA5 data is causing this problem. What I've done is simply apply the same masking to SST as is done for SKINTEMP in METGRID.TBL. This produces good SST at all fine-scale offshore points. It does whacky things over land but I don't think that matters. Anyway, the simulations look reasonable. Could have also just done land masking, which would have produced the same result over water. I'm curious if what I'm guessing is true, about previous SST grids always having physically plausible filled in land values, including ERA5 from NCAR. If that's not the case, I don't get why METGRID.TBL has, for many WRF versions in a row now, NOT done masking on the SST field, because it seems like it would cause problems without masking. Unfortunately I don't have time to investigate at the moment. PS Thought just occurred, might also be a matter of whether missing values are used in the source data, and whether those are properly recognized by metgrid.
 
Top