(RESOLVED) Horizontal shifting when interpolating WRF file with NDOWN.exe (compared to CDO remapbil)

[Arty]

'Ia ora na,

I've used a combination of CDO et NCO features to remap some wrfout file to wrfinput (see script below). After I compared CDO remapped file with the wrfinput from ARW feature "ndown.exe", it appeared via NCVIEW that there's a horizontal shift (zonal & meridional) between both files ; also field values are very alike, but not exactly the same (as indicated by min/max, and visually sometimes).
Please see following screenshots for PSFC (unstaggered), U10 (x-staggerd) and V10 (y-staggered). Also note that the same shift can be oserved from 3D-space variables.

I also checked the namelist.input files for both config (to be sure) : they're identical (except for the start-end dates).

Has anyone got an idea of what went wrong and where ? I suspect the problem comes from CDO being unable to handle WRF netCDF files staggered grids - hence the script below dealing with each grid one-by-one.

I could upload the WRF files I used if necessary.

Thank you

# DEFINE INPUTS

# USUALLY $1 = wrfinput_d01_2005_01 (or d02)
# USUALLY $2 = wrfoutcyril

# SELECT FIRST TIMESTEP FROM CYRILS'S WRFOUT

if [ ! -f wrfout ]
then
    ncks -O -d Time,0 $2 wrfout
fi
# WRFOUT (1 timestep) ACCORDING TO T,U,V,W

ncks -v Times,MU,MUB,P,P00,PB,PSFC,P_STRAT,Q2,QCLOUD,QGRAUP,QICE,QRAIN,QSNOW,QVAPOR,SST,T,T00,T2,TH2,TSK,U10,V10,PH,PHB,W wrfout wrfout_T
ncks -v Times,U wrfout wrfout_U
ncks -v Times,V wrfout wrfout_V

#  ACCORDING TO T,U,V,W

cp $1 wrfin

ncks -v Times,ALBBCK,CANWAT,CF1,CF2,CF3,CFN,CFN1,CLAT,COSALPHA,DN,DNW,DTBC,DTS,DTSEPS,E,F,FCX,FNDALBSI,FNDICEDEPTH,FNDSNOWH,FNDSNOWSI,FNDSOILW,FNM,FNP,FRC_URB2D,GCX,HGT,ISLTYP,IVGTYP,LAI,LAKE_DEPTH,LAKE_DEPTH_FLAG,LAKEFLAG,LAKEMASK,LANDMASK,LU_INDEX,MAPFAC_M,MAPFAC_MX,MAPFAC_MY,P_HYD,P_TOP,QV_BASE,RDN,RDNW,RDX,RDY,RESM,SAVE_TOPO_FROM_REAL,SEAICE,SHDMAX,SHDMIN,SINALPHA,SNOALB,SNOW,SNOWC,SNOWH,SR,STEP_NUMBER,T_BASE,T_INIT,TISO,TLP,TLP_STRAT,TMN,TOPOSLPX,TOPOSLPY,U_BASE,U_FRAME,UOCE,VAR,VAR_SSO,V_BASE,VEGFRA,V_FRAME,VOCE,XLAND,XLAT,XLONG,Z_BASE,ZETATOP,ZNU,CPLMASK,DZS,LANDUSEF,SMCREL,SMOIS,SOILCBOT,SOILCTOP,TSLB,ZNW,ZS wrfin wrfin_T
ncks -v Times,MAPFAC_U,MAPFAC_UX,MAPFAC_UY,XLAT_U,XLONG_U wrfin wrfin_U
ncks -v Times,MAPFAC_V,MAPFAC_VX,MAPFAC_VY,MF_VX_INV,XLAT_V,XLONG_V wrfin wrfin_V

# GRID DESCRIPTION

cdo griddes wrfin_T > grid_T.txt
cdo griddes wrfin_U > grid_U.txt
cdo griddes wrfin_V > grid_V.txt

# REMAP (BILINEAR)

cdo remapbil,grid_T.txt wrfout_T wrfout_T_remap
cdo remapbil,grid_U.txt wrfout_U wrfout_U_remap
cdo remapbil,grid_V.txt wrfout_V wrfout_V_remap

# CONCATENATE ALL SUB-FILES

cp wrfout_T_remap wrfout_remap

ncks -A wrfout_U_remap wrfout_remap
ncks -A wrfout_V_remap wrfout_remap

# REPLACE TIME VARIABLE TYPE AND TIME COORDINATE BY WRFIN

ncrename -d Times,Time wrfout_remap
ncks -O -s 'Times=char(Times)' wrfout_remap wrfout_remap
ncks -O -x -v Times wrfout_remap wrfout_remap
ncks -A -v Times wrfin wrfout_remap

# CHANGE XLAT* & XLONG* ATTRIBUTES

ncks -C -x -v XLONG,XLAT,XLONG_U,XLAT_U,XLONG_V,XLAT_V wrfout_remap wrfout_remap_xlonglat_ok
ncks -A -v XLONG,XLAT,XLONG_U,XLAT_U,XLONG_V,XLAT_V,HGT wrfin wrfout_remap_xlonglat_ok

# APPEND ALL REMAPPED VARIABLES TO WRFINPUT

#cp wrfin control_0

ncks -A -C -v MU wrfout_remap_xlonglat_ok wrfin
ncks -A -C -v MUB wrfout_remap_xlonglat_ok wrfin
ncks -A -C -v P wrfout_remap_xlonglat_ok wrfin

(ans so on... because I had issue with several variables I had to test each one)

# RENAME PROPERLY ACCORDING TO INPUT FILE

name="$(basename -- $1)"
cp wrfin $name

FYI : same post has been posted on CDO Forum for the local team to llok at : Horizontal shifting when interpolating WRF file with CDO remapbil (compared to ndown.exe) - CDO - Project Management Service

 

[Arty]

After further investigation, it would appear the problem comes from ndown and the shift is zonal only ; please see screenshots attached below.

 

[Arty]

Even though I did find a work-around, I'd be glad to understand what could have been wrong with NDOWN as the evidence suggest there's a shift caused by the tool.

 

[Arty]

I finally did find the origin of the shift ; even though I don't fully understand why the i_start I used in WPS to create a dummy wrfinput_d02 (= wrfndi_d02) does not give the same domain as when used within NDOWN namelist.

Could someone confirm if I want to create a dummy d02 from a given d01 :

1) That the domain size should be taken from " END_STAG " , i.e. in my case 441x166 :

nch wrfout_d01_2016-01-27.nc | grep PATCH
:WEST-EAST_PATCH_START_UNSTAG = 1 ;
:WEST-EAST_PATCH_END_UNSTAG = 440 ;
:WEST-EAST_PATCH_START_STAG = 1 ;
:WEST-EAST_PATCH_END_STAG = 441 ;
:SOUTH-NORTH_PATCH_START_UNSTAG = 1 ;
:SOUTH-NORTH_PATCH_END_UNSTAG = 165 ;
:SOUTH-NORTH_PATCH_START_STAG = 1 ;
:SOUTH-NORTH_PATCH_END_STAG = 166 ;
:BOTTOM-TOP_PATCH_START_UNSTAG = 1 ;
:BOTTOM-TOP_PATCH_END_UNSTAG = 32 ;
:BOTTOM-TOP_PATCH_START_STAG = 1 ;
:BOTTOM-TOP_PATCH_END_STAG = 33 ;

2) That I must use CEN_LON/LAT :

nch wrfout_d01_2016-01-27.nc | grep CEN
:CEN_LAT = -17.90401f ;
:CEN_LON = -172.7851f ;

3) Then define i_start and j_start and x/y sizes to determine my d02 ?

I followed these steps and found i_start and j_start to be 318;65 (x=y=41) to define my region of interest (d02) ; but it appeared, when running NDOWN, that the i_start I would have needed to center my domain on the main island of interest was more 319-320.

I've got further tests to do on that matter. I'll post my conclusion then.

 

[Ming Chen]

When you run NCO commands to remap the data on the the fine-resolution domain, did you consider the map projection information?

Note that when we run ndown, the fine-relation domain shares the same map projection information used in its parent domain. When we remap the parent domain data to the fine-resolution domain, we need to be cautious because different map projection used may lead to slight difference, just as you have seen in your case.

 

[Arty]

When you run NCO commands to remap the data on the the fine-resolution domain, did you consider the map projection information?

Note that when we run ndown, the fine-relation domain shares the same map projection information used in its parent domain. When we remap the parent domain data to the fine-resolution domain, we need to be cautious because different map projection used may lead to slight difference, just as you have seen in your case.

Hello Ming,

I had to use a combination of CDO and NCO to remap input data (wrflowinp and wrfinput) because unfortunately NDOWN doesn't account for surface variables (in particular SST for my case - see HERE) and I thought (but I was wrong) that wrfinput I got from NDOWN were corruptedly East-shifted (see HERE).
So, in short : I used CDO for remapping, based on the d02 grid I defined with WPS, itself based on the d01 grid it is nested in (I had to also use NCO for other stuff to get my input file compliant with WRF).

To explain more clearly the process I followed :

1) Get the information of d01 (grid size, center lat/lon...)
2) Define a namelist with those information to create an identical d01 ; add a nested d02 domain using i_start / j_start (ratio and all...) to the namelist
3) Iterate 2-3 times the geogrid step until finding the right i_start / j_start so that my d02 be centered as wanted
4) Create wrfinputs for d01 and d02, forced by CFSR, using REAL (this step provide a "dummy" wrfinput_d02 I use as wrfndi_d02 for NDOWN)
----- SO : at this step, the wrfinput_d02/wrfndi_d02 are the same file and are well centered ----
5) Use NDOWN with wrfndi_d02 and wrfout_d01* to create new wrfinputs (same namelist as WPS/REAL ; i.e. same i_start / j_start)
6) Get ill-centered wrfinput_d02 (and, presumably, ill-centered wrfbdy*)

See attached REAL and NDOWN namelists.

I also attached ncview screenshots from differents variables for geo_em_d02 ; original wrfinput_d02 ; wrfndi_d02 (identical to wrfninput_d02) and wrfinput_d02 as output from NDOWN.

We can clearly see the shift on T2 between wrfndi_d02 and wrfinput_d02 as calculated by NDOWN. But the namelist are exactly the same between the one used in REAL to create first wrfinput_d02/wrfndi_d02 and the one used in NDOWN to create new wrfinput from wrfout_d01 data.

Feel free to ask for any more details/files you would need to clear that up.

 

[Arty]

In addition, please see the original wrfout_d01 I'm using and differents crops I made out of ncks from i_start = 318 to 321.

Even thouth at 21 km (original wrfout ; d01) resolution, HGT is not as accurate as lower 7 km (d02) resolution, one can see that with i_start = 318, the crop over Tahiti is ill-centered.

And BTW : the projection for all domains is Mercator.

 

[Arty]

I was aware since few months I was using NDOWN in a quite borderline manner, but I think I've finally put my finger on something very interesting, thanks to Ming's comment about projection. The West-East shift I'm having is, indeed, due to Mercator projection (used on all domains I working on/with).

I was using previous simulation results already based on a downscaling, i.e. some wrfout_d02*. I had/have no access to the input files, only the wrfouts. I tried to recreate a identical grid from the information contained in these *d02 files (dx, dy, cen_lon, cen_lat, sizes), but didn't check the coherence of xlats/xlongs* between original file and newly created WRF inputs (forced by CFSR) to be used as grid files. Spoiler alert : the coherence is wrong.

This is because original wrfout_d02* grid is based on a different cen_lon/lat (from the wrfout_d01 it derives from), than the (not so) equivalent-recreated d02 (d00 for me), which grid is based on the cen_lon/lat from wrfout_d02, hence creating slight shifts due to projection method.

I'll wright a more detailed and illustrated post on that matter, but one should know that using NDOWN correctly implies to base one's grid(s) upon the original (coarsest) grid domain even though one's working with nested domain data ; i.e. grids should always be based on wrfout_d01* if - as in my case - one is using wrfout_d02* as forcing inputs.
I had to re-create d01 (105 km) and d02 (21 km) original grids to implement d03 (7 km) and d04 (2,333 km) grids, which respectively are my d01 and d02 grids. This is the only way to get grids fitting correctly together, as needed.

The User Guide may benefit from a more detailed chapter about NDOWN and requirements (currently less than 5 pages long, including figures). Moreover, since UPP for WRF is not updated/supported anymore, WRF should really incoporate a more versatile tool to make the use of wrfouts as input more convenient : WPS, for example, would really benefit to be able to use wrfout files as input - or maybe an upgrade of NDOWN ?

Nevertheless, thanks for the help and support since the beginning.

 

[Ming Chen]

Arty,

Thank you for the update. As I stated previously, the slight shift is caused by difference in map projection.
We will see whether we can update the user's guide to further clarify the issue.

 

[Arty]

Arty,

Thank you for the update. As I stated previously, the slight shift is caused by difference in map projection.
We will see whether we can update the user's guide to further clarify the issue.

Indeed, that helped a lot. I wish this point had been brought to my attention earlier ; nevertheless I'm glad having overcome such a significant issue.

Moreover, since UPP for WRF is not updated/supported anymore, WRF should really incoporate a more versatile tool to make the use of wrfouts as input more convenient : WPS, for example, would really benefit to be able to use wrfout files as input

If I may discuss areas of improvement :

1. For a potential WPS upgrade, managing "wrfout" files as inputs would be more suitable than using NDOWN. This approach allows for a more flexible vertical refinement.
2. NDOWN could also benefit from including wrflowinp in case SST or other surface variables are important. However, if the first point is satisfied (using WPS to manage "wrfout" files), then there's no longer a need for NDOWN.