Using urban land use fraction (FRC_URB2D) from an external source

[brfr1212]

Hi All,

I'm trying to use my own source of urban land use data by creating a FRC_URB2D field following the process outlined here: https://www2.mmm.ucar.edu/wrf/users/docs/user_guide_v4/v4.2/users_guide_chap3.html#_Creating_an_Urban .

I followed that process, and reviewed a previous post on this topic (https://forum.mmm.ucar.edu/phpBB3/viewtopic.php?f=35&t=10290&p=21498&hilit=FRC_URB2D#p21498) but am still running into issues. geogrid.exe seems to run fine, but the data created in the FRC_URB2D field ranges from large negative numbers to large positive numbers (when I'm expecting values between 0 and 1).

I've attached the namelist.wps, index, GEOGRID.TBL, geogrid.log, and .bil file containing the urban fraction data in the file titled brfr1212_landuse.zip in nextcloud.

Any help on this issues would be greatly appreciated!
Thanks,
Brian

 

[kwerner]

Hi Brian,
Thanks for sending those files. Have you tried to plot the data file to see if anything looks odd after you've followed the process for creating your FRC_URB2D field? Take a look at this post for an example Python script you can use to plot it.

 

[brfr1212]

Hi Kelly,

Thanks for the suggestion. I just tried plotting it in that way, which led to an error because of the dimensions, so I checked the shape of the binary file. It appears to only be 1-dimensional, when it should be 2-dimensional (I think). Using gdalinfo on the .tif file, that file seems to be fine, so I'm guessing something went wrong in the gdal_translate step.

I've attached the original .tif file and .tfw file on nexcloud under Archive.zip in case there is an issue with the original file, or if you could potentially try running gdal_translate to see if you get the same issue?

Thanks,
Brian

 

[brfr1212]

Update: After looking at gdalinfo for the .bil file as well, it seems to have the right dimensions and correct range of data - the error arose from converting it into a numpy array with np.fromfile(). Even though the .bil file seems to have reasonable data though (from 0 to 1), the geo_em files still have large negative and positive numbers for FRC_URB2D. I set up the index file and GEOGRID.TBL in a similar way to what was suggested in previous posts (these files are on nextcloud). Do you have any recommendations on how to proceed?

Thanks,
Brian

 

[kwerner]

Hi Brian,
I apologize for the delay. I looked at this today and plotted your file using another python script I have for plotting geogrid data. What I'm seeing is that the plot shows the world map upside-down. I noticed you're using Cheyenne (based on the path in GEOGRID.TBL). If you're interested in taking a look at my plot script (plot_tile.py) and grabbing the output plot (the *.png file), you can find both in /glade/scratch/kkeene/brfr1212/data. To use the script, you'll need to load python and then run ncar_pylib, i.e.,

> module load python
> ncar_pylib

 

[brfr1212]

Hi Kelly,

No worries. Thanks for sharing the script and plot, I plotted it the same way and found the same thing. When I do gdalinfo on the data.bil file (that's the same file as the on we plotted just with the original name from the conversion), it gives a reasonable min and max of 0 and 1, and a reasonable mean value. Do you know why when we try to plot it or use it in geogrid, the data seems to be so different (with values on the order of 10^9)?

Thanks,
Brian

 

[mgduda]

The directions in the "Creating an Urban Fraction Field from NLCD Data" section of the User's Guide, along with the C code referenced in that section, assume that the urban fraction field is stored as one-byte integer values in the range [0,100]. You can of course use different representations of the urban fraction field, which ultimately ends up as fractional values in the range [0,1], but in that case you'd need to modify the 'index' file for your dataset (and note that in any case, the values read by the geogrid program are expected to be integers).

Do you know how the values for your urban fraction field are stored in the binary file you've prepared (e.g., are they 32-bit floating point values, 16-bit floating-point values, 32-bit integers, etc.)?

 

[brfr1212]

Thanks for the information, that's good to know.

The data type is Float32, and the wordsize=4 in the index file. The values in my binary file should be between 0 and 1 (also from gdalinfo), so would I need to change those into integers somehow?

Thanks,
Brian

 

[mgduda]

Something like the following Python code could be used to convert from 4-byte floats to 1-byte integers:

import numpy as np
a = np.fromfile('data.bin',dtype='f4')
a = a * 100.0
b = a.astype('i1')
b.tofile('data_int.bin')

Note that you'll need to ensure the following is present in your 'index' file:

scale_factor=0.01

This will instruct the geogrid program to scale the integer values that were read from the binary file, and that are in the range [0,100], by a factor of 0.01 to get fractions in the range [0.0,1.0].

 

[brfr1212]

Thanks for the response! I tried that out, and after a few tweaks to the attached plot_tile.py file was able to plot data that was in the correct range and had the correct spatial patterns.

However, when I try to use the data in geogrid, it doesn't seem to be using it. The strange thing is that when I use the old version of the file that has incorrect values, they showed up in the FRC_URB2D variable, but when I use this file with the exact same index and GEOGRID.TBL files, it's all 0's.

I've attached the relevant files below. Any help regarding why this might be happening would be greatly appreciated!

Thanks,
Brian

 

[brfr1212]

Update:

I was able to get the right spatial patterns to show up in the FRC_URB2D variable of the geo_em files using the attached file (screenshots), but the magnitude is off. Initially, I used the gdal.GDT_Int32 data type, but the values were to the power of 7, instead of between 0 and 1 (with wordsize of 4 and scaling factor of 0.01 in the index file). When I changed the data type to gdal.GDT_Float32, the values ranged up to 512.

Since wordsize = 4 corresponds to 4 bytes (32 bits) per data point, I'm not sure why I'm getting these values. Do you have any suggestions?

Thanks,
Brian

 

[brfr1212]

Second update:

I tried using ArcGIS Pro to convert the urban fraction data into integers between 0-100 before using gdal_translate, and this worked! It did end up having a datatype of "byte", so I switched wordsize to 1 in the index file instead of 4.

I noticed in a previous post that it was mentioned that noah-mp uses FRC_URB2D. I'm using the MYNN2.5 PBL scheme, and ETA similarity scheme for surface layer physics, do these use use FRC_URB2D as well, or are there additional changes I need to make to integrate the urban changes into the landusef and lu_index variables?

Thanks,
Brian

 

[Ming Chen]

Brian,
You don't need to change MYNN or SFCLAY codes only because you introduce a new FRC_URB2D data. This is because FRC_URB2D is only used in the urban physics scheme.

 

[brfr1212]

Okay, thanks the help!