error with generating limited regional domain using create_region executable

[jpen]

Following error when using MPAS_LimitedArea tool for limited area simulation. The x1.5898242.grid.nc is used for generating static fields with x1.5898242.cvt.part.64 decomposition file. The create_region executable works with low resolution grid (240 km). What could be the reason for the failure with high resolution fields? Please advise how to resolve this error.
File SA.pts

Name: LAS
Type: ellipse
Point: -23.8, -53.4
Semi-major-axis: 160000 # Meters
Semi-minor-axis: 120000 # Meters
Orientation-angle: 45

./create_region SA.pts x1.5898242.static.nc

Traceback (most recent call last):
File "/prj/cptec/jayant.pendharkar/Models/RegModel/MPAS-Limited-Area/./create_region", line 55, in <module>
regional_area = LimitedArea(args.grid,
File "/prj/cptec/jayant.pendharkar/Models/RegModel/MPAS-Limited-Area/limited_area/limited_area.py", line 50, in __init__
self.mesh = MeshHandler(mesh_file, 'r', *args, **kwargs)
File "/prj/cptec/jayant.pendharkar/Models/RegModel/MPAS-Limited-Area/limited_area/mesh.py", line 32, in __init__
self._load_vars()
File "/prj/cptec/jayant.pendharkar/Models/RegModel/MPAS-Limited-Area/limited_area/mesh.py", line 101, in _load_vars
self.sphere_radius = self.mesh.sphere_radius
File "src/netCDF4/_netCDF4.pyx", line 3141, in netCDF4._netCDF4.Dataset.__getattr__
File "src/netCDF4/_netCDF4.pyx", line 3082, in netCDF4._netCDF4.Dataset.getncattr
File "src/netCDF4/_netCDF4.pyx", line 1532, in netCDF4._netCDF4._get_att
File "src/netCDF4/_netCDF4.pyx", line 2028, in netCDF4._netCDF4._ensure_nc_success
AttributeError: NetCDF: Attribute not found

 

[mgduda]

I just tried creating a regional static file with your SA.pts contents and an x1.5898242.static.nc file that I created myself. I wasn't able to reproduce the problem, and from your backtrace, it looks as though the issue may be in reading the sphere_radius attribute from your x1.5898242.static.nc file.

Can you post the output of

ncdump -h x1.5898242.static.nc

in this thread?

 

[jpen]

Surprisingly, I could run create_region after exporting proper environment variables. However, I am getting elliptical region as per the pts file but for a box region the desired region is not seen in the plot. Please see the attached images and pts file for the box region. Why is the box region shown in the plot erratic?

Name: SouthAmerica10kms
Type: custom
Point: -36.0, -57.0
12.00, -82.0
12.00, -32.0
-60.0, -82.0
-60.0, -32.0

I proceeded with creating ICs and LBCs for the elliptical region. Now getting segmentation fault when running atmosphere_model. The namelist.atmosphere is attached. Upon debugging following is displayed

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Missing separate debuginfos, use: debuginfo-install glibc-2.17-325.el7_9.x86_64 infinipath-psm-3.3-26_g604758e_open.2.el7.x86_64 libibverbs-50mlnx1-1.49315.x86_64 libnl3-3.2.28-4.el7.x86_64 librdmacm-50mlnx1-1.49315.x86_64 libuuid-2.23.2-65.el
7_9.1.x86_64 lustre-client-2.12.4.6_cray_14_g0a8e54d-1.el7_9.x86_64 munge-libs-0.5.13-1.el7.x86_64 numactl-libs-2.0.12-5.el7.x86_64 slurm-20.11.8-2.el7.x86_64 slurm-libs-20.11.8-2.el7.x86_64 slurm-pmi-20.11.8-2.el7.x86_64 zlib-1.2.7-19.el7_9.x
86_64
Core was generated by `/scratch/cptec/jayant.pendharkar/Sim_ExtrEvts/60-3km_SaoSebastiao_Feb2023/atmos'.
Program terminated with signal 11, Segmentation fault.
#0 0x000000000064631a in taumol16.6257 ()
(gdb)bt
#0 0x000000000064631a in taumol16.6257 ()
#1 0x00000000006553a0 in __rrtmg_sw_taumol_MOD_taumol_sw ()
#2 0x000000000066d9f5 in __rrtmg_sw_spcvmc_MOD_spcvmc_sw ()
#3 0x0000000000679d5c in __rrtmg_sw_rad_MOD_rrtmg_sw ()
#4 0x000000000068275a in __module_ra_rrtmg_sw_MOD_rrtmg_swrad ()
#5 0x0000000000524789 in __mpas_atmphys_driver_radiation_sw_MOD_driver_radiation_sw ()
#6 0x0000000000509343 in __mpas_atmphys_driver_MOD_physics_driver ()
#7 0x00000000004a2b07 in __atm_core_MOD_atm_do_timestep ()
#8 0x00000000004a37b3 in __atm_core_MOD_atm_core_run ()

A quick aside - until the error pertaining to create_region is sorted out, I am using variable mesh "x20.835586.grid.nc" and using grid_rotate changed the center over South America creating a new file SSCentered.grid.nc. My question is whether "x20.835586.graph.info.part.NNN" can be used to run in parallel for creating init.nc file and also while running atmosphere_model?

 

[mgduda]

Thanks for attaching the NetCDF header info. The 'sphere_radius' attribute is definitely there.

Looking around through the MPAS-Limited-Area code, it appears that it tries to open mesh NetCDF file by creating an in-memory dataset; see lines 55 - 59 of mesh.py. Perhaps this could be problematic for larger meshes like the quasi-uniform 10-km mesh?

Would you be willing to try modifying the mesh.py code by replacing these lines beginning at line 55

mesh = open(fname, 'rb') nc_bytes = mesh.read() mesh.close() self.mesh = Dataset(fname, 'r', memory=nc_bytes)

with just this single line that opens the NetCDF file without creating an in-memory copy:

self.mesh = Dataset(fname, 'r')

?

 

[mgduda]

Surprisingly, I could run create_region after exporting proper environment variables.

Following up on my previous post, I think I had an outdated version of this thread loaded in a browser tab and didn't see your reply regarding the environment variables and rectangular region. Could you say exactly which environment variables you needed to export in order to extract the region from your "SA.pts" definition from the quasi-uniform 10-km mesh?

Regarding the rectangular region, you'll need to specify the points in either clockwise or counter-clockwise order.

For your other questions, could you create separate threads to keep this thread focused on the issue of not being able to create regional domains with the MPAS-Limited-Area tool?

 

[jpen]

Thank you! It's working. As per our helpdesk team, in my system the default loads module anaconda 2018 which has some dependency issues while installing netCDF4. Now it loads anaconda 2020.11 by default and is running as of now. I made the indicated changes in mesh.py file. I will post other questions as a separate thread.