Error while running 60 km mesh

[518er2005]

Hello everyone, hope everyone is doing good in these hard times. I am trying this mail because I have come up with an error while running the atmosphere model usimg 60-km quasi uniform mesh. The model runs but after some time it terminated with an error showing " program received signal sigsegv : segmentation fault - invalid memory referenve." I had given 12 process for mpi to run. So please if anyone can help me in resolving this issue it will be much appreciated. Below I am attaching the error i got, the error log file along with name list.
With regards,
Debashis

 

[mcurry]

The error in log.atmosphere.0000.err-3.txt is mostly likely not the cause of the segmentation fault that's occurring. That error occurs when a file that MPAS is going to write is present (perhaps from a previous run). By default, the model does not overwrite these files and produces this error. This error is not a critical error, and thus does not halt the model.

As the error log says, you can add `clobber_mode` to the history stream if you wish to overwrite any file that may be there from another run. See section 5.2 Optional stream attributes of the MPAS Users Guide for more information and additional options for clobber_mode.

At this time I'm not exactly sure what is causing the segmentation fault that's occurring. Can you upload your log.atmosphere.0000.out and your streams.atmosphere file?

As well, I have noticed that for config_block_decomp_file_prefix you have set: x4.163842.graph.info.part. This is not the 60 km quasi uniform grid, but the 92-25 km variable resolution mesh. Did you mean to use the 60 km mesh instead? (For reference, the 60 km quasi-uniform mesh filename is x1.163842.grid.nc).

If you are running on the x4.163842.grid.nc by mistake, then your namelist values for config_dt and config_len_disp will be set incorrectly and could cause a segmentation fault as the model would probably become unstable if it ran with the 60 km values for those namelist options.

Do you want to ensure that you are using the correct mesh? The download page for MPAS meshes are here: http://mpas-dev.github.io/atmosphere/atmosphere_meshes.html (in case you don't have it).

 

[518er2005]

Sorry for the late reply. Anyways thanks for pointing out the mistake, yes you are right the config_block_decomp_file_prefix was wrongly given. I corrected it and the model is running now properly. I have one more query. I am trying to run again with a different mesh, this time with a variable resolution mesh of 60-15 km. I want to do it over the Arabian sea. I did generate the grid data and plotted the mesh to see whether my grid is correct or not. So can you please look into it and say whether I have done it in a correct way or it's wrong. I am attaching the mesh_resolution figure along with the namelist_input.
With regards,
Debashis

 

[mgduda]

The plot of your variable resolution mesh looks fine to me. It may be worth pointing out that you may not need to set the rotation of the mesh as 180 degrees in your namelist.input file:

config_birdseye_rotation_counter_clockwise_degrees = 180

Rotating an elliptical region by 180 degrees about its center will give the same elliptical region.

 

[518er2005]

Thanks for the suggestion. I did it accordingly. It may sound naive but I want to know can we use variable resolution meshes for limited area simulations, will the process be similar to how we do for quasi-uniform meshes??

 

[mgduda]

That's a good question (not naive at all!). Indeed, you can use variable-resolution meshes for limited-area simulations. The one point about this that comes to my mind is that you'll need to ensure that config_len_disp is set to the smallest grid distance in the limited-area subset of the variable-resolution mesh, and that config_dt is set proportional to this smallest grid distance. In most cases, this means setting config_len_disp and config_dt according to the smallest grid distance in the variable-resolution mesh (assuming that the limited-area subset of the mesh includes the high-resolution region). But, in principle, if you were to, say, create a subset of the 60-15 km variable-resolution mesh that only included cells with distances ranging from 60 km to 48 km (i.e., you didn't use the high-resolution part of the 60-15 km mesh for some reason), you would set config_len_disp and config_dt based on a 48-km grid distance.

 

[518er2005]

Thanks, Michael for the reply. I did it accordingly and got the mesh as I wanted. But while running the atmosphere model it got terminated with an error saying " YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Killed (signal 9). This typically refers to a problem with your application. Please see the FAQ page for debugging suggestions ". My first guess is, it might be because of memory allocation. MY desktop is having 16 GB of ram with an 8 core processor. So can you suggest to me a way of running the atmosphere model without any error??

With regards,
Debashis

 

[mgduda]

As discussed in this thread, you can estimate the memory that will be required for a given simulation as 0.175 MB/grid column. With 163842 columns in the mesh, you may need around 29 GB of memory. If you have only 16 GB of memory, I think you'll need to find a way to run with fewer grid columns -- either by using a different (and most likely, coarser) global mesh, or by cutting down the size of your limited-area domain.