MPAS Initialization Failure with 3 km Quasi-Uniform Mesh

[ZIZU296]

Hi everyone,

I am running tropical cyclone simulations with MPAS and plan to use a quasi-uniform 3 km mesh as the reference experiment. However, during the initialization stage (input data reading and memory allocation), the model crashes and reports errors.

Has anyone encountered a similar issue with high-resolution quasi-uniform meshes? Could this be related to memory limits, MPI decomposition, or mesh configuration?​

 

[Ming Chen]

Which version of MPAS you are running? Is this a global run or a regional simulation? What is your input data?

Can you clarify step by step how you set up your case? Also, did you modify MPAS codes?

Please also upload your namelist.init_atmosphere , namelist.atmopshere, and stream files for me to take a look. Thanks.

 

[ZIZU296]

Thank you for your response.

I am running MPAS version 8.2.1 for a global simulation. For the input data, I am using ERA5 reanalysis data at a 0.25-degree resolution to generate the init files. I also incorporate hourly sfc_update files as part of the model's input.

Regarding the code, I have not modified the MPAS source code; I only configured the environment settings for the ifort compiler prior to compilation.

I have included my namelist.init_atmosphere, namelist.atmosphere, and streams files below for your review. Additionally, I have attached my job submission script for your reference.

 

[Ming Chen]

Hi,

Thank you for uploading the files. Your namelist files look fine to me except that: (1) the radiation time step of 30min is way too large for the 3km mesh. Please change it to 3 or 5 min in your future run, and (2) please turn off convection scheme since you run with 3km mesh.

One issue I am concerned is that, you run this big case with only 1600 processors. This makes me suspicious that you don't have sufficient memory. Can you increase the number of processors (this will give you a larger memory) and rerun this case? We recommend that a single processor should handle ~ 100 cells.

By the way, I don't know how your input data looks like and I simply assume it is fine. Just for your information, we provide a pre-processing tool to create MPAS initial condition based on ERA5. Please dee details at : GitHub - NCAR/era5_to_int: A simple Python script for converting ERA5 model-level netCDF files to the WPS intermediate format

Hope this is helpful for you.

 

[ZIZU296]

Thanks a lot for the feedback! This is exactly what I needed.

I'm going to update my configuration and restart the 3km simulation following your advice. I'll make sure to sync the new results with you as soon as they are ready.

Thanks again for your support！

Hi,

Thank you for uploading the files. Your namelist files look fine to me except that: (1) the radiation time step of 30min is way too large for the 3km mesh. Please change it to 3 or 5 min in your future run, and (2) please turn off convection scheme since you run with 3km mesh.

One issue I am concerned is that, you run this big case with only 1600 processors. This makes me suspicious that you don't have sufficient memory. Can you increase the number of processors (this will give you a larger memory) and rerun this case? We recommend that a single processor should handle ~ 100 cells.

By the way, I don't know how your input data looks like and I simply assume it is fine. Just for your information, we provide a pre-processing tool to create MPAS initial condition based on ERA5. Please dee details at : GitHub - NCAR/era5_to_int: A simple Python script for converting ERA5 model-level netCDF files to the WPS intermediate format

Hope this is helpful for you.