Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

MPAS-A simulation stopped automatically

518er2005

Member
I have given a simulation for a tropical cyclone (8 days). It is a 10-kilometer limited area simulation. The simulation is automatically terminated after four days of successful operation. It gives the error "Program received signal SIGSEGV: Segmentation fault - invalid memory reference." I am attaching the log file along with the namelist for reference. If anyone can tell me how to solve this error, it will be much helpful.
 

Attachments

  • mpas.log
    1.3 KB · Views: 3
  • namelist.atmosphere.txt
    2.1 KB · Views: 3
As a starting point, you could try reducing the timestep (config_dt in your namelist.atmosphere file) from 60 s to 50 s to see whether that helps. You can also get more information in the log.atmosphere.0000.out file about maximum wind speeds by setting config_print_detailed_minmax_vel = true in your namelist.atmosphere file. If the model still crashes with a 50 s timestep, could you attach your log.atmosphere.0000.out file (which should include global maximum winds if you've set config_print_detailed_minmax_vel in your namelist)?
 
Actually, I followed the steps as suggested by considering a different case study, and it worked; however, when I ran the model again for the case in which the error was encountered before, I got a similar error. I am attaching the streams.atmosphere and the namelist.atmosphere. Along with it, I am attaching the log.atmosphere.0000.out too. And also, ERA5 data is used as initial conditions for the model run. The error encountered is
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= PID 4178930 RUNNING AT localhost.localdomain
= EXIT CODE: 139
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestions
 

Attachments

  • namelist.atmosphere.txt
    1.7 KB · Views: 2
  • streams.atmosphere.txt
    2 KB · Views: 0
  • log.atmosphere.0000.out.txt
    529.4 KB · Views: 5
Ah ~ I have the similar error, showing "segmentation fault". So, I uploaded my files here. I attached the namelist.input, rsl.error.0000, and slurm error output. Any suggestions would be much appreciated! Thank you!
 

Attachments

  • slurm-err.txt
    928 bytes · Views: 1
  • rsl.error.0000.txt
    182.8 KB · Views: 2
  • namelist.input.txt
    3.8 KB · Views: 4
@518er2005 Looking through your log file, there's a timestep in which the normal component of the horizontal velocity reaches 1899 m/s in magnitude:
Begin timestep 2020-05-16_12:39:10
--- time to run the LW radiation scheme L_RADLW =F
--- time to run the SW radiation scheme L_RADSW =F
--- time to run the convection scheme L_CONV =T
--- time to apply limit to accumulated rainc and rainnc L_ACRAIN =F
--- time to apply limit to accumulated radiation diags. L_ACRADT =F
--- time to calculate additional physics_diagnostics =F
split dynamics-transport integration 3

global min, max w -10.7122 153.284
global min, max u -1899.06 1050.50
Timing for integration step: 20.3251 s
It would be helpful to know whether these unphysically large velocities are occurring near a lateral boundary, and to also know whether they're occurring, e.g., near the model top. As I suggested earlier, you can set config_print_detailed_minmax_vel = true in the &printout namelist group in your namelist.atmosphere file to get a better idea of where the domain maximum values are occurring. If the issue is near the model top, there may be some additional filters that could be helpful. If the issue is near the surface along a lateral boundary, there are pre-processing options that may be helpful when generating ICs and LBCs.
 
I ran this time with 28 cores and received the error as attached below. Also, for your convenience, I am attaching the log.atmosphere.out file and the log.atmosphere.0000.err file.
 

Attachments

  • log.atmosphere.0000.out.txt
    2.4 MB · Views: 0
  • log.atmosphere.0015.err.txt
    344 bytes · Views: 0
  • namelist.atmosphere.txt
    1.7 KB · Views: 1
Top