Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

WRF runs on convection permitting resolution crashes randomly in summer

Josipa

New member
Hi everyone,

I have set up the WRF model 4.5.1 for a long-term historical and projection run, spanning 20 years, with forcing data derived from NoeESM2 GCM. The model is configured to run with SLUCM coupled to the Boulac PBL schemes. The outer domain covers Europe at a 12 km grid resolution, while the inner domain operates at a convection-permitting scale over the Alps region.
Typically, the model runs smoothly during the colder part of the year, but it crashes during the summer months, specifically in July or August. Unfortunately, I do not receive any clear error messages that would help diagnose the issue.
We have managed to overcome the crashing points so far using several approaches:
  • Performing a simple restart from wrfrst file without changing anything, or just changing the restart point
  • The run past the crashing point using WRF compiled with -O2 optimizations instead of -O3 (note that we are using WRF v4.5.1 compiled with Intel)
  • The run past the crashing point using WRF compiled in debug mode (compiled usising configure -d; version compiled with configure -D could not run)
Currently, I am unable to pass through August 2083. I also attempted to use the latest version of WRF, but the issue persists.

The challenge is that the model never crashes at the same point or line, making it difficult to pinpoint the exact cause of the failure. I suspect that the issue might be related to the complex terrain in the Alps, since the same configuration runs without issues for a convection-permitting domain over northern Europe.

I have tested various timesteps values (e.g., 60s, 30s) and increased epssm values (up to 0.5). Additionally, I attempted to change the PBL scheme, as the WRF debug run suggests that the crash might originate in the Boulac PBL scheme (see an example of the rsl.error.0000 file here).

I am also attaching the namelist.input that I am using.

What could be the cause of these crashes, and what additional tests or adjustments should I consider?
 

Attachments

  • namelist.input
    7.6 KB · Views: 11
Last edited:
Hi,
Apologies for the delay in response. For one of the tests that crashes, can you share all of the rsl* files from that case? You can package them into a single *.tar file and attach that here. Please also share the namelist.input file for that specific case, if it's not the one you already shared.

Just FYI - I will be out of the office starting tomorrow (12/24) and won't be back until Jan 6th, so if you need help prior to that, you may want to start a new thread so one of my colleagues will see that it needs to be addressed. If they see posts from me in the thread, they will ignore it since they see someone is already assisting you.
 
Hi @kwerner,
Thank you for you response. I was testing several more thing in the meanwhile, but nothing changed - the model continues to crash even if I change the number of vertical levels, or increase the height of the first model level.
I was also out of the office until today, and now I am sending you the data. All the requested files are packed, and you can download them from this link.
I am looking forward to any suggestion and/or comment, and thank you in advance for that.
 
Hi @kwerner,
After many trials and errors, I am happy to inform you that I managed to get our simulation running. I realized that the model runs smoothly with the urban parameterization scheme turned off (i.e., sf_urban_physics = 0, 0). For this run, we used SLUCM, and the issue was with the TS_SCHEME defined in the URBPARM_LCZ.TBL table. The default option is set to 1 (i.e., 4-layer model), but it seems that for convection-permitting (CP) runs over Europe, this does not work. Instead, it should be switched to 2 (i.e., Force-Restore method). Several CP runs performed by our colleagues from Charles University in Prague and by us showed that this switch is necessary to ensure the simulation runs smoothly with SLUCM. Typically, with option 1, the model crashes during the summer season.
 
That's great news! I'm glad you got it working and thanks for posting the solution. This may be helpful to someone in the future.
 
Top