Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

STOP in Noah-MP using Bias-Corrected CMIP5 CESM dataset

This post was from a previous version of the WRF&MPAS-A Support Forum. New replies have been disabled and if you have follow up questions related to this post, then please start a new thread from the forum home page.

A

akgv19

New member
Hello,

I'm running WRF Version 3.9.1.1 on an HPC system, and I'm using the Bias-Corrected CMIP5 CESM dataset on 3 domains centered over the CONUS. I'm performing a 10-year simulation with Noah-MP as my LSM, and beginning to perform model spin-up at the beginning of 1989. The wrf executable seems to produce output for about the first 40 days without any issue. But during the date of February 12 1989, wrf exits with the following error at the end of one of the rsl error files...

-------------- FATAL CALLED ---------------
FATAL CALLED FROM FILE: <stdin> LINE: 1749
STOP in Noah-MP
-------------------------------------------
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 111 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
In: PMI_Abort(1, N/A)



I'm not finding any obvious cfl errors in any of the rsl files. Here is a copy of my namelist.input file...
View attachment namelist.input

At your convenience, would you be able to give advice on what the issue might be?

~Ali
 
The rsl file shows that the vegetation leaf temperature, TV = 5153.334, which is apparently wrong.
I suspect that this case went wrong probably long before the time it crashed. The namelist settings below are not correct. Please reset and try again:
cu_physics = 1, 1, 1,
sf_sfclay_physics = 2, 2, 2,
sf_surface_physics = 4, 4, 4,
bl_pbl_physics = 6, 6, 6,

(1) Note that for 3km domain, you should turn off cumulus scheme
(2) MYNN 3rd level TKE PBL scheme only works with MYNNSFC surface layer scheme
(3) Did you modify the codes? What is the option of dveg = 10?
 
Thanks for your quick response. I will make sure to turn off the Cumulus scheme for my 3 km domain. Also, some lab-mates in my office specified the exact same physics schemes I did. However, they used the regular Noah LSM (not Noah-MP). They have had no apparent issues running MYNN3 with other surface layer schemes. But I will continue to keep your suggestions in mind.

As for (3), a few months ago I had issues running wrf.exe at all, although real.exe ran fine with no issue. Kelly at WRF suggested maybe I was having issues with the bucket option in my namelist...

https://forum.mmm.ucar.edu/phpBB3/viewtopic.php?f=40&t=5273&p=9861&hilit=Registry.EM_COMMON#p9861

I did what she recommended and recompiled. I also updated a few of the physics options in my namelist, which is what I shared in my post yesterday. After that, I began the simulation and it ran for about the first 40 days, before failing.

I wasn't exactly sure if it was an openMPI issue with the HPC system I was running on or whether it was something to do with WRF. So then I had an HPC administrator test an abbreviated version of my simulation to see if it failed at the exact same point (using the same input data and same namelist options) and it did not. He had to stop the simulation because he ran out of storage space. I suppose it may have failed for him eventually, had he kept running, but I can't say for sure. Also, that administrator never had an issue with the bucket option like I originally had, and never had to change the Registry file and recompile. So I'm still not entirely sure what is causing the "STOP in Noah-MP" error. Do you have any thoughts or ideas, regarding this?

Pertaining to dveg =10, this activates the built-in crop models (related to WRF-Crop or Noah-MP-Crop) inside Noah-MP. I was interested in comparing the output from these built-in crop models to other crop models. I also changed a value in the MPTABLE.TBL file, before running this simulation. I don't believe that requires any recompilation.

Do you know of anyone else who has had similar issues? If you have any other suggestions, I would appreciate hearing back from you at your convenience. Thanks!
 
I don't think WRF has the option of dveg=10. Please correct me if I am wrong.
I am not sure whether this is the reason for the failed case.
Please try other dveg options and see if that narrows down the problem to this option.
 
WRF does have a dveg=10 option it can be found in the user guide here https://www2.mmm.ucar.edu/wrf/users/docs/user_guide_v4/contents.html. It is an option with in &noah_mp that activates the Noah-MP-Crop model. We have used it before but currently when using it, it can't be found when trying to run WRF. Has this option been taken out recently?
 
Yes this option is deactivated in WRF. Please see the code below (in phys/module_sf_noahmplsm.F)

IF(DVEG == 1 .or. DVEG == 6 .or. DVEG == 7) THEN
FVEG = SHDFAC
IF(FVEG <= 0.05) FVEG = 0.05
ELSE IF (DVEG == 2 .or. DVEG == 3 .or. DVEG == 8) THEN
FVEG = 1.-EXP(-0.52*(LAI+SAI))
IF(FVEG <= 0.05) FVEG = 0.05
ELSE IF (DVEG == 4 .or. DVEG == 5 .or. DVEG == 9) THEN
FVEG = SHDMAX
IF(FVEG <= 0.05) FVEG = 0.05
ELSE
WRITE(*,*) "-------- FATAL CALLED IN SFLX -----------"
CALL wrf_error_fatal("Namelist parameter DVEG unknown")
ENDIF
 
You must log in or register to reply here.
Top