Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

wrfchem 1km, forrtl: severe (174): SIGSEGV, segmentation fault occurred

This post was from a previous version of the WRF&MPAS-A Support Forum. New replies have been disabled and if you have follow up questions related to this post, then please start a new thread from the forum home page.

Lily_Zh

Member
Hello,

I tested run Jun.24 for 36km, 6km and 1km WRF/Chem 3.9 using one way nesting and FNL data. I ran geogrid.exe and metgrid.exe in WPS3.9.0.1.
I can run WRF/Chem for 36km and 6km successfully, but has this 174 segmentation error when run wrf.exe for 1km, only generated one file wrfout_d01_2019-06-24_00:00:00. It stopped quickly after running wrf.exe, so the input data may has issue. I tried to reduce time_step to 1 or node=1, still not work.
GFL data downloaded from https://rda.ucar.edu/datasets/ds084.1/, I also tested this one https://rda.ucar.edu/datasets/ds083.2/ and https://rda.ucar.edu/datasets/ds083.3/, still exist this error.

I attached the namelist.input. Does anyone have any idea about this issue?

rsl.error.0006
Ntasks in X 12 , ntasks in Y 16
--- WARNING: traj_opt is zero, but num_traj is not zero; setting num_traj to zero.
--- NOTE: sst_update is 0, setting io_form_auxinput4 = 0 and auxinput4_interval = 0 for all domains
--- NOTE: grid_fdda is 0 for domain 1, setting gfdda interval and ending time to 0 for that domain.
--- NOTE: both grid_sfdda and pxlsm_soil_nudge are 0 for domain 1, setting sgfdda interval and ending time to 0 for that domain.
--- NOTE: obs_nudge_opt is 0 for domain 1, setting obs nudging interval and ending time to 0 for that domain.
--- NOTE: bl_pbl_physics /= 4, implies mfshconv must be 0, resetting
--- NOTE: RRTMG radiation is in use, setting: levsiz=59, alevsiz=12, no_src_types=6
--- NOTE: num_soil_layers has been set to 4
WRF V3.7.1 MODEL
*************************************
Parent domain
ids,ide,jds,jde 1 91 1 91
ims,ime,jms,jme 39 59 -4 13
ips,ipe,jps,jpe 46 52 1 6
*************************************
DYNAMICS OPTION: Eulerian Mass Coordinate
alloc_space_field: domain 1 , 98621504 bytes allocated
med_initialdata_input: calling input_input
INPUT LandUse = "MODIFIED_IGBP_MODIS_NOAH"
LANDUSE TYPE = "MODIFIED_IGBP_MODIS_NOAH" FOUND 33 CATEGORIES 2 SEASONS WATER CATEGORY = 17 SNOW CATEGORY = 15
INITIALIZE THREE Noah LSM RELATED TABLES
*********************************************************************
* PROGRAM:WRF/CHEM V3.7.1 MODEL
* *
* PLEASE REPORT ANY BUGS TO WRF/CHEM HELP at *
* *
* wrfchemhelp.gsd@noaa.gov *
* *
*********************************************************************
call ftuv phot initialization
ftuv_init: calling photo_inti for id = 1
mediation_integrate: med_read_wrf_chem_emissions: Open file wrfchemi_00z_d01
HOURLY EMISSIONS UPDATE TIME 0.0 0.0
mediation_integrate: med_read_wrf_chem_emissions: Read emissions for time 2019-06-24_00:00:00
**WARNING** Time in input file not being checked **WARNING**
Tile Strategy is not specified. Assuming 1D-Y
WRF TILE 1 IS 46 IE 52 JS 1 JE 6
WRF NUMBER OF TILES = 1
calculate MEGAN emissions at ktau, gmtp, tmidh = 1 0.0000000E+00 1.3888889E-04
photolysis_driver: called for domain 1
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
wrf.exe 0000000004181BAD for__signal_handl Unknown Unknown
libpthread-2.17.s 00002B8324C3A5D0 Unknown Unknown Unknown
wrf.exe 00000000030CA111 Unknown Unknown Unknown
wrf.exe 00000000030C7006 Unknown Unknown Unknown
wrf.exe 000000000221A174 Unknown Unknown Unknown
wrf.exe 00000000022112D1 Unknown Unknown Unknown
wrf.exe 0000000001B44FC1 Unknown Unknown Unknown
wrf.exe 0000000001B2E107 Unknown Unknown Unknown
wrf.exe 0000000001A07775 Unknown Unknown Unknown
wrf.exe 0000000000587385 Unknown Unknown Unknown
wrf.exe 000000000040DB21 Unknown Unknown Unknown
wrf.exe 000000000040DADF Unknown Unknown Unknown
wrf.exe 000000000040DA7E Unknown Unknown Unknown
libc-2.17.so 00002B832516B3D5 __libc_start_main Unknown Unknown
wrf.exe 000000000040D969 Unknown Unknown Unknown

Thank you in advance.

Lily
 
Just update, for 1km run, when I change sf_urban_physics from 1 to 0, it can work. And without chem part, it also work.
Try WRFChem4.3, it does not work.
I will appreciate for any suggestion

Thank you in advance.
 
just update,

I used wrfchem 4.3.
Based on :
d01 2019-06-24_00:00:00 SORGAM aerosols initialization
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
wrf.exe 000000000700440D Unknown Unknown Unknown
libpthread-2.17.s 00002B5752A485D0 Unknown Unknown Unknown
wrf.exe 0000000004E1BB0B module_ftuv_subs_ 2032 module_ftuv_subs.f90
wrf.exe 0000000004E195B0 module_ftuv_subs_ 1768 module_ftuv_subs.f90
wrf.exe 0000000004E0F7A6 module_ftuv_subs_ 444 module_ftuv_subs.f90
wrf.exe 0000000002E7F10F module_ftuv_drive 1226 module_ftuv_driver.f90
wrf.exe 0000000002E6B77E module_ftuv_drive 508 module_ftuv_driver.f90
wrf.exe 0000000002159F5D photolysis_driver 194 photolysis_driver.f90
wrf.exe 0000000002131F9F chem_driver_ 1035 chem_driver.f90
wrf.exe 0000000001EFC455 solve_interface_ 186 solve_interface.f90
wrf.exe 00000000005DFD47 module_integrate_ 329 module_integrate.f90
wrf.exe 0000000000411988 module_wrf_top_mp 326 module_wrf_top.f90
wrf.exe 0000000000411145 MAIN__ 29 wrf.f90
wrf.exe 00000000004110FE Unknown Unknown Unknown
libc-2.17.so 00002B5752F793D5 __libc_start_main Unknown Unknown
wrf.exe 0000000000410FE9 Unknown Unknown Unknown

The error may be related in the line2032 in module_ftuv_subs.f90:
rjm(k) = x_table(ki,iw) + dels(k)*(x_table(ki+1,iw) - x_table(ki,iw))

I am not familiar with codes of WRF and WRF/Chem. I will try to understand the code.
 
Update.

it seems related with
z(ip1) in deltaz = km2cm * (z(ip1) - z(i)) of module_ftuv_subs.F

it suddenly becomes NaN. Not sure the reason.
z(ip1), z(i) 50.0000000000000 48.0000000000000
z(ip1), z(i) NaN 0.000000000000000E+000

NaN PH lead to that.
 
Hi Jordan,

Thank you for your reply.
I tried it, still has fault.

d01 2019-06-24_00:00:00 optical driver: after call optical averaging
d01 2019-06-24_00:00:00 calling photolysis driver
photolysis_driver: called for domain 1
d01 2019-06-24_00:00:00 calling madronich1_driver
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
wrf.exe 0000000005B4174D Unknown Unknown Unknown
libpthread-2.17.s 00002B54DFF0B5D0 Unknown Unknown Unknown
wrf.exe 00000000027DB69D module_phot_mad_m 1656 module_phot_mad.f90
wrf.exe 00000000027D8814 module_phot_mad_m 1385 module_phot_mad.f90
wrf.exe 0000000001DEE6EF photolysis_driver 155 photolysis_driver.f90
wrf.exe 0000000001DC9C6C chem_driver_ 971 chem_driver.f90
wrf.exe 0000000001C06E69 solve_interface_ 174 solve_interface.f90
wrf.exe 00000000005AA9CF module_integrate_ 326 module_integrate.f90
wrf.exe 000000000040E1AF module_wrf_top_mp 322 module_wrf_top.f90
wrf.exe 000000000040D985 MAIN__ 28 wrf.f90
wrf.exe 000000000040D93E Unknown Unknown Unknown
libc-2.17.so 00002B54E043C3D5 __libc_start_main Unknown Unknown
wrf.exe 000000000040D829 Unknown Unknown Unknown
 
Hi,

Thank you for testing phot_opt =1 - Can you also try setting aer_ra_feedback = 0?

Also, please see if there is further information in one of the rsl.out/error files as they can be different. You can attach a text file created this way if you are on a linux/unix machine: "for file in rsl.out*; do echo ${file} >> rsl.tail.out ; tail -n 10 ${file} >> rsl.tail.out ; done"
 
Hi Jordan,

I set aer_ra_feedback = 0 for 1km run, same error.
I used the command you suggested, get rsl.tail.out:
rsl.out.0000
d01 2019-06-24_00:00:00 emissions_driver not calling add_emiss_cptec
d01 2019-06-24_00:00:00 emissions_driver calling sorgam_vbs_addemiss
d01 2019-06-24_00:00:00 calling optical driver
d01 2019-06-24_00:00:00 optical driver: process aerosols true
d01 2019-06-24_00:00:00 optical averaging
d01 2019-06-24_00:00:00 optical driver: call optical averaging
d01 2019-06-24_00:00:00 optical driver: after call optical averaging
d01 2019-06-24_00:00:00 calling photolysis driver
photolysis_driver: called for domain 1
d01 2019-06-24_00:00:00 calling madronich1_driver

It seems no differences between rsl.error and rsl.output.

I tried to find out variable with NaN when running 1km. Now I found t_2 and w have NaN values by printing out them from dyn_em/module_small_step_em.F, solve_em.F call subroutines in it.

Thank you.
 
Hi,

I apologize if I was unclear, but I was more concerned if there were differences between the rsl.out(s) for the individual processors (i.e., rsl.out.0000, rsl.out.0001, rsl.out.0002, etc..). The command I suggest would print the last 10 lines of each of the rsl files to a single text file - often times a single processor with through the fatal error and it will not be visible in rsl.out/error.0000.

Given that you are getting NaNs, I wonder if you you have issues with your input data. Please try setting chem_opt = 0 to make sure it is the chemistry that is causing your issue.

Jordan
 
Hi Jordan,

Thank you for your reply.
I just used one node to run, so only one rsl.out file.
For 1km, I can run wrf successfully, I can also run wrfchem successfully without urban canopy model.
Let me try only change chem_opt=0.
 
I see - one processor may not be enough for what you are trying to do. And if the model runs in WRF-only mode, and WRF-Chem w/no Urban Canopy Model, then your problem seems to be related to the UCM.

Jordan
 
Hi Jordan,

The error was solved, it is due to the issue about my MEGAN wrfbiochemi_d03, it should be USGS for three domains, but for d03, MMINLU = ..MODIS, so UTYPE_URB = 0 but not 2, so Z0C =0 as a denominator in code.

Now I regenerate wrfbiochem files, it can generate more wrfoutput files now.

Thanks for your help.
 
Top