Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

WRFPLUS parallel operation problem

This post was from a previous version of the WRF&MPAS-A Support Forum. New replies have been disabled and if you have follow up questions related to this post, then please start a new thread from the forum home page.

Jiwon Lee

New member
Hi,

I keep getting errors in the process of generating prediction fields(trajectory for adjoint run) using WRFPLUS to calculate FSO.
The model is run using mpirun -n 3 wrf.exe
If more than 4 CPUs are used, only the following process is calculated and the model is no longer calculated.


<rsl.out.0000>
Ntasks in X 4 , ntasks in Y 4
Setting blank km_opt entries to domain #1 values.
--> The km_opt entry in the namelist.input is now max_domains.
Setting blank diff_opt entries to domain #1 values.
--> The diff_opt entry in the namelist.input is now max_domains.
--- WARNING: traj_opt is zero, but num_traj is not zero; setting num_traj to zero.
--- NOTE: sst_update is 0, setting io_form_auxinput4 = 0 and auxinput4_interval = 0 for all domains
--- NOTE: sst_update is 0, setting io_form_auxinput4 = 0 and auxinput4_interval = 0 for all domains
--- NOTE: grid_fdda is 0 for domain 1, setting gfdda interval and ending time to 0 for that domain.
--- NOTE: both grid_sfdda and pxlsm_soil_nudge are 0 for domain 1, setting sgfdda interval and ending time to 0 for that domain.
--- NOTE: obs_nudge_opt is 0 for domain 1, setting obs nudging interval and ending time to 0 for that domain.
--- NOTE: grid_fdda is 0 for domain 2, setting gfdda interval and ending time to 0 for that domain.
--- NOTE: both grid_sfdda and pxlsm_soil_nudge are 0 for domain 2, setting sgfdda interval and ending time to 0 for that domain.
--- NOTE: obs_nudge_opt is 0 for domain 2, setting obs nudging interval and ending time to 0 for that domain.
--- NOTE: bl_pbl_physics /= 4, implies mfshconv must be 0, resetting
Need MYNN PBL for icloud_bl = 1, resetting to 0
--- NOTE: RRTMG radiation is not used, setting: o3input=0 to avoid data pre-processing
*************************************
No physics suite selected.
Physics options will be used directly from the namelist.
*************************************
--- NOTE: num_soil_layers has been set to 4
WRF V3.9.1 MODEL
*************************************
Parent domain
ids,ide,jds,jde 1 352 1 352
ims,ime,jms,jme -4 95 -4 95
ips,ipe,jps,jpe 1 88 1 88
*************************************
DYNAMICS OPTION: Eulerian Mass Coordinate
alloc_space_field: domain 1 , 1746440840 bytes allocated
med_initialdata_input: calling input_input
Timing for processing wrfinput file (stream 0) for domain 1: 29.93507 elapsed seconds
Max map factor in domain 1 = 1.05. Scale the dt in the model accordingly.
INPUT LandUse = "MODIFIED_IGBP_MODIS_NOAH"
LANDUSE TYPE = "MODIFIED_IGBP_MODIS_NOAH" FOUND 33 CATEGORIES 2 SEASONS WATER CATEGORY = 17 SNOW CATEGORY = 15
INITIALIZE THREE Noah LSM RELATED TABLES
Skipping over LUTYPE = USGS
LANDUSE TYPE = MODIFIED_IGBP_MODIS_NOAH FOUND 20 CATEGORIES
INPUT SOIL TEXTURE CLASSIFICATION = STAS
SOIL TEXTURE CLASSIFICATION = STAS FOUND 19 CATEGORIES
Max map factor in domain 1 = 1.05. Scale the dt in the model accordingly.
INPUT LandUse = "MODIFIED_IGBP_MODIS_NOAH"
LANDUSE TYPE = "MODIFIED_IGBP_MODIS_NOAH" FOUND 33 CATEGORIES 2 SEASONS WATER CATEGORY = 17 SNOW CATEGORY = 15
INITIALIZE THREE Noah LSM RELATED TABLES
Skipping over LUTYPE = USGS
LANDUSE TYPE = MODIFIED_IGBP_MODIS_NOAH FOUND 20 CATEGORIES
INPUT SOIL TEXTURE CLASSIFICATION = STAS
SOIL TEXTURE CLASSIFICATION = STAS FOUND 19 CATEGORIES
*************************************
Nesting domain
ids,ide,jds,jde 1 250 1 361
ims,ime,jms,jme -4 73 -4 100
ips,ipe,jps,jpe 1 63 1 90
INTERMEDIATE domain
ids,ide,jds,jde 143 398 125 491
ims,ime,jms,jme 138 216 120 225
ips,ipe,jps,jpe 141 206 123 215
*************************************
d01 2018-08-28_06:00:00 alloc_space_field: domain 2 , 397765000 bytes allocated
d01 2018-08-28_06:00:00 alloc_space_field: domain 2 , 1440220176 bytes allocated
d01 2018-08-28_06:00:00 *** Initializing nest domain # 2 from an input file. ***


The model is not turned off and nothing is produced.
Using 200 cpu and waiting 3 hours, nothing was calculated.
The same phenomenon occurs in WRFPLUS V3.8, V3.9.1 and V4.1.
Below is the namelist.input I used.
What is the problem?
Please help.


<namelist.input>
&time_control
run_days = 0,
run_hours = 0,
run_minutes = 0,
run_seconds = 0,
start_year = 2018, 2018, 2018,
start_month = 08, 08, 08,
start_day = 28, 28, 28,
start_hour = 06, 06, 06,
start_minute = 00, 00, 00,
start_second = 00, 00, 00,
end_year = 2018, 2018, 2018,
end_month = 08, 08, 08,
end_day = 28, 28, 28,
end_hour = 18, 18, 18,
end_minute = 00, 00, 00,
end_second = 00, 00, 00,
interval_seconds = 21600,
input_from_file = .true., .true., .true.,
fine_input_stream = 0, 0, 0,
history_interval = 540, 180, 60,
frames_per_outfile = 1, 1, 1,
restart = .false.,
restart_interval = 7200,
io_form_history = 2
io_form_restart = 2
io_form_input = 2
io_form_boundary = 2
debug_level = 0
/

&domains
time_step = 30,
time_step_fract_num = 0,
time_step_fract_den = 1,
max_dom = 3,
e_we = 352, 250, 226,
e_sn = 352, 361, 196,
e_vert = 60, 60, 60,
p_top_requested = 5000,
num_metgrid_levels = 32,
num_metgrid_soil_levels = 4,
dx = 9000, 3000, 1000,
dy = 9000, 3000, 1000,
grid_id = 1, 2, 3,
parent_id = 0, 1, 2,
i_parent_start = 1, 145, 78,
j_parent_start = 1, 127, 155,
parent_grid_ratio = 1, 3, 3,
parent_time_step_ratio = 1, 3, 3,
feedback = 1,
smooth_option = 1,
/

&physics
mp_physics = 16, 16, 16,
ra_lw_physics = 1, 1, 1,
ra_sw_physics = 1, 1, 1,
radt = 10, 10, 10,
sf_sfclay_physics = 1, 1, 1,
sf_surface_physics = 2, 2, 2,
bl_pbl_physics = 1, 1, 1,
bldt = 0, 0, 0,
cu_physics = 1, 1, 0,
cudt = 5, 5, 5,
isfflx = 1,
ifsnow = 1,
icloud = 1,
surface_input_source = 1,
num_soil_layers = 4,
sf_urban_physics = 0, 0, 0,
sst_update = 0,
do_radar_ref = 1,
/

&fdda
/

&dynamics
w_damping = 0,
diff_opt = 1,
km_opt = 4,
diff_6th_opt = 0, 0, 0,
diff_6th_factor = 0.12, 0.12, 0.12,
base_temp = 290.
damp_opt = 0,
zdamp = 5000., 5000., 5000.,
dampcoef = 0.2, 0.2, 0.2
khdif = 0, 0, 0,
kvdif = 0, 0, 0,
non_hydrostatic = .true., .true., .true.,
moist_adv_opt = 1, 1, 1,
scalar_adv_opt = 1, 1, 1,
/

&bdy_control
spec_bdy_width = 5,
spec_zone = 1,
relax_zone = 4,
specified = .true., .false.,.false.,
nested = .false., .true., .true.,
/

&grib2
/
&perturbation
trajectory_io=true,
jcdfi_use=false,
/
&namelist_quilt
nio_tasks_per_group = 0,
nio_groups = 1,
/





Thanks for your help!

Best regards,
Ji-Won Lee
 
Hi,

Has your problem been solved?When I set cu_physics=98 for adjoint run,I have a similar problem.Using parallel computing, there is no integration process.When I use ./wrf.exe directly, It can run normally with integral record. However, It's result file is empty.

Do you have any suggestions? Thank you for your help!

Best regards,
Tang
 
I have moved this topic to the WRFDA section of the forum, as this seems to pertain specifically to WRFPLUS. Someone will respond to your inquires soon.
 
Top