Corrupt wrfout Files after Restart -- No Errors

[kholman]

Hi everyone

I am running WRF version 4.2.1 on a Cray linux system. I completed a one-month real simulation and output restart files on the last day of the month, wrfrst_d0*_1993-07-31_00:00:00. The restart file is a valid netcdf file. I restarted the simulation with a start date of 1993-07-01 00:00:00. The restart simulation runs without errors in the rsl.error.* or rsl.out.* files. New wrfout files are created (rsl.out* confirms the writing of this data). Here is the confusing part. If I output one hour per output file (in the restart simulation), the files are created normally (I can view, extract, plot, etc.). If I write multiple timesteps per output file (like in the original simulation), the new wrfout files are corrupt. They grow, but I cannot access the data in them. Instead, I get a NetCDF: HDF error. Does this seem like a setting issue in the namelist.input file or a system problem? I am not sure where exactly to start here. Here is a copy of my namelist.input file.

&time_control
run_days = 0,
run_hours = 0,
run_minutes = 0,
run_seconds = 0,
start_year = 1993, 1993, 1993,
start_month = 07, 07, 07,
start_day = 31, 31, 31,
start_hour = 00, 00, 00,
start_minute = 00, 00, 00,
start_second = 00, 00, 00,
end_year = 1993, 1993, 1993,
end_month = 09, 09, 09,
end_day = 01, 01, 01,
end_hour = 00, 00, 00,
end_minute = 00, 00, 00,
end_second = 00, 00, 00,
interval_seconds = 21600,
input_from_file = .true., .true., .true.,
history_interval = 60, 60, 60,
frames_per_outfile = 360, 360, 1,
restart = .true.,
restart_interval = 43200,
write_hist_at_0h_rst = .true.,
io_form_history = 2,
io_form_restart = 2,
io_form_input = 2,
io_form_boundary = 2,
debug_level = 0,
/

&domains
time_step = 30,
time_step_fract_num = 0,
time_step_fract_den = 1,
max_dom = 3,
e_we = 300, 418, 484,
e_sn = 248, 427, 511,
e_vert = 50, 50, 50,
p_top_requested = 5000,
num_metgrid_levels = 38,
num_metgrid_soil_levels = 4,
dx = 9000, 3000, 1000,
dy = 9000, 3000, 1000,
grid_id = 1, 2, 3,
parent_id = 1, 1, 2,
i_parent_start = 1, 81, 140,
j_parent_start = 1, 55, 118,
parent_grid_ratio = 1, 3, 3,
parent_time_step_ratio = 1, 3, 3,
feedback = 1,
smooth_option = 0,
/

&physics
mp_physics = 8, 8, 8,
ra_lw_physics = 1, 1, 1,
ra_sw_physics = 1, 1, 1,
radt = 30, 30, 30,
sf_sfclay_physics = 2, 2, 2,
sf_surface_physics = 2, 2, 2,
bl_pbl_physics = 2, 2, 2,
bldt = 0, 0, 0,
cu_physics = 1, 1, 0,
cudt = 5, 5, 5,
isfflx = 1,
ifsnow = 0,
icloud = 1,
surface_input_source = 1,
num_soil_layers = 4,
sf_urban_physics = 0, 0, 0,
maxiens = 1,
maxens = 3,
maxens2 = 3,
maxens3 = 16,
ensdim = 144,
sf_lake_physics = 0, 0, 1,
lakedepth_default = 10., 10., 10.,
lake_min_elev = 1., 1., 1.,
use_lakedepth = 0, 0, 1,
/

&fdda
grid_fdda = 2, 0, 0,
gfdda_inname = "wrffdda_d<domain>",
gfdda_end_h = 721, 721, 721,
gfdda_interval_m = 360, 360, 360,
fgdt = 0, 0, 0,
if_no_pbl_nudging_uv = 1, 1, 1,
if_no_pbl_nudging_t = 1, 1, 1,
if_no_pbl_nudging_q = 1, 1, 1,
guv = 0.0003, 0.0003, 0.0003,
gt = 0.0003, 0.0003, 0.0003,
gq = 0.0000, 0.0000, 0.0000,
if_ramping = 0,
dtramp_min = 60.0,
io_form_gfdda = 2,
xwavenum = 2, 0, 0,
ywavenum = 2, 0, 0,
/

&dynamics
w_damping = 0,
diff_opt = 1,
km_opt = 4,
diff_6th_opt = 0, 0, 0,
diff_6th_factor = 0.12, 0.12, 0.12,
base_temp = 290.,
damp_opt = 0,
zdamp = 5000., 5000., 5000.,
dampcoef = 0.2, 0.2, 0.2,
khdif = 0, 0, 0,
kvdif = 0, 0, 0,
non_hydrostatic = .true., .true., .true.,
moist_adv_opt = 1, 1, 1,
scalar_adv_opt = 1, 1, 1,
/

&bdy_control
spec_bdy_width = 5,
spec_zone = 1,
relax_zone = 4,
specified = .true., .false., .false.,
nested = .false., .true., .true.,
/

&grib2
/

&namelist_quilt
nio_tasks_per_group = 0,
nio_groups = 1,
/

Any tips or suggestions on this issue will be greatly appreciated.

Thanks,
Katie

 

[kwerner]

Hi Katie,
That is very odd and should not be the case. To be sure, I just tested out your scenario and it ran fine, and I was able to access the resulting netcdf file without any problems. I assume when you run the restart simulation, the rsl files are also showing that the run was successful at the end? If that's the case, then it sounds like this is a system problem, and that unfortunately you'll need to discuss the problem with a systems administrator at your institution.

 

[kdholman]

Hi Kelly

Thank you for trying this out. The rsl.out* and rsl.error* files show no issues, so I think I need to contact a system admin. I'm having similar issues with other NetCDF files as well. Thank you again for taking the time on this problem. I appreciate it.

Katie