Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

"real.exe" aborting prematurely

beuraieon_a

Member
Hello. I really need help in solving a problem I have regarding my WRF Model, particularly on the "real.exe" executable file. It seems to be aborting(?) prematurely, with the following appearing in my Terminal:
starting wrf task 0 of 1
starting wrf task 0 of 1
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------

This issue initially arose when I attempted to do a real-data WRF Model model run using GFS analysis dataset for an extratropical cyclone that affected the Northeastern USA and Atlantic Canada in early January 2010 (simulation duration: 00:00 UTC January 2, 2010 - 00:00 UTC January 7, 2010) with a domain having dx and dy both set at 50000 m (time_step = 300), e_we = 100, e_sn = 84 and centered near Nova Scotia.

I did not understand the nature of the problem, so I tried running the WRF Model for a real-data simulation I already had successfully done before, this time for a concurrent extratropical cyclone that affected East Asia (except that the simulation period started at 00:00 UTC January 3, 2010 and the domain had a smaller dx & dy of 40000 m and is centered over the Korean Peninsula). It dawned on me that something is really wrong with my WRF Model when the same failure happened when I ran "real.exe", even though I applied that exact same namelist settings that made the previous run successful. At first, I assumed that this was just due to my input dataset being rather "outdated" (compared to the most recent GFS datasets), so I tried to run the model using newer dataset for another real-data simulation that I already had successfully done before, this time for Typhoon Saola of 2023 (simulation duration: 00:00 UTC August 25, 2023 - 00:00 UTC August 31, 2023; dx & dy = 20000 m; time_step = 120). That third attempt still ended up with the same failure of the "real.exe". I also assumed that the failure happened due to activation of the Runtime I/O option, but I tried removing their corresponding settings in the "namelist.input" and ran the model; it still ended up with a prematurely aborting "real.exe".

I think I had to mention all of these first since I suspect (but I might be wrong) that there must be something I had done (perhaps an edit I had unknowingly did or a detail I failed to notice about the input data, met_em files etc.) when I attempted the WRF Model run for the North American extratropical cyclone, especially that it started when I made that attempt. And this was the first time I encounted this problem and the first time that I cannot replicate previous successful runs. The problem appears so vague to a beginner like me that I had difficulty making the title of this post, i.e. how to describe this problem.

Anyway, these are the things I observed for all of these "real.exe" failures:
1. There are lines I have never seen before inside the "rsl.out.0000" and "rsl.error.0000" files, such as:
d01 2023-08-25_00:00:00 Yes, this special data is acceptable to use: OUTPUT FROM METGRID V4.5
d01 2023-08-25_00:00:00 Input data is acceptable to use: met_em.d01.2023-08-25_00:00:00.nc
metgrid input_wrf.F first_date_input = 2023-08-25_00:00:00
metgrid input_wrf.F first_date_nml = 2023-08-25_00:00:00
d01 2023-08-25_00:00:00 Timing for input 1 s.
d01 2023-08-25_00:00:00 flag_soil_layers read from met_em file is 1
d01 2023-08-25_00:00:00 Turning off use of MAX WIND level data in vertical interpolation
d01 2023-08-25_00:00:00 Turning off use of TROPOPAUSE level data in vertical interpolation
Max map factor in domain 1 = 1.08. Scale the dt in the model accordingly.
Using sfcprs3 to compute psfc
d01 2023-08-25_00:00:00 No average surface temperature for use with inland lakes
Assume Noah LSM input
d01 2023-08-25_00:00:00 forcing artificial silty clay loam at 103 points, out of 14161
d01 2023-08-25_00:00:00 Timing for processing 1 s.
d01 2023-08-25_00:00:00 Timing for output 2 s.
d01 2023-08-25_00:00:00 Timing for loop # 1 = 4 s.
d01 2023-08-25_06:00:00 Yes, this special data is acceptable to use: OUTPUT FROM METGRID V4.5
d01 2023-08-25_06:00:00 Input data is acceptable to use: met_em.d01.2023-08-25_06:00:00.nc
metgrid input_wrf.F first_date_input = 2023-08-25_06:00:00
metgrid input_wrf.F first_date_nml = 2023-08-25_00:00:00
d01 2023-08-25_06:00:00 Timing for input 1 s.
d01 2023-08-25_06:00:00 flag_soil_layers read from met_em file is 1
d01 2023-08-25_06:00:00 Turning off use of MAX WIND level data in vertical interpolation
d01 2023-08-25_06:00:00 Turning off use of TROPOPAUSE level data in vertical interpolation
Using sfcprs3 to compute psfc
d01 2023-08-25_06:00:00 No average surface temperature for use with inland lakes
Assume Noah LSM input
d01 2023-08-25_06:00:00 forcing artificial silty clay loam at 19 points, out of 14161
d01 2023-08-25_06:00:00 Timing for processing 0 s.
d01 2023-08-25_06:00:00 Timing for output 1 s.
d01 2023-08-25_06:00:00 Timing for loop # 2 = 2 s.
2. The resulting "wrfbdy_d01" and "wrfinput_d01" files appear faulty or corrupted (I don't know how to describe them exactly), i.e. "wrf.exe" can still run but also aborts after some few time steps. This definitely happened during the run attempt for the North American extratropical cyclone, but I haven't tried for the two other simulations scenarios.

Sadly, I forgot to save the rsl.error files for the run attempt of the North American extratropical cyclone wherein I first encountered the problem. What I have right now, and attached to this post, are the rsl.error file and other possibly relevant files for the latest run (for 2023 Typhoon Saola) for your perusal. I am hoping that you can figure out the source of this problem and teach me how to resolve this.

Thank you in advance.
 

Attachments

  • ungrib.log
    720 KB · Views: 1
  • rsl.out.0000
    51.4 KB · Views: 0
  • rsl.error.0000
    35 KB · Views: 0
  • namelist.wps
    766 bytes · Views: 1
  • namelist.input
    4.3 KB · Views: 1
  • metgrid.log
    680.3 KB · Views: 0
  • geogrid.log
    22.6 KB · Views: 0
Would you please clarify what data you ungribed? The error message doesn't really help. I guess we need to start from the beginning of data process.
 
Top