real.exe run attempt aborted

[WRF-Chem_CRG_Reed20]

I have been working on my WRF-Chem model, including creating my "geo_em.d0*.nc" and "met_em.d0*.nc" files for real.exe and wrf.exe on the Cheyenne system. However, after I edited my "namelist.input" file in WRF, I cannot seem to get real.exe to run. I am using these commands in '~/WRF/test/em_em/' directory, after editing the "namelist.input":
> ln -sf ../../../WPS/met_em.d01* .
> mpirun -np 1 ./real.exe

I got this error message:
allocations failed: array service not available
mpirun: all_launch.c:715: newash: Assertion 'new_ash != old_ash' failed.
Aborted (core dump)

Next, I thought that the problem might be that I needed more processors, so I got "runreal.csh" and "runwrf.csh" and requested to uses 28 nodes with 36 MPI ranks each with PBS on Cheyenne. The PBS job terminated in about 7 seconds and I found a "real.o7514662" file and several "rsl.out.XXXX" and "rsl.error.XXXX" files in my run directory in WRF. None of my desired output files were in the directory I indicated in the PBS script for "runreal.sch". Additionally, I will attach the tarred "rsl.*.*" files and the "namelist.input" file to this topic soon.

I am wondering if anyone has any ideas what the original error means and how to fix it, so I can get real.exe to run successfully.

 

[WRF-Chem_CRG_Reed20]

Here are the "rsl.*.*" files and the "namelist.input" file.

 

[kwerner]

Hi,
If you take a look at the end of rsl.error.0000, you'll see:

For domain            1 , the domain size is too small for this many processors, or the decomposition aspect ratio is poor.
   Minimum decomposed computational patch size, either x-dir or y-dir, is 10 grid cells.
  e_we =    74, nproc_x =   28, with cell width in x-direction =    2
  e_sn =    61, nproc_y =   36, with cell width in y-direction =    1
  --- ERROR: Reduce the MPI rank count, or redistribute the tasks.
-------------- FATAL CALLED ---------------

This error basically means that you are using too many processors for the domain size you are using. Take a look at this FAQ for more information regarding how many processors is ideal, based on domain size.

Another thing that is odd, though, is that in the namelist you attached, you have your domain sizes set as:

e_we                                = 130,    151,   94,
 e_sn                                = 140,    151,    91,

but the error message above seems to indicate that you are using a domain with settings:

e_we                                = 74
 e_sn                                = 61

so make sure that you are using the correct namelist that you intend to use when running real and WRF.