(RESOLVED) Error: Fix non-monotonic "eta_levels" in the namelist.input file

[lquitian]

Hi,

I am trying to run a simulation with this particular configuration:

e_we = 1000,1000,
e_sn = 1000,1000,
dx = 7500,2500,
dy = 7500,2500,
e_vert = 36,36,
i_parent_start = 1,330,
j_parent_start = 1,330,
parent_grid_ratio = 1,3,

However, although the geogrid, metgrid, ungrib, and real modules finish without problems. The wrf module does not finish and I do not get any error messages.
I obtain the met_em files (I attach the screenshot) and the rsl.error file (I also attach the screenshot) indicates that I am using the WRF-Chem (which I have never compiled or used before) and it directly stops with an alloc_space_domain message.

Could someone tell me what is going on and how can I get it to work?

PD. I have run another simulation with another configuration and it was able to finish successfully.

Kind regards and thanks in advance.
Lara

 

[kwerner]

Hi Lara,
I think it's very likely that the model stops running because you aren't using enough processors. According to your rsl file, you are using 72 processors, but your domains are 1000x1000, which is very large. Take a look at this FAQ that discusses choosing the correct number of processors.

 

[lquitian]

Thank you very much!

Kind regards,
Lara

 

[lquitian]

Hello,

Finally, the simulation was able to finish by using this PBS configuration:

#PBS -l Walltime=30:00:00
#PBS -l EC_total_tasks=128
#PBS -l EC_threads_per_task=1
#PBS -l EC_hyperthreads=1

However, I would now need to increase the vertical levels from 32 to 65. Does anyone know if I have to modify something else on the namelist.input script?

Moreover, I have increased the total tasks to 256 but it still does not work. Do I have to increase them even more?
#PBS -l EC_total_tasks=256

Thanks in advance.
Kind regards,
Lara

 

[kwerner]

Lara,

So the simulation ran okay with 128 processors, but not with 256?

As for increasing your vertical levels, you just need to change e_vert to 65; however, you mention that it was 32 previously. Are you referring to the num_metgrid_levels (which is the number of vertical levels from the input data)? If so, you will just modify that setting, instead. Either way, you'll need to save the namelist changes and re-run real.exe before running wrf.exe.

 

[lquitian]

Hello!

I'm referring to the e_vert option. In a first test, the simulation ran OK with e_vert=36 and 128 processors.

Now, I would like to modify the e_vert option to 65. I have tried with 128, 196, and 256 processors but none of these options worked. So I'm beginning to think that it's not a problem of using too few processors.

Could you please help me?

Kind regards,
Lara

 

[kwerner]

Lara,
Can you package your rsl.error.* files together into a single *.TAR file and attach that so that I can take a look? Please also attach your latest namelist.input file. Thanks!

 

[lquitian]

Hi,

Thank you very much. I attach the tar file to this mail.

Kind regards,
Lara

 

[lquitian]

Hi,

Could you please provide me with information on how to set the vertical levels in the namelist.input? I would like to know how they are calculated and what would be the correct procedure to define them?

I have now defined 65 levels in this way (I have not followed any criteria and maybe the error I get has a lot to do with it):

eta_levels = 1.00, 0.998, 0.995, 0.993, 0.99, 0.99, 0.988, 0.985, 0.983, 0.98, 0.978, 0.975, 0.973, 0.97, 0. 968, 0.965, 0.963, 0.96, 0.955, 0.95, 0.94, 0.93, 0.92, 0.91, 0.90, 0.895, 0.89, 0.885, 0.88, 0.875, 0.87, 0. 865, 0.86, 0.855, 0.85, 0.845, 0.84, 0.825, 0.82, 0.805, 0.80, 0.755, 0.75, 0.70, 0.655, 0.65, 0.63, 0.60, 0. 555, 0.55, 0.53, 0.50, 0.54, 0.45, 0.43, 0.40, 0.35, 0.33, 0.30, 0.25, 0.23, 0.20, 0.15, 0.13, 0.10, 0.00,

Thanks in advance.

Kind regards,
Lara

 

[kwerner]

Lara,
Thanks for sending that. The error message in the rsl.error file is:

-------------- FATAL CALLED ---------------
FATAL CALLED FROM FILE:  <stdin>  LINE:      75
 program wrf: error opening wrfinput_d01 for reading ierr=       -1021

So it's looking for a complete wrfinput_d01 file, but isn't able to find it. Does that file exist in your running directory? Did the real.exe program finish to completion (i.e., did it finish writing the full wrfinput_d01 file)?

As for vertical levels, unless you have specific levels in mind that you want to use, you don't have to set those, yourself. If you simply set e_vert = 65, the model will create the levels in the most reasonable way.

Additionally, 256 processors is certainly more than you were using, but ideally you would want a lot more for 1000x1000 grid spaces in your domains. When you are adding 30 additional vertical levels, that is even more calculating the model needs to do. I'm not sure how many processors you have access to, or how many are on each node, but using something closer to 2500 would be ideal for this domain.

 

[lquitian]

Hi,

No, the wrfinput_d01 files were not generated.

Regarding the vertical levels, I thought you had to define the number of levels you wanted in the namelist.input.

I have now defined the simulation with 65 vertical levels, leaving the model to set them on its own. I have now used 1504 processor considering the maximum I should use are 1600 in accordance with the link you facilitated me to choose the correct number of processors.

Considering my configuration for the smallest-sized domain:

((e_we)/25) * ((e_sn)/25) = most amount of processors you should use
(1000/25) * (1000/25) = 1600

I should use a maximum of 1600 processors.

I would let you know the results.

Kind regards and thank you,
Lara

 

[lquitian]

Hi,

As I told you in the previous message I have run a simulation with 65 vertical levels, letting the model choose for itself the most convenient levels. I have used 1504 processors first and then 1568. However, it still does not work. The real.exe does not generate the wrfinput_d01 files (I attach the screenshot).

I do not know what to do now

Any recommendation?

Kind regards,
Lara

 

[kwerner]

Hi,
The screenshot simply shows that real.exe stopped running. Can you package the rsl.* files together for the real.exe run and attach those for me, along with your updated namelist? Thanks!

 

[lquitian]

Hello,

The issue is that I think the rsl. files are related to the wrf.exe but, sincerely, I'm not sure enough. I think those files are rewritten with the last active process and that is wrf.exe.

In any case, I attach those files here. I have also included the namelist I used and the run scripts for real and wrf so you can see the ideal PBS configuration. Maybe I have to modify the number of threads used.

Thanks in advance and kind regards,
Lara

 

[kwerner]

Lara,
Unfortunately I am not going to be able to help without seeing the rsl files for real.exe. If real did not run successfully, then wrf will not run successfully, so there is no benefit in hanging on to your most recent wrf run. Can you please run real.exe again and send me the new rsl.* files for real.exe? Thanks!

 

[lquitian]

Hi,

I was able to save the rsl. file for the real.exe. I have performed two types of experiments this time (attached are the rsl.error and rsl.output files of both experiments):

1.- Defining myself the 65 levels in the namelist.input (which I have called rsl.error.0000_65 and rsl.output.0000_65).
2.- Without defining by myself and letting the model choose by itself the 65 most convenient levels (which I have called rsl.error.0000 and rsl.output.0000).

Thanks in advance for all your help.

Kind regards,
Lara

 

[kwerner]

Hi,
Thanks for sending those. Can you send the two namelists that go with each of those runs (with your defining the 65 levels, and without)? The errors seem to potentially be related to namelist problems, so I'd like to take a close look at each of those. Thanks!

 

[lquitian]

Hi!

I enclose both namelists.

Thanks in advance.

Kind regards,
Lara

 

[kwerner]

Lara,
Thanks. In the namelist_ERA5_1000.input file (where you are letting the model choose the levels), you need to remove the eta_levels from the namelist file. It sees that you have those defined, and you only have 36 levels listed, but are setting e_vert = 65, which is why you are getting the error:
First eta level should be 1.0 and the last 0.0 in namelist

 

[lquitian]

Hi,

Thank you very much. Once I removed the eta_level option from the namelit.input, it finally worked and is running. However, I don't know how to fix the other experiment (where I define the eta_levels by myself) because I get the same error. However, in that case, I defined 65 levels in the eta_level option and set e_vert=65 in the namelist.input.

Thanks in advance.

Kind regards,
Lara