points exceeded cfl=2 in domain d01

[Tom]

Hello!
I am trying to run a case with 9km resolution and 3 nests.But it crashed after run serval hours.The error is:
"
d01 2024-04-19_07:02:24 1 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:02:24 hours
d01 2024-04-19_07:02:24 MAX AT i,j,k: 193 192 27 vert_cfl,w,d(eta)= 2.00234652 27.6386280 2.01599300E-02
d01 2024-04-19_07:03:00 3 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:03:00 hours
d01 2024-04-19_07:03:00 MAX AT i,j,k: 193 192 27 vert_cfl,w,d(eta)= 2.08140683 28.6493530 2.01599300E-02
d01 2024-04-19_07:03:00 4 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:03:00 hours
d01 2024-04-19_07:03:00 MAX AT i,j,k: 193 192 28 vert_cfl,w,d(eta)= 2.12504053 29.3598995 1.87329054E-02
d01 2024-04-19_07:03:00 5 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:03:00 hours
d01 2024-04-19_07:03:00 MAX AT i,j,k: 193 192 28 vert_cfl,w,d(eta)= 2.13372540 28.9528923 1.87329054E-02
d01 2024-04-19_07:03:36 5 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:03:36 hours
d01 2024-04-19_07:03:36 MAX AT i,j,k: 193 192 29 vert_cfl,w,d(eta)= 2.17697453 29.2024059 1.74069554E-02
d01 2024-04-19_07:03:36 6 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:03:36 hours
d01 2024-04-19_07:03:36 MAX AT i,j,k: 193 192 29 vert_cfl,w,d(eta)= 2.22462153 30.0599709 1.74069554E-02
d01 2024-04-19_07:03:36 6 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:03:36 hours
d01 2024-04-19_07:03:36 MAX AT i,j,k: 193 192 29 vert_cfl,w,d(eta)= 2.22312737 29.6857796 1.74069554E-02
d01 2024-04-19_07:04:12 7 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:04:12 hours
d01 2024-04-19_07:04:12 MAX AT i,j,k: 193 192 30 vert_cfl,w,d(eta)= 2.24796057 29.9699535 1.61748379E-02
d01 2024-04-19_07:04:12 6 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:04:12 hours
d01 2024-04-19_07:04:12 MAX AT i,j,k: 193 192 30 vert_cfl,w,d(eta)= 2.26332855 30.1286755 1.61748379E-02
........................
"
The timestep is 36s(9km),12s(3km),4s(1km).
The namelist.input is attached! Thanks you!

 

[haiqingsong]

Hello!
I am trying to run a case with 9km resolution and 3 nests.But it crashed after run serval hours.The error is:
"
d01 2024-04-19_07:02:24 1 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:02:24 hours
d01 2024-04-19_07:02:24 MAX AT i,j,k: 193 192 27 vert_cfl,w,d(eta)= 2.00234652 27.6386280 2.01599300E-02
d01 2024-04-19_07:03:00 3 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:03:00 hours
d01 2024-04-19_07:03:00 MAX AT i,j,k: 193 192 27 vert_cfl,w,d(eta)= 2.08140683 28.6493530 2.01599300E-02
d01 2024-04-19_07:03:00 4 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:03:00 hours
d01 2024-04-19_07:03:00 MAX AT i,j,k: 193 192 28 vert_cfl,w,d(eta)= 2.12504053 29.3598995 1.87329054E-02
d01 2024-04-19_07:03:00 5 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:03:00 hours
d01 2024-04-19_07:03:00 MAX AT i,j,k: 193 192 28 vert_cfl,w,d(eta)= 2.13372540 28.9528923 1.87329054E-02
d01 2024-04-19_07:03:36 5 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:03:36 hours
d01 2024-04-19_07:03:36 MAX AT i,j,k: 193 192 29 vert_cfl,w,d(eta)= 2.17697453 29.2024059 1.74069554E-02
d01 2024-04-19_07:03:36 6 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:03:36 hours
d01 2024-04-19_07:03:36 MAX AT i,j,k: 193 192 29 vert_cfl,w,d(eta)= 2.22462153 30.0599709 1.74069554E-02
d01 2024-04-19_07:03:36 6 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:03:36 hours
d01 2024-04-19_07:03:36 MAX AT i,j,k: 193 192 29 vert_cfl,w,d(eta)= 2.22312737 29.6857796 1.74069554E-02
d01 2024-04-19_07:04:12 7 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:04:12 hours
d01 2024-04-19_07:04:12 MAX AT i,j,k: 193 192 30 vert_cfl,w,d(eta)= 2.24796057 29.9699535 1.61748379E-02
d01 2024-04-19_07:04:12 6 points exceeded cfl=2 in domain d01 at time 2024-04-19_07:04:12 hours
d01 2024-04-19_07:04:12 MAX AT i,j,k: 193 192 30 vert_cfl,w,d(eta)= 2.26332855 30.1286755 1.61748379E-02
........................
"
The timestep is 36s(9km),12s(3km),4s(1km).
The namelist.input is attached! Thanks you!

timestep=9s or less is recommended.

 

[Tom]

timestep=9s or less is recommended.

Thank for you reply!
It means that the d01 (9km) ues timestep=9s?.In that case ,the timestep = 1 X dx.Is this number too small?

 

[haiqingsong]

Thank for you reply!
It means that the d01 (9km) ues timestep=9s?.In that case ,the timestep = 1 X dx.Is this number too small?

Yes. Since your inner nests has a resolution of 1km, there is a high probability of non-convergence. The time step of the D01 in 9S may also be too long.

 

[Tom]

Yes. Since your inner nests has a resolution of 1km, there is a high probability of non-convergence. The time step of the D01 in 9S may also be too long.

Thank for you qiuck reply!
But there are some papers also have the same resolution and its timestep=30s.

 

[kwerner]

@Tom
The recommendation is to use a timestep of 6xDX; however, when you get CFL errors, you may need to reduce that some - probably not as low as 1xDX. Take a look at What is the most common reason for a segmentation fault? which discusses CFL errors.

 

[Pechudin]

What do your vertical levels look like and what's the terrain like? I once got a CFL violation on a coarse (9 km) domain using dt=45 s when using a very fine vertical grid (~ 20 m dz_bottom and 40 levels up to 2 km height).

 

[Ming Chen]

By reducing time step to 4xdx, turning on w_damping and increasing the value of epssm, we expect that the case can run successfully.
Please keep us updated how it works. CFL violation is common and we would like to gain more experiences how to overcome it.

 

[hesue]

What do your vertical levels look like and what's the terrain like? I once got a CFL violation on a coarse (9 km) domain using dt=45 s when using a very fine vertical grid (~ 20 m dz_bottom and 40 levels up to 2 km height).

I also want to try to use a finer vertical levels, but the eta_levels setting is always not as good as it should be, I'd like to know how you designed it, what were the problems you encountered with the design, and have the problems been solved?

 

[Pechudin]

I also want to try to use a finer vertical levels, but the eta_levels setting is always not as good as it should be, I'd like to know how you designed it, what were the problems you encountered with the design, and have the problems been solved?

What do you mean by "not good enough vertical levels"? As for my situation, I increased the epssm parameter which helps damp the vertical motions.

Now, the real problem is that it is not good that the numerical grid has too large of a ratio between dx and dz (paper 1). This is a bit of a problem when using one vertical grid for several nests, and the smallest grid has a much smaller dx than the largest grid (for example, when nesting LES domain into several mesoscale domains).

There is an option to also use different vertical grids for each domain (read the manual for that). I intend to use it since it would allow to use larger dz on larger grids and smaller dz on smaller grids.

Paper 1 doi, describing issues with large dx/dz ratios: 10.1175/MWR-D-16-0049.1

 

[hesue]

By reducing time step to 4xdx, turning on w_damping and increasing the value of epssm, we expect that the case can run successfully.
Please keep us updated how it works. CFL violation is common and we would like to gain more experiences how to overcome it.

Specific Situation Explanation
In the WPS settings, I modified the terrain dataset and the land use type dataset. The innermost domain uses 3-second terrain data, and all domains use high-resolution land use data: Global 30-m land-cover dynamic monitoring products with a fine classification system from 1985 to 2020 (GLC_FCS30-1985_2020). The remaining datasets use default datasets.
#geog_data_res = '2m+GLC_FCS30', '30s+GLC_FCS30', 'QH_GDEM_3s+GLC_FCS30'

I aimed to study near-surface conditions. In the INPUT settings, I set more layers and adjusted related parameters to increase the vertical resolution near the ground surface to capture some wind field information. However, an error occurred at the beginning of the run, which is a CFL error (Fig1). This issue caused the computation window to remain on the running interface (Fig2), and the size of the output file (wrfout_d03) remained at 96.52M (Fig3). Upon consulting relevant materials and experience-sharing posts, I learned that the issue may be due to errors in the input data, steep terrain, or very strong convection. I have tried six different approaches to address the problem, but it persists (see namelist.input for specific settings). The wrfinput data can be as seen in the wrfinput_d03 variable（Fig4-6). I'm interested in knowing what other aspects I could investigate to uncover potential issues and their solutions. If you've encountered and overcome similar problems or have relevant information, kindly share it with me. It would be helpful. I would be very grateful.


Fig.1 CFL error


Fig.2

Fig.3


some wrfinput_d03 variable

Fig.4 HGT

Fig.5 LU_INDEX

Fig.6 LANDUSEF

The following measures have been attempted, but the problem persists:
① Memory release: Entered the command ulimit -s unlimited in the terminal.
② Reducing time step: Decreased the time_step to 4dx or 3dx.
③ Reducing vertical levels: For example, set e_vert=50.
④ Added smooth_cg_topo = .true.
⑤ Set epssm = 0.2 (maximum 0.5).
⑥ Set w_damping = 1.

 

[haiqingsong]

Specific Situation Explanation
In the WPS settings, I modified the terrain dataset and the land use type dataset. The innermost domain uses 3-second terrain data, and all domains use high-resolution land use data: Global 30-m land-cover dynamic monitoring products with a fine classification system from 1985 to 2020 (GLC_FCS30-1985_2020). The remaining datasets use default datasets.
#geog_data_res = '2m+GLC_FCS30', '30s+GLC_FCS30', 'QH_GDEM_3s+GLC_FCS30'

I aimed to study near-surface conditions. In the INPUT settings, I set more layers and adjusted related parameters to increase the vertical resolution near the ground surface to capture some wind field information. However, an error occurred at the beginning of the run, which is a CFL error (Fig1). This issue caused the computation window to remain on the running interface (Fig2), and the size of the output file (wrfout_d03) remained at 96.52M (Fig3). Upon consulting relevant materials and experience-sharing posts, I learned that the issue may be due to errors in the input data, steep terrain, or very strong convection. I have tried six different approaches to address the problem, but it persists (see namelist.input for specific settings). The wrfinput data can be as seen in the wrfinput_d03 variable（Fig4-6). I'm interested in knowing what other aspects I could investigate to uncover potential issues and their solutions. If you've encountered and overcome similar problems or have relevant information, kindly share it with me. It would be helpful. I would be very grateful.

View attachment 15701
Fig.1 CFL error

View attachment 15702
Fig.2
View attachment 15703
Fig.3


some wrfinput_d03 variable
View attachment 15704
Fig.4 HGT
View attachment 15705
Fig.5 LU_INDEX
View attachment 15706
Fig.6 LANDUSEF

The following measures have been attempted, but the problem persists:
① Memory release: Entered the command ulimit -s unlimited in the terminal.
② Reducing time step: Decreased the time_step to 4dx or 3dx.
③ Reducing vertical levels: For example, set e_vert=50.
④ Added smooth_cg_topo = .true.
⑤ Set epssm = 0.2 (maximum 0.5).
⑥ Set w_damping = 1.

Reducing your time step: Decreased the time_step to 1dx or 1s.
Have a try.

 

[hesue]

What do you mean by "not good enough vertical levels"? As for my situation, I increased the epssm parameter which helps damp the vertical motions.

Now, the real problem is that it is not good that the numerical grid has too large of a ratio between dx and dz (paper 1). This is a bit of a problem when using one vertical grid for several nests, and the smallest grid has a much smaller dx than the largest grid (for example, when nesting LES domain into several mesoscale domains).

There is an option to also use different vertical grids for each domain (read the manual for that). I intend to use it since it would allow to use larger dz on larger grids and smaller dz on smaller grids.

Paper 1 doi, describing issues with large dx/dz ratios: 10.1175/MWR-D-16-0049.1

I encountered a CFL error while running the model and initially thought it was due to an inappropriate setting of vertical levels. However, after many attempts at debugging, I still couldn't get it to run successfully. The specific issues have been presented above. Additionally, thank you very much for sharing literature; I have downloaded and partially read it, and it seems like it should be somewhat helpful.

 

[hesue]

Reducing your time step: Decreased the time_step to 1dx or 1s.
Have a try.

Thanks for your suggestion, I tried to set time_step = 9 and entered the command "ulimit -s unlimited" in the terminal, the rsl.error.0000 has no CFL, but has the running file size does not change at all. Based on that, I'll stop trying to set time_steps=1

 

[hesue]

Reducing your time step: Decreased the time_step to 1dx or 1s.
Have a try.

Thank you very much, it ran successfully. Yesterday I didn't wait long enough after running it and today I noticed that the output file size was 8.60 GB. I will wait for the WRF to finish running and check the simulation results.

 

[Ming Chen]

@hesue

Hi,

I have a few concerns for this case:

(1) In your namelist.input, you set

sf_surface_physics = 3, 3, 3, which is the RUC land surface module. However, you set

num_soil_layers = 4, which should be 6 or 9 for RUC.

(2) I would suggest that turn off cumulus scheme for child domains D02 and D03, i..e,

dx = 9000, 3000, 1000

cu_physics = 5, 0, 0

(3) It is better to set the same radiation calculation period for all domains, i..e,

radt = 9, 9, 9

(4) For a simulation of 8-day period, it is helpful to update SST, i.e.,

sst_update =1

Please modify your namelist options and try again.