WRF-Fire crashes at start when adding nested domains (d02+d03). d01-only runs, but first ignition doesn’t start.

[Jesufemi Olowe]

Hello everyone,
I’m running a WRF-Fire simulation and I’m stuck with two related issues:

1) Model crashes when I include nested domains (d02 and d03)​

• d01-only: runs successfully.
• d01 + d02 + d03: crashes very early (near the beginning of the run).

Domain setup:

• d02 and d03 are both nested inside d01 (they are separate child domains, not one inside the other).
• (I will attach my namelist.input)

MPI / decomposition:

• For d01-only, I typically use a decomposition like nproc_x=8, nproc_y=6 (48 ranks) and it runs.
• For d01+d02+d03, I have tried 48, 64, and 128 MPI ranks but it still crashes at the beginning.
• The crash happens regardless of the rank count I tried.

I have attached:

• the rsl.error.* and rsl.out.* from the most recent failing run (128 ranks),
• the full namelist.input

2) In d01-only runs, first fire does not ignite but second fire burns​

In the d01-only case (which runs), I have two ignitions configured:

• Ignition 1 should start at ~6.5 hours, but it never ignites.
• Ignition 2 start 14.3 hrs but ignites and burns as expected.

So I’m wondering:

• Could there be an issue with the ignition timing (start time / run time / conversion), ignition coordinates, or ignition line definition?

Questions​

1. What are the most common reasons WRF-Fire runs fine on d01-only but crashes immediately when adding nested domains?
2. Are there recommended checks for nest configuration (parent/child ratios, i/j start locations, time step ratios, feedback settings, etc.) specific to WRF-Fire?
3. For the ignition problem: what are the typical reasons an ignition scheduled at 6.5 hours never starts, while another ignition does?

Thanks a lot for any help—happy to provide any additional diagnostics.

 

[William.Hatheway]

Hello everyone,
I’m running a WRF-Fire simulation and I’m stuck with two related issues:

1) Model crashes when I include nested domains (d02 and d03)​

• d01-only: runs successfully.
• d01 + d02 + d03: crashes very early (near the beginning of the run).

Domain setup:

• d02 and d03 are both nested inside d01 (they are separate child domains, not one inside the other).
• (I will attach my namelist.input)

MPI / decomposition:

• For d01-only, I typically use a decomposition like nproc_x=8, nproc_y=6 (48 ranks) and it runs.
• For d01+d02+d03, I have tried 48, 64, and 128 MPI ranks but it still crashes at the beginning.
• The crash happens regardless of the rank count I tried.

I have attached:

• the rsl.error.* and rsl.out.* from the most recent failing run (128 ranks),
• the full namelist.input

2) In d01-only runs, first fire does not ignite but second fire burns​

In the d01-only case (which runs), I have two ignitions configured:

• Ignition 1 should start at ~6.5 hours, but it never ignites.
• Ignition 2 start 14.3 hrs but ignites and burns as expected.

So I’m wondering:

• Could there be an issue with the ignition timing (start time / run time / conversion), ignition coordinates, or ignition line definition?

Questions​

1. What are the most common reasons WRF-Fire runs fine on d01-only but crashes immediately when adding nested domains?
2. Are there recommended checks for nest configuration (parent/child ratios, i/j start locations, time step ratios, feedback settings, etc.) specific to WRF-Fire?
3. For the ignition problem: what are the typical reasons an ignition scheduled at 6.5 hours never starts, while another ignition does?

Thanks a lot for any help—happy to provide any additional diagnostics.

@JeremyB
any ideas?

 

[kwerner]

What are the most common reasons WRF-Fire runs fine on d01-only but crashes immediately when adding nested domains?

There isn't a nesting-specific reason for this, but typically when the model stops immediately, it's due to the number of processors (too few or too many - see Choosing an Appropriate Number of Processors for details), CFL errors, bad input data, and/or a bad domain configuration. I did find CFL errors in your rsl* files, which indicates the model becomes unstable (see Segmentation Faults and CFL Errors).

The domain configuration may be contributing to issues, as well. For WRF, it is recommended that the parent_grid_ratio is no greater than 5:1, and you're using a 9:1 ratio. This recommended ratio also includes the resolution between the input data (assuming you're running a real-data case) and your parent domain. Since your parent domain - d01 - is set to 1km, that means the input data shouldn't be much coarser than about 5km - though this doesn't have to be exact. See WRF and WPS Best Practices for details on setting up a reasonable domain.

1. Are there recommended checks for nest configuration (parent/child ratios, i/j start locations, time step ratios, feedback settings, etc.) specific to WRF-Fire?
2. For the ignition problem: what are the typical reasons an ignition scheduled at 6.5 hours never starts, while another ignition does?

I'll need to reach out to someone else to answer the WRF-fire specific questions, as I'm not trained in the fire version of the model. I'll either get back to your with their response, or they may respond directly.

 

[frediani]

I'm not aware of WRF-Fire being run in multiple nests concurrently, so there's a chance it just won't work. From a quick look in the source code, I don't see why it shouldn't work, but the fire code is clunky, so I wouldn't be surprised if it doesn't.

I'd suggest you try your nesting configuration with the fire turned off (set all your ifire columns in the namelist to 0). If that crashes, the issue is not in the fire code. If you are able to run the atmosphere in that configuration but not the fires, you can use ndown to simulate multiple fires from more than one nested domain.

Also, from your namelist, it looks like you're trying to simulate the same fire in the nest and parent (i.e. same ignition location). If the multiple-domain fire configuration works, you'd be double counting the fire fluxes into the atmosphere. It'd be better to have no fire in the parent, and fires igniting in the separate nests. But again, I'm not aware we've ever tested this.