Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

Multiple WRF simulations from same system via different set of cores

Tanmoy

New member
Hello WRF community,

This is just a general question. But can we run 2 simulations simultaneously, if same version of WRF is copied from current folder to another folder.
What I mean is that I already have a WRF directory, what if I copied it in a separate directory and to do a separate simulations. Will it work??? I'm currently not using the all the cores of my system, so I want to utilize it also. If it will work, what changes should I do ???
 
I usually use the template directory created by soft links (ln -sf) for instance simulation, and when using it, I copy a copy and put the namelist.input into it (and then "ln -sf" met file).
As shown in the figure, running multiple instances simultaneously can be achieved by copying multiple working directories.
1753255602075.png
1753255574060.png
 
Does it help to run two simulations at the same time? I mean, is there any risk of the system crashing, as WRF is memory-intensive? Additionally, does it expedite the time required for multiple simulations? I mean, let's say one normal simulation would take 1.5 days, then 2 normal simulations would take 3 days. Does this method reduce the time to 3 days? (thThis is just an example)
I usually use the template directory created by soft links (ln -sf) for instance simulation, and when using it, I copy a copy and put the namelist.input into it (and then "ln -sf" met file).
As shown in the figure, running multiple instances simultaneously can be achieved by copying multiple working directories.
View attachment 18575
View attachment 18574
 
Does it help to run two simulations at the same time? I mean, is there any risk of the system crashing, as WRF is memory-intensive? Additionally, does it expedite the time required for multiple simulations? I mean, let's say one normal simulation would take 1.5 days, then 2 normal simulations would take 3 days. Does this method reduce the time to 3 days? (thThis is just an example)
Running two WRF simulations simultaneously can lead to resource contention (I/O , memory bandwidth or other aspects), causing each 32‑core job to run slightly slower than if executed alone. (Note that it is a single node)

However, it is generally more time‑efficient to split the available CPUs between multiple simulations rather than allocate all CPUs to a single case sequentially. (For multitasking situations) This is because WRF’s parallel efficiency decreases as the number of cores increases — for example, a case that takes 1 day on 32 cores may not finish in 0.5 days on 64 cores, but perhaps in 0.7 days due to communication overhead. When there are two similar tasks, one 64‑core job running sequentially would take roughly 1.4 days in total, while running two 32‑core jobs in parallel might complete both in about 1.2 days, despite some slowdown per job.

Therefore, if you have multiple cases to run, running them concurrently typically results in a shorter total wall‑clock time, even though each job individually may run slightly slower.
 
The actual performance depends on many factors, so it is best to benchmark in your own system (for example, to determine how many simultaneous cases yield the best overall performance). A short test run is usually enough — you can use the rsl output over a brief simulation period to estimate performance before committing to long runs.
 
Dear Hans,

Thank you for your kind input and help. It has been a very helpful conversation and I hope this also helps any new WRF users in the future also.

Best Wishes,
Tanmoy.
The actual performance depends on many factors, so it is best to benchmark in your own system (for example, to determine how many simultaneous cases yield the best overall performance). A short test run is usually enough — you can use the rsl output over a brief simulation period to estimate performance before committing to long runs.
 
Top