Regarding Decomposition for large number of processors.

[Kazim Hussain]

I am doing a LES simulation nested in a coarser grid. the result is such that outer domain has much fewer number of grid cells as compared to the nested domain. This brings up a problem in the decomposition since the variables nproc_x , nproc_y are single entries, I cannot attribute more resources to the finer domain(Ideally attributing one processor for every 10 x 10 pixel in the group). I want to know if there is any workaround this other than the idea of using ndown, because with ndown the temporal frequency of the data cannot match the continuous forcing in a concurrent run, additionally I don't have enough space to save the data at these frequency requirements.
I would also like to point out that automatic decomposition(atleast in WRF 4.3) does not optimally distribute nproc_x, nproc_y for the given number of processors even when the stencil requirement of atleast 10 pixels in each direction can be satisfied. It should be a relatively small part to add, but it is not an issue since you give us the option to manually enter npoc_x , nproc_y . If these variables would be max_dom entries then it would be possible for me to run my simulation with a large number of processors.

In my situation I can use only 1456 processors (nproc_x * nproc_y = 56 x 26 =1456 tiles for coarser domain to meet stencil requirement) but the inner domain could do well using 64 x 64 tiles and hence 4096 processors, otherwise my simulation is very slow and infeasible
e_we = 566, 643,
e_sn = 261, 643,

Can I get help in this matter?

 

[kwerner]

Unfortunately the option to use ndown is the only work-around I know of for this issue. One thought is that you could make the e_sn setting larger for domain 01, even if you don't necessarily need that much domain. The bulk of the cost of computation is going to be taken by d02, so increasing the size of your d01 domain may not put that much stress on computation. If the size were larger, you could find a reasonable middle-ground for decomposition to run both domains together.