WRF missing dependencies in makefile and parallel make

[youyu3]

Hi. I've been building WRF using parallel make (like `make -j 10` and `make -j 20`). Looks like some source files use modules from other source files but the dependences are not setup correctly in the makefile. This causes intermittent build failures due to missing module files. Like in `phys` I might get `Unable to open MODULE file module_mp_thompson.mod (module_madwrf.f90: 243)` because `module_mp_thompson` and `module_madwrf` are being compiled simultaneously (WRF/phys/Makefile at master · wrf-model/WRF).

I see that the default is `-j 2` in the `compile` script, but what's the best practice for parallel making of the various subdirectories?
Thanks!

 

[kwerner]

Hi,
The advantage to compiling the model with multiple processors is to simply compile it faster. In testing this, we have found that the time is reduced "significantly" up to about 4-6 processors, at which point the time reduced is no longer very great (see slide 20 in this presentation). It is also true that sometimes compiling with multiple processors can cause issues as you're seeing. I would recommend doing this with no more than 6 processors, and if you still see issues, reduce that number. Occasionally it's necessary to only compile with a single processor to ensure all files are built in the correct order.

 

[youyu3]

Thanks for the explanation and the presentation, Kelly. In my case, there is a big speedup from 2 to 6 processers.

The speedup from 6 to 40 is pretty small. I tried adding the missing dependencies in configure.wrf, which seemed to fix the failures I had before.