Is it possible to run da_wrfvar.exe on multicore machine?

[satjemkes]

Hi
i would like to know if it is possible to compile and run da_wrfvar.exe on a multi-core machine.
wrf version 4.3.3, standard gfortran, reasonable recent mpich

if i select open 34 (dmpar) for the configure and try to compile -j 8 all_wrfvar, the system will break during compilation of the crtm routine.
I can not see anything obviously wrong. It complains about a missing module, but this module is available. Hence there is something behind the scene which avoids the compilaiton with the -j 8 option.

i can compile (still with option 34 of configuration) without -j option.

running the code with mpiexec.hydra -n 4 even without the -j option compilation fires off 4-jobs and it appears that this is faster than when i complie the all_wrfvar using the serial option (32). (i have no exact timing information yet.)


Is the performance a linear function of the number of cores used for the calculations? -n 8 runs twice as fast as -n 4 ?
I have timing results for -n 8 and -n 16, but there seems little difference.
question is if there is anybody having experience with this.
any feedback would be appreciated

 

[Deleted member 3607]

Can you attach the log files and config files from when you try to install it with multiple cores?

 

[Deleted member 3607]

Hi
i would like to know if it is possible to compile and run da_wrfvar.exe on a multi-core machine.
wrf version 4.3.3, standard gfortran, reasonable recent mpich

if i select open 34 (dmpar) for the configure and try to compile -j 8 all_wrfvar, the system will break during compilation of the crtm routine.
I can not see anything obviously wrong. It complains about a missing module, but this module is available. Hence there is something behind the scene which avoids the compilaiton with the -j 8 option.

i can compile (still with option 34 of configuration) without -j option.

running the code with mpiexec.hydra -n 4 even without the -j option compilation fires off 4-jobs and it appears that this is faster than when i complie the all_wrfvar using the serial option (32). (i have no exact timing information yet.)


Is the performance a linear function of the number of cores used for the calculations? -n 8 runs twice as fast as -n 4 ?
I have timing results for -n 8 and -n 16, but there seems little difference.
question is if there is anybody having experience with this.
any feedback would be appreciated

As for your question about performance. I do not believe their is a linear correlation between number of cores and install time. But I may be mistaken.

@Ming Chen or @kwerner may know that answer

 

[satjemkes]

Hi Will,
What exactly are you looking for? is there a specific environmental variable i should be watching?
the config.txt is a cut&paste of the interactive configuration process which outputs to the screen.

hdf5-netcdf version are recently recent
and so is the mpich
when compiled with -j 16 then wrfvar is not compiled because of an error with the crtm module. I looked the module but nothing strange there.
after a ./clean -a twice (to ensure all is cleaned, a single instance is not sufficient)
and compiling without any -j option system compiles nicely, but it is not obvious that there I can take advantage of the multiple processors.
Is there any other way to increase performance? linking to a recent version of OpenBLAS (also installed)
We plan to do some experiments over an extended time period (at least one month) and it would be good if the wrfvar.exe part goes much faster than now (about 3 - 4 hours)
thanks
stephen