Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

Request to add the support for seperate netcdf and netcdf-fortran path

J

jianglizhi

New member
Hello, I would like to install the wrf using seperate netcdf-c and netcdf-fortran library, since these libraries are installed seperately in our servers.
In order to complete the configuration process. I have to add the flag -lnetcdff and the path of netcdf-fortran into the file configure.wrf manually if I wanted to use a newer version of netcdf. As it is known, the netcdf-c and netcdf-fortran are two seperately libraries after the verson of netcdf-4.1.1 or so. It's less likely for me to use an ancient verson of netcdf in order to avoid add the NETCDF-fortran path to configuration file manually.
So, my urge is that could you please to add support of netcdf-fortran environmental value path? Thank you.
 
Hello islas, Thank you for your reply. That sounds good. Actually, that issue occurred when I was trying to compile WRF-4.6.0, so perhaps PR #1923 has not been applied to the new version.
Thank you.
 
While #1923 did go into the latest versions, some further testing shows that the PR did not fully pass in the path to an alternate netcdf c location. Have you been able to test the ./configure_new and ./compile_new options of building to see if that works for your use case?
 
Hello Islas,
Thank you for your advice. I'm sorry for not finishing the test. I tried using the new configure kits, but I found the frequent prompts for library paths a bit challenging. It became less convenient than I had expected. Setting up the environment for the dependent libraries is already quite time-consuming. When the interactive window asked for the library paths a third time, I eventually decided to give up. 😢 At the moment, the legacy configure option seems more manageable.
 
I'm sorry to hear that. Though, I am a little confused as well, since there shouldn't be any prompts for library paths in the new configuration as it should require even less environment setup than the legacy configuration. There should only be yes/no ([Y/n] or [y/N]) prompts to enable or disable available features, aside from selecting a core, nesting, and potentially case option:
Code: 
./configure_new 
Using default build directory : _build
Using default install directory : /home/aislas/wrf-model/wrf/install
0   Linux         pgf90       /    gcc         / !! mpif90      / !! mpicc      PGI (pgf90/gcc)
1   Linux         pgf90       /    pgcc        /    pgf90       /    pgcc       PGI (pgf90/pgcc): SGI MPT
2   Linux         pgf90       /    gcc         / !! mpif90      / !! mpicc      PGI (pgf90/gcc): PGI accelerator
3   Linux         gfortran    /    gcc         / !! mpif90      / !! mpicc      GNU (gfortran/gcc)
4   Linux         pgf90       /    pgcc        / !! mpif90      / !! mpicc      PGI (pgf90/pgcc)
5   Linux         pgf90       /    gcc         / !! mpif90      / !! mpicc      PGI (pgf90/gcc): -f90=pgf90
6   Linux         pgf90       /    pgcc        / !! mpif90      / !! mpicc      PGI (pgf90/pgcc): -f90=pgf90
!! - Compiler not found, some configurations will not work and will be hidden
Select configuration [0-6] Default [0] (note !!)  : 3
Select option for WRF_CORE from WRF_CORE_OPTIONS [0-4] 
        0 : ARW
        1 : CONVERT
        2 : DA
        3 : DA_4D_VAR
        4 : PLUS 
Default [0] : 
Select option for WRF_NESTING from WRF_NESTING_OPTIONS [0-3] 
        0 : NONE
        1 : BASIC
        2 : MOVES
        3 : VORTEX 
Default [1] : 
Select option for WRF_CASE from WRF_CASE_OPTIONS [0-13] 
        0 : EM_REAL
        1 : EM_FIRE
        2 : EM_SCM_XY
        3 : EM_TROPICAL_CYCLONE
        4 : EM_HELDSUAREZ
        5 : EM_B_WAVE
        6 : EM_GRAV2D_X
        7 : EM_HILL2D_X
        8 : EM_LES
        9 : EM_QUARTER_SS
        10 : EM_SEABREEZE2D_X
        11 : EM_CONVRAD
        12 : EM_SQUALL2D_X
        13 : EM_SQUALL2D_Y 
Default [0] : 0
[SM] Use OpenMP? Default [N] [y/N] : 
Configure additional options? Default [N] [y/N] : 
-- Retrieving git information...
-- git SHA  : v4.6.0-32-ge0a5ee40e9cd9cf12ea2f48bcd7e9596c0050e7c
-- git diff : 
-- Setting project version to 4.6.0
-- Set default build type to Release
-- netCDF large file support not suppressed, if available it will be used
-- Found NETCDF_PROGRAM : /home/aislas/dependencies/bin/nc-config
-- Found NETCDF-FORTRAN_PROGRAM : /home/aislas/dependencies/bin/nf-config
-- No pnetcdf-config found : PNETCDF_PROGRAM-NOTFOUND
-- WRF_CONFIG           : _build/wrf_config.cmake
-- CMAKE_BUILD_TYPE     : Release
-- WRF_CORE             : ARW
-- WRF_NESTING          : BASIC
-- WRF_CASE             : EM_REAL
-- USE_DOUBLE           : OFF
-- USE_MPI              : OFF
-- USE_OPENMP           : OFF
-- USE_IPO              : OFF
-- ENABLE_CHEM          : OFF
-- ENABLE_CLM           : 
-- ENABLE_CMAQ          : OFF
-- ENABLE_DFI_RADAR     : OFF
-- ENABLE_HYDRO         : OFF
-- ENABLE_KPP           : OFF
-- ENABLE_MARS          : OFF
-- ENABLE_TERRAIN       : OFF
-- ENABLE_TITAN         : OFF
-- ENABLE_VENUS         : OFF
-- USE_ALLOCATABLES     : ON
-- wrfmodel             : ON
-- GRIB1                : ON
-- INTIO                : ON
-- KEEP_INT_AROUND      : ON
-- LIMIT_ARGS           : ON
-- FORCE_NETCDF_CLASSIC  : OFF
-- BUILD_RRTMG_FAST     : OFF
-- BUILD_RRTMK          : OFF
-- BUILD_SBM_FAST       : ON
-- SHOW_ALL_VARS_USED   : OFF
-- WRFIO_NCD_NO_LARGE_FILE_SUPPORT      : OFF
-- Setting gen_comms to RSL_LITE
-- Setting module_dm to RSL_LITE
-- Configuring done (2.3s)
-- Generating done (0.2s)
-- Build files have been written to: /home/aislas/wrf-model/wrf/_build
 
Hello islas,
Thank you for your reply.
I apologize for the mistakes in my previous posts. I followed your instructions for configure_new and compile_new_again, and I successfully completed all the steps. My previous mistake occurred in step 3 during the additional options stage. I mistakenly thought that the configure_program asked for the full path of each library. When I saw a list of "ENABLE_xxx" options, I quitted because of the scare of having to fulfill each library manually.

After your guidance, I tried again and realized that I only needed to type the option numbers without providing the library paths.I then used the compile_new kits, which ran well and indicated the building progress. I am pleased to say that the new version is more convenient than the old one. :)

I am curious about how CMake finds the libraries. If there are different versions of the same libraries, how can I ensure that the program finds them correctly?
I also tested the ./compile_new -j option, which can break through the limit of a maximum of 20 cores, and I found it very useful.

Thank you!
 
Right, the additional options aggregates all the available but not common WRF options which can be overwhelming.

Finding the libraries depends on the whether it is natively supported by CMake or not. For instance, MPI is natively supported so we use FindMPI — CMake 3.30.5 Documentation to locate the wrapper and libraries. Most searches use what is natively in your environment with a priority list given via find_package() (find_package — CMake 3.30.5 Documentation).

This is mostly very CMake-esoteric, so the gist is for well supported common things (MPI, flex, zlib, curl, hdf5) if it is installed in a common system location it will be found. You can control where CMake looks by generally using <PackageName>_ROOT as an option to ./configure_new such as ./configure_new -- -DZLIB_ROOT=<my_path>. This works well if different versions are installed in different searchable directories. If they aren't... I'm not sure how to specifically select one without going into the source code and setting a required version (e.g. find_package( ZLIB 1.3.1))

You might see that in the source there is now a cmake/modules/ folder. This handles finding libraries not natively or well supported by CMake. As an example, if you want to select a different netCDF (Fortran or C) you can still use the above *_ROOT style (...-DnetCDF_ROOT=<my path>) or (my preferred method) just make sure the version you want is listed first in you PATH, that is the nc-config and nf-config you use on the command line is what CMake will use. If you change your PATH to use a different version, the search will use that selected version. In fact, in the example output I posted you can see that netCDF is not in a standard install path, and all I have set in my environment is PATH=~/dependencies/bin:...<rest of system path>... where nc-config/nf-config are installed there. The classic NETCDF variable is not used anymore, but if that's something we get feedback on wanting supported again it can be re-evaluated.

Note: netCDF_ROOT and netCDF-Fortran_ROOT control netCDF and netCDF Fortran, respectively. Likewise, the paths for nc-config and nf-config do not need to match which is why I proposed this as an alternate solution since the new build configuration treats the two separately.
 
islas said:
Right, the additional options aggregates all the available but not common WRF options which can be overwhelming.

Finding the libraries depends on the whether it is natively supported by CMake or not. For instance, MPI is natively supported so we use FindMPI — CMake 3.30.5 Documentation to locate the wrapper and libraries. Most searches use what is natively in your environment with a priority list given via find_package() (find_package — CMake 3.30.5 Documentation).

This is mostly very CMake-esoteric, so the gist is for well supported common things (MPI, flex, zlib, curl, hdf5) if it is installed in a common system location it will be found. You can control where CMake looks by generally using <PackageName>_ROOT as an option to ./configure_new such as ./configure_new -- -DZLIB_ROOT=<my_path>. This works well if different versions are installed in different searchable directories. If they aren't... I'm not sure how to specifically select one without going into the source code and setting a required version (e.g. find_package( ZLIB 1.3.1))

You might see that in the source there is now a cmake/modules/ folder. This handles finding libraries not natively or well supported by CMake. As an example, if you want to select a different netCDF (Fortran or C) you can still use the above *_ROOT style (...-DnetCDF_ROOT=<my path>) or (my preferred method) just make sure the version you want is listed first in you PATH, that is the nc-config and nf-config you use on the command line is what CMake will use. If you change your PATH to use a different version, the search will use that selected version. In fact, in the example output I posted you can see that netCDF is not in a standard install path, and all I have set in my environment is PATH=~/dependencies/bin:...<rest of system path>... where nc-config/nf-config are installed there. The classic NETCDF variable is not used anymore, but if that's something we get feedback on wanting supported again it can be re-evaluated.

Note: netCDF_ROOT and netCDF-Fortran_ROOT control netCDF and netCDF Fortran, respectively. Likewise, the paths for nc-config and nf-config do not need to match which is why I proposed this as an alternate solution since the new build configuration treats the two separately.
Click to expand...
Thank you islas,

I had read your reply. And I looked up for the FindnetCDF file in the cmake/modules folder. As you said, the cmake use the output of default nc-config and nf-config to find out the path of netCDF libraries. This is a highly intuitive approach that is suitable for most users. If some advanced users need to set custom paths for other libraries, they are likely to know how to configure the cmake option. So this script is very clever!

The only feedback/requirement is that it could be better to tell the user the name of the environment variable of the dependency libraries, such as netCDF instead of NETCDF. For me, when compiling some software using cmake, I always struggle to distinguish between environment variables such as -DCMAKE_XXX_PATH, -DCMAKE_XXX_PREFIX, and -DCMAKE_XXX_ROOT, which usually leads to the inability to pass the path of the required dependency libraries. Maybe this can be done by printing the names of environment variables in an interactive window when configuring the library during the run of configure.
 
You must log in or register to reply here.
Top