compiling WRF usinf the PHDF5 library

[francescomaicu]

Dear WRF supporters,
I recently upgraded to WRF4.4 to test the parallel netcdf writing (with quilting) of the outputs with the HDF5 compression.
I want to run WRF with io_form_history=13

Befor configuring WRF I load the following modules:

module purge
module load impi20.1/19.7.217
module load intel20.1/20.1.217
module load impi20.1/hdf5/1.12.0
module load impi20.1/netcdf/C_4.7.4-F_4.5.3_CXX_4.3.1
module load impi20.1/parallel-netcdf/1.12.3
module load intel20.1/szip/2.1.1
module load curl/7.66.0


export PHDF5="/zeus/opt/impi20.1/hdf5/1.12.0"
export I_MPI_EXTRA_FILESYSTEM=1
export I_MPI_EXTRA_FILESYSTEM_FORCE=gpfs

The HDF5 module loaded is already the parallel one:

 module show impi20.1/hdf5/1.12.0
-------------------------------------------------------------------
/zeus/opt/modulefiles-intel20.1/impi20.1/hdf5/1.12.0:

module-whatis    Name: HDF5 (Parallel version)
module-whatis    Version: 1.12.0
module-whatis    Category: library, runtime support
module-whatis    Keywords: I/0, Library
module-whatis    Description: General purpose library and file format for storing scientific data.
module-whatis    URL: http://www.hdfgroup.org/HDF5/
setenv           HDF5 /zeus/opt/impi20.1/hdf5/1.12.0
setenv           HDF5_DIR /zeus/opt/impi20.1/hdf5/1.12.0
prepend-path     PATH /zeus/opt/impi20.1/hdf5/1.12.0/bin
prepend-path     LD_LIBRARY_PATH /zeus/opt/impi20.1/hdf5/1.12.0/lib
prepend-path     LD_RUN_PATH /zeus/opt/impi20.1/hdf5/1.12.0/lib
-------------------------------------------------------------------

While running the configure procedure I see:

Will use NETCDF in dir: /zeus/opt/impi20.1/netcdf/C_4.7.4-F_4.5.3_CXX_4.3.1
Will use PNETCDF in dir: /zeus/opt/impi20.1/parallel-netcdf/1.12.3
Will use HDF5 in dir: /zeus/opt/impi20.1/hdf5/1.12.0
Will use PHDF5 in dir: /zeus/opt/impi20.1/hdf5/1.12.0

The configure.wrf obtained is attached, but I cannot see neither reference to PHDF5 path and referece to the external library in ./external/io_phdf5/

Initially, I set:

 nocolons                            = .true.
 io_form_history                     = 13
..
..
 nproc_x                             = 18
 nproc_y                             = 47
.
.
 nio_tasks_per_group = 8,
 nio_groups = 1,

and run with 854 cores (18*47+8*1), but at the time of writing the initial time, the run stops and rsl.err.0853 tells:

taskid: 853 hostname: n241
 module_io_quilt_old.F        2931 F
Quilting with   1 groups of   8 I/O tasks.
Server polling is does not work with one io group.  Disabled poll_servers.
-------------- FATAL CALLED ---------------
FATAL CALLED FROM FILE:  <stdin>  LINE:    2094
 quilt: bad tag:   1112817664  icurs           71
-------------------------------------------
Abort(1) on node 853 (rank 853 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, 1) - process 853

It seems to me it is a problem of quilting, so I decide to swith it off, (nio_tasks_per_group = 0), and re run.
WRF runs perfeclty, but nothing is written.
Please, can you tell me what am I missing?
Thank you very much.
Francesco

 

[kwerner]

Hi,
It looks like the PHDF5 option in WRF was contributed by an outside group, and written back in 2004. It's not a format our group supports. However, if you want to try to output your files in NetCDF4 with the parallel option, that should essentially be the same. An alternative option could be for you to try to contact the HDF group to obtain support for this. If you take a look at the external/io_phdf5/README.phdf5 file, there is some older contact information that may still work, or may be useful in helping you to contact the right people.