WRF 4DVAR v4.6.0 - Intel LLVM Compilers

[William.Hatheway]

Hello admins @islas @weiwang @kwerner , @mgduda

With the release of WRF v4.6.0, I have conducted a series of evaluations on the Weather Research and Forecasting Model (WRF) version 4.6.0, utilizing the Intel LLVM Compilers and associated libraries. The purpose of these tests was to assess the performance and compatibility of WRF v4.6.0 under the specified computational environment. Given the diverse nature of the issues encountered, I will be documenting each separately in subsequent communications, ensuring detailed discussions per topic. Attached to each of these communications, you will find a zip file containing all relevant logs.

System Specifications:

• Operating System: Ubuntu 22.04.4
• Memory: 64 GB DDR5 RAM
• Processor: Intel Core i9-13900K
• Storage: 1TB SSD

Compiler and Library Environment:

• Intel LLVM Compilers, Version 2024.1.0:
  • ifx (Fortran compiler)
  • icx (C compiler)
  • icpx (DPC++/C++ compiler)
  • mpiifx, mpiicx, mpiicpx (Intel MPI wrappers for respective compilers)
• Libraries:
  • HDF5 v1.14.4.2
  • PHDF5 v1.14.4.2
  • ZLIB v1.3.1
  • Libpng v1.6.39
  • Jasper v1.900.1
  • Pnetcdf v1.13.0
  • Netcdf-c v4.9.2
  • Netcdf-fortran v4.6.1

Preliminary Observations:The testing required additional compiler flags to address compatibility issues with Jasper and Libpng libraries, which appear to stem from advancements in compiler technology beyond the versions recommended by NCAR. These flags include:

• -Wno-implicit-function-declaration
• -Wno-incompatible-function-pointer-types
• -Wno-unused-command-line-argument

Methodology:Tests were conducted in both single-threaded and multi-threaded modes to evaluate performance scalability. The procedure involved:

1. Fresh installation of the specified packages and configuration in a new terminal.
2. Execution of the WRF model using the command ./compile with -j 1 for single-threaded and -j 16 (utilizing half of the available CPU threads) for multi-threaded scenarios.
3. Collection, archiving, and zipping of all log files and configurations post-testing.

Conclusion:The tests conducted in the single-threaded configuration for WRF 4DVAR were sucessful and multi-threaded configurations using WRF 4DVAR with Intel LLVM compilers were not sucessful.

In the log files for the multi-threaded test there appears to be a module or two that are not linking or building properly. This causes a cascading error that proprogates through the installation.

 

[JeremyB]

Hello William,

I got the same errors when compiling WRFDA with dmpar. I didn't bother trying serial since I need dmpar for what I'm using it for. Thank you for this in depth analysis of the software. I also tried it with rttov, and I couldn't get it working with that either. If you'd like me to upload my files I can, however they are very similar to what you have.

Thanks,
JeremyB

 

[JeremyB]

Hello,

Here are the files from my recent compilation attempt. Here are the commands I used to configure and compile as well

configuring:
./configure 4dvar
Option: 40

Compiling:
./compile -j 6 all_wrfvar >& compile_wrfvar.log &

The errors I received from running the command with and without the -j 6 option are very similar.

Please let me know if you need anything else and I would be happy to help out.


Thanks,
JeremyB

 

[William.Hatheway]

Hello,

Here are the files from my recent compilation attempt. Here are the commands I used to configure and compile as well

configuring:
./configure 4dvar
Option: 40

Compiling:
./compile -j 6 all_wrfvar >& compile_wrfvar.log &

The errors I received from running the command with and without the -j 6 option are very similar.

Please let me know if you need anything else and I would be happy to help out.


Thanks,
JeremyB

very helpful to see it was not just mine

 

[JeremyB]

Hello,

After further investigation, I was able to compile WRFDA without rttov using the new intel compilers. I believe my issue before may have been an incorrect link to icpx for cxx. I also compiled all the libraries with the intel compilers. There were issues with netcdf c when I ran make check, but disregarding those errors still yielded a compiled code. I have not tested it yet, however I was able to get da_wrfvar.exe

Thanks,
JeremyB

 

[William.Hatheway]

Hello,

Here are the files from my recent compilation attempt. Here are the commands I used to configure and compile as well

configuring:
./configure 4dvar
Option: 40

Compiling:
./compile -j 6 all_wrfvar >& compile_wrfvar.log &

The errors I received from running the command with and without the -j 6 option are very similar.

Please let me know if you need anything else and I would be happy to help out.


Thanks,
JeremyB

da_radiance.f(7): error #7013: This module file was not generated by any release of this compiler.   [HDF5]
   use hdf5
-------^
da_radiance.f(107): error #6457: This derived type name has not been declared.   [IV_TYPE]
   type (iv_type), intent(in)    :: iv          ! Innovation vector.
---------^
da_radiance.f(108): error #6457: This derived type name has not been declared.   [Y_TYPE]
   type (y_type) , intent(inout) :: re          ! Residual vector.
---------^
da_radiance.f(109): error #6457: This derived type name has not been declared.   [Y_TYPE]
   type (y_type) , intent(inout) :: jo_grad_y   ! Grad_y(Jo)
---------^
da_radiance.f(140): error #6457: This derived type name has not been declared.   [IV_TYPE]
   type (iv_type), intent(inout)  :: iv       ! O-B structure.
---------^
da_radiance.f(227): error #6457: This derived type name has not been declared.   [IV_TYPE]
   type (iv_type),  intent (inout) :: iv
---------^
da_radiance.f(232): error #6457: This derived type name has not been declared.   [DATALINK_TYPE]
   type (datalink_type) :: p
---------^
da_radiance.f(236): error #6457: This derived type name has not been declared.   [INFO_TYPE]
   type(info_type)       ::  info
--------^
da_radiance.f(237): error #6457: This derived type name has not been declared.   [MODEL_LOC_TYPE]
   type(model_loc_type)  ::  loc
--------^
da_radiance.f(337): error #6457: This derived type name has not been declared.   [Y_TYPE]
   type (y_type),     intent(in)  :: ob       ! Observation structure.
---------^
da_radiance.f(338): error #6457: This derived type name has not been declared.   [IV_TYPE]
   type (iv_type),    intent(in)  :: iv       ! O-B structure.
---------^
da_radiance.f(408): error #6457: This derived type name has not been declared.   [IV_TYPE]
   type (iv_type)    ,  intent (inout) :: iv
---------^
da_radiance.f(412): error #6683: A kind type parameter must be a compile-time constant.   [I_KIND]
   integer(i_kind), allocatable :: nread(:)
-----------^
da_radiance.f(414): error #6683: A kind type parameter must be a compile-time constant.   [I_KIND]
   integer(i_kind),parameter:: n1bhdr=15
-----------^
da_radiance.f(415): error #6683: A kind type parameter must be a compile-time constant.   [I_KIND]
   integer(i_kind),parameter:: n2bhdr=2
-----------^
da_radiance.f(416): error #6683: A kind type parameter must be a compile-time constant.   [I_KIND]
   integer(i_kind),parameter:: maxinfo=12
-----------^
da_radiance.f(417): error #6683: A kind type parameter must be a compile-time constant.   [I_KIND]
   integer(i_kind),parameter:: maxchanl=100
-----------^
da_radiance.f(427): error #6683: A kind type parameter must be a compile-time constant.   [I_KIND]
   integer(i_kind) ihh,i,j,k,ifov,idd,ireadmg,ireadsb
-----------^
da_radiance.f(428): error #6683: A kind type parameter must be a compile-time constant.   [I_KIND]
   integer(i_kind) iret,idate,im,iy,nchan
-----------^
da_radiance.f(433): error #6683: A kind type parameter must be a compile-time constant.   [I_KIND]
   integer(i_kind) itt,itx,iobs,iout
-----------^
da_radiance.f(434): error #6683: A kind type parameter must be a compile-time constant.   [R_KIND]
   real(r_kind) terrain,timedif,crit,dist
--------^
da_radiance.f(435): error #6683: A kind type parameter must be a compile-time constant.   [R_KIND]
   real(r_kind) dlon_earth,dlat_earth
--------^
da_radiance.f(437): error #6683: A kind type parameter must be a compile-time constant.   [R_KIND]
   real(r_kind) tbmin,tbmax, tbbad
--------^
da_radiance.f(438): error #6683: A kind type parameter must be a compile-time constant.   [R_KIND]
   real(r_kind) panglr,rato
--------^
da_radiance.f(440): error #6683: A kind type parameter must be a compile-time constant.   [R_KIND]
   real(r_kind) step,start
--------^
da_radiance.f(442): error #6683: A kind type parameter must be a compile-time constant.   [R_DOUBLE]
   real(r_double),dimension(maxinfo+maxchanl):: data1b8
--------^
da_radiance.f(443): error #6683: A kind type parameter must be a compile-time constant.   [R_DOUBLE]
   real(r_double),dimension(n1bhdr):: bfr1bhdr
--------^
da_radiance.f(444): error #6683: A kind type parameter must be a compile-time constant.   [R_DOUBLE]
   real(r_double),dimension(n2bhdr):: bfr2bhdr
--------^
da_radiance.f(460): error #6457: This derived type name has not been declared.   [DATALINK_TYPE]
   type (datalink_type), pointer    :: head, p, current, prev
---------^
da_radiance.f(463): error #6457: This derived type name has not been declared.   [INFO_TYPE]
   type(info_type)                ::  info
--------^
da_radiance.f(18816): catastrophic error: Too many errors, exiting
compilation aborted for da_radiance.f (code 1)

Looks like this is the module that starts the chain of errors?

 

[JeremyB]

Hello,

I finally got around to testing out da_wrfvar.exe with the new compilers and I came across a segmentation fault:


I also got seg faults with wrf.exe so there may be something wrong with my installation on that end too.

Thanks,
JeremyB

 

[William.Hatheway]

@islas @JeremyB

I have also tried building it without pnetcdf and phdf5 and it still runs into the same problems.