Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

REAL: error opening wrfinput for writing

This post was from a previous version of the WRF&MPAS-A Support Forum. New replies have been disabled and if you have follow up questions related to this post, then please start a new thread from the forum home page.

N

neel14

Member
Hi, I am getting this error with WRF 4.2. There are similar threads in the forum but no exact solution.
WRF was installed successfully. I have tried the same steps in installation as with previous versions of WRF (4.1.3) and I never faced such an error.Also large file support was enabled and sufficient disk space is there.
Kindly help!

Also
When configuring WRF-4.1.3 , This build of WRF will use classic (non-compressed) NETCDF format is shown. 4.1.3 is working properly.
While configuring WRF-4.2, This build of WRF will use NETCDF4 with HDF5 compression is shown.
Both are configured with the same libraries of NETCDF4 and HDF5. I am confused why is the difference. Could it be a reason for the above error?
Also tried tried different HDF5 versions if it matters.
 

Attachments

  • rsl.error.txt
    1.7 KB · Views: 84
Hi,
I don't think the message you got upon building has anything to do with this running problem. I'm not sure why you got a different message when building, unless some environment settings were modified/updated between the 2 builds.

As for the real.exe problem, there is no error message in the file you sent. Can you attach your namelist.input file, and the rest of the the rsl.error.* files so that I can take a look? Thanks!
 
Hi,
I recompiled 4.2, the the issue went away. But I definitely get a different building message with 4.1.3 and 4.2. I have tested several times. I was using netcdf4-4.4.1 and hdf5-1.10.0

Do you have any guide for installing wrf with netcdf4 compression, version nos of NETCDF4 and hdf5 which works without any issues with 4.1.3?
It would be helpful.
Thanks.

Update:
I have installed NETCDF4 4.7.3 with netcdf4 enabled, HDF5 1.10.6 , NETCDFF-4.2, set NETCDF4=1, still I get non compressed format while configuring.
 

Attachments

  • configure.wrf
    23.4 KB · Views: 57
  • Capture.PNG
    Capture.PNG
    61.8 KB · Views: 2,769
Hi,
Unfortunately I'm not aware of the best versions of software for the builds, but I would think any of them should work. Have you followed the guidelines on this page?
https://www2.mmm.ucar.edu/wrf/users/building_netcdf4.html

Have you set the new paths for netcdf-4 and hdf5 in your environment?
 
kwerner said:
Hi,
Unfortunately I'm not aware of the best versions of software for the builds, but I would think any of them should work. Have you followed the guidelines on this page?
https://www2.mmm.ucar.edu/wrf/users/building_netcdf4.html

Have you set the new paths for netcdf-4 and hdf5 in your environment?
Click to expand...
Yes, I followed that guide and did set the paths.
I tried installing 4.1.4 with netcdf4 compression and it installed without any problem. v4.1.3 definitely has got some bugs.
 
Hi,
I just tested this build on our HP system and was able to get WRFV4.1.3 to show the correct output when configuring for a NetCDF-4 build. Are you certain that you LD_LIBRARY_PATH is pointing to both the NetCDF library and the HDF5 library?
 
kwerner said:
Hi,
I just tested this build on our HP system and was able to get WRFV4.1.3 to show the correct output when configuring for a NetCDF-4 build. Are you certain that you LD_LIBRARY_PATH is pointing to both the NetCDF library and the HDF5 library?
Click to expand...

Yes ma'am, I am sure about that or else I wont have been able to configure and install v4.1.4 or v4.2. Kindly mention the versions of HDF5 and NETCDF4 you are using. Also it would be helpful if you share the configure.wrf.

However with 4.1.4 and 4.2 with NETCDF4, I still get REAL: error opening wrfinput for writing. I have to use_netcdf_classic = T, to remove the problem. I had mentioned above post regarding the issue getting solved, but it isn't the case, apologies. I have attached the rsl files you had asked before.

If it matters, I had installed the WRF-Chem versions of 4.1.4 and 4.2 with NETCDF4 but still was unsuccessful with both wrf and wrfchem versions of 4.1.3. I need the NETCDF4 compression since WRF-Chem produces very big files.Do you know if NETCDF4 compression works with WRF-Chem?

In short:
v4.1.3 doesn't even get configured with NETCDF4 compression.
v4.1.4 and v4.2 gets configured and installed with NETCDF4 but error while running.

Update: Tried v 4.1.5 and 4.1 also. Conclusion:All versions above and including 4.1.4 gets installed with NETCDF4 but does not work.
v4.1 is same as 4.1.3. So (probably all) versions below and including 4.1.3 does not even get configured with NETCDF4.
In github "The incorrect use of whereis in configure to locate nc-config is fixed" is mentioned regarding to 4.1.4, so may be related.

Thanks a lot you for sparing your time.

Neeldip
 

Attachments

  • rsl.tar.gz
    46.8 KB · Views: 56
Hi,
1) My NetCDF version is 4.7.3, and HDF5 is 1.10.5. I'm attaching the configure.wrf file I used.
2) I'm not sure about compression with WRF-Chem. I would assume that it's okay, but you may want to post in the chem section to see what their group says about it.
3) When building NetCDF-4, are you building with parallel I/O? If so, don't do that, and try again.
4) If nothing else, can you send me a couple of time periods of your met_em* files, along with your namelist.input file so that I can test with your input on my built version of WRF with netcdf-4 compression? If the files are too large, see the front page of this forum for information on sharing large files.
 

Attachments

  • configure.wrf
    23.1 KB · Views: 69
Hi,
I have uploaded the met_* files. File name:metem.tar.gz . Namelist is also within that file.
I have tried everything i know of. Still dont know why its not working. I am mentioning the steps. Please correct me if I made any mistake.
1.Installed hdf5-1.10.6 with fortran support
2.Added hdf5 to library path
3.Installed netcdf-4.7.3 with --enable-netcdf4
nc-config output
Code: 
   This netCDF 4.7.3 has been built with the following features: 

  --cc            -> icc
  --cflags        -> -I/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/include -I/home/neeldip/WRF-Chem/LIBRARIES/grib2/include
  --libs          -> -L/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/lib -lnetcdf
  --static        -> -lhdf5_hl -lhdf5 -lm -lcurl 
  --has-c++       -> no
  --cxx           -> 
  --has-c++4      -> no
  --cxx4          -> 
  --has-fortran   -> yes
  --fc            -> ifort
  --fflags        -> -m64 -I/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/include
  --flibs         -> -L/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/lib -lnetcdff -L/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/lib -lnetcdf -lnetcdf
  --has-f90       -> yes
  --has-f03       -> unknown option: --has-f03
Usage: nf-config [OPTION]

Available values for OPTION include:

  --has-dap       -> yes
  --has-dap2      -> yes
  --has-dap4      -> yes
  --has-nc2       -> yes
  --has-nc4       -> yes
  --has-hdf5      -> yes
  --has-hdf4      -> no
  --has-logging   -> no
  --has-pnetcdf   -> no
  --has-szlib     -> no
  --has-cdf5      -> yes
  --has-parallel4 -> no
  --has-parallel  -> no
  --prefix        -> /home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3
  --includedir    -> /home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/include
  --libdir        -> /home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/lib
  --version       -> netCDF 4.7.3

4. Installed netcdf fortran 4.2
Code: 
  --cc        -> icc
  --cflags    ->  -I/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/include -I/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/include
  --libs      -> -L/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/lib -L/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/lib -lnetcdf -lnetcdf 
  --fc        -> ifort
  --fflags    -> -m64 -I/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/include
  --flibs     -> -L/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/lib -lnetcdff -L/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/lib -lnetcdf -lnetcdf 
  --has-f90   -> yes
  --has-nc2   -> yes
  --has-nc4   -> yes
  --prefix    -> /home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3
  --includedir-> /home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/include
  --version   -> netCDF-Fortran 4.2
5. Added netcdf to the library and path
6. export NETCDF4=1
7. configured and installed wrf

While comparing the configure.wrf shared by you, I noticed a difference. You had the libararies(in bold) mentioned. I never get this. I wonder if that makes a difference:
Code: 
#### NETCDF4 pieces

NETCDF4_IO_OPTS = -DUSE_NETCDF4_FEATURES -DWRFIO_NCD_LARGE_FILE_SUPPORT
GPFS            =
CURL            =
HDF5            = [b]-lhdf5 -lhdf5_hl[/b]
ZLIB             = [b]-lz[/b]
DEP_LIB_PATH    = [b]-L/glade/u/apps/ch/opt/netcdf/4.7.3/intel/19.0.5//lib[/b]
NETCDF4_DEP_LIB = $(DEP_LIB_PATH) $(HDF5) $(ZLIB) $(GPFS) $(CURL)
 
Thanks for sending those files. I was able to test V4.1.3 and V4.2 (both using NetCDF-4) on my system, using your namelist and met_em* files. Real.exe runs without any problems. So I'm kind of at a loss for why this would be happening. I'll reach out to one of our software engineers to see if they have any ideas.
 
Between version 4.1.3 and version 4.2 we modified the way we determined which netcdf type to select (classic vs version4).

Here is a snippet of the new logic from the configure file:
Code: 
 205 # If the user asked for classic netcdf, acquiesce to the request.
 206 
 207 ans="`which nf-config`"
 208 status="$?"
 209 if [ "$ans" = "nf-config:" -o "$ans" = "" -o "$status" != "0" ] ; then
 210     export NETCDF_classic=1
 211     unset NETCDF4
 212 else
 213   if [ -n "$NETCDF_classic" ] ; then
 214     if [ $NETCDF_classic -eq 1 ] ; then
 215       unset  NETCDF4
 216     else
 217       unset  NETCDF_classic
 218       export NETCDF4=1
 219     fi
 220   else
 221     if [ "`nf-config --has-nc4`" = "yes" ] ; then
 222       unset  NETCDF_classic
 223       export NETCDF4=1
 224     else
 225       export NETCDF_classic=1
 226       unset  NETCDF4
 227     fi
 228   fi
 229 fi

Here is the original logic:

Code: 
 205 # If the user asked for classic netcdf, acquiesce to the request.
 206 
 207 if [ "`uname`" = "Linux" ] ; then
 208   ans="`whereis nf-config`"
 209 elif [ "`uname`" = "Darwin" ] ; then
 210   ans="`which nf-config`"
 211 else
 212   ans=""
 213 # echo "Add in your architecture's uname and the command to find executables in the path"
 214 # exit 1
 215 fi
 216 if [ "$ans" = "nf-config:" -o "$ans" = "" ] ; then
 217     export NETCDF_classic=1
 218     unset NETCDF4
 219 else
 220   if [ -n "$NETCDF_classic" ] ; then
 221     if [ $NETCDF_classic -eq 1 ] ; then
 222       unset  NETCDF4
 223     else
 224       unset  NETCDF_classic
 225       export NETCDF4=1
 226     fi
 227   else
 228     if [ "`nf-config --has-nc4`" = "yes" ] ; then
 229       unset  NETCDF_classic
 230       export NETCDF4=1
 231     else
 232       export NETCDF_classic=1
 233       unset  NETCDF4
 234     fi
 235   fi
 236 fi

With the older version, notice that we used "whereis" on line #208. This was incorrect.

When you issue the command "which nf-config", what is the output?
 
davegill said:
When you issue the command "which nf-config", what is the output?
Click to expand...
Code: 
  --cc        -> icc
  --cflags    ->  -I/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/include -I/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/include
  --libs      -> -L/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/lib -L/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/lib -lnetcdf -lnetcdf 
  --fc        -> ifort
  --fflags    -> -m64 -I/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/include
  --flibs     -> -L/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/lib -lnetcdff -L/home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/lib -lnetcdf -lnetcdf 
  --has-f90   -> yes
  --has-nc2   -> yes
  --has-nc4   -> yes
  --prefix    -> /home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3
  --includedir-> /home/neeldip/WRF-Chem/LIBRARIES/netcdf4.7.3/include
  --version   -> netCDF-Fortran 4.2
 
Here are a couple of machines that I have. When I issue the command, I get these outputs:
Code: 
machine 1> which nf-config
nf-config: Command not found.

Code: 
machine 2> which nf-config
/usr/local/gnu/9.3.0/netcdf/4.7.4/f-4.5.2-cxx-4.3.1/bin/nf-config
 
davegill said:
Here are a couple of machines that I have. When I issue the command, I get these outputs:
Code: 
machine 1> which nf-config
nf-config: Command not found.

Code: 
machine 2> which nf-config
/usr/local/gnu/9.3.0/netcdf/4.7.4/f-4.5.2-cxx-4.3.1/bin/nf-config
Click to expand...
I get this with which nf-config
~/WRF-Chem/LIBRARIES/netcdf4.7.3/bin/nf-config

But I think the issue is with my netcdf installation or with compiler.I was using intel compiler 2016. Do you know of any issue with this compiler?
While running make check again, i am getting several errors.
While trying nccopy I was able to convert to classic format but got errors with any of the netcdf4 formats. So issue is either of the two.

Appreciate your help and Kelly's for your time.
 
What we want is for the build to work automatically for you, as it did with versions up through v4.1.2 (at least). The way forward is to look at the build changes that were introduced into the configure script.

What we need you to do is to have two separate directories, one for v4.1.2 and one for v4.2 (or later). In each of these directories you are going to edit the top level configure script.

In each step of the nested IF tests for the netcdf type (classic vs netcdf4), put in an echo statement. The lines of interest start around 205 and continue for about 25 more lines.

Based on the output of the debugging print, we want to see where the difference in the path through the IF test logic is different.
 
davegill said:
What we want is for the build to work automatically for you, as it did with versions up through v4.1.2 (at least). The way forward is to look at the build changes that were introduced into the configure script.

What we need you to do is to have two separate directories, one for v4.1.2 and one for v4.2 (or later). In each of these directories you are going to edit the top level configure script.

In each step of the nested IF tests for the netcdf type (classic vs netcdf4), put in an echo statement. The lines of interest start around 205 and continue for about 25 more lines.

Based on the output of the debugging print, we want to see where the difference in the path through the IF test logic is different.
Click to expand...
I replaced the configure file in 4.1.3 from 4.1.4, the netcdf4 compression is getting detected now. Yet to test the working. Will look into your suggestions.
As for the error occuring, my compilers were the issue, they were too old for any newer versions of hdf5 causing errors in make check. Tried older versions and I was able to install netcdf4 without any errors and wrf with netcdf4 compression.
 
Sir I have got the netcdf4 compression to work. But writes are significantly slow.
Kindly provide me suggestions how to speed up the model writes.
 
Since this thread is "REAL: error opening wrfinput for writing", please open a new issue. We would like users to be able to search for solutions based from the title of the issue.
 
You must log in or register to reply here.
Top