(RESOLVED) Ungrib compilation failed: Integer too big for integer kind 8

All topics related to compiling the WPS, including library installation and environment set-up
Post Reply
CarterHumphreys
Posts: 1
Joined: Tue Nov 05, 2019 11:16 pm

(RESOLVED) Ungrib compilation failed: Integer too big for integer kind 8

Post by CarterHumphreys » Mon Nov 30, 2020 5:33 pm

Hi,

I was able to get WRF to compile without any trouble but WPS doesn't seem to finish correctly. When I run ./compile it goes for a few minutes and I get all 3 executables but all are 0 bytes.

I was following these directons https://github.com/DevinSmoot/HPC-Guide ... ilding-wps for use on ARM processor. I also tried the official guide but was unable to with that either.
Attachments
configure.wps
(3.3 KiB) Downloaded 9 times
compile.log
(94.94 KiB) Downloaded 10 times

mgduda
Posts: 422
Joined: Mon Feb 26, 2018 7:35 pm

Re: Ungrib compilation failed: Integer too big for integer kind 8

Post by mgduda » Wed Dec 02, 2020 9:06 pm

The error message from the compilation log is
gfortran -c -ffixed-form -O -fconvert=big-endian -frecord-marker=4 intmath.f
intmath.f:91:35:

if(iand(i,Z'FFFFFFFF00000000')/=0) then
1
Error: Integer too big for integer kind 8 at (1)
intmath.f:95:9:

endif
1
Error: Expecting END FUNCTION statement at (1)
Could you try compiling and running the following test program:

Code: Select all

program i8size

    implicit none

    integer(kind=8)::i

    write(0,*) huge(i)

    stop

end program i8size
The goal is to determine whether the GNU 8.3.0 compilers on the Raspberry Pi are treating kind=8 integers as 32-bit integers or as 64-bit integers; if they're treated as the former, the test program should print "2147483647", and if the latter, it should print "9223372036854775807".

(Note: I've modified the title of this thread to more specifically describe the issue.)
NCAR/MMM

davegill
Posts: 89
Joined: Mon Apr 23, 2018 9:03 pm

Re: Ungrib compilation failed: Integer too big for integer kind 8

Post by davegill » Wed Dec 02, 2020 10:46 pm

I have had this same issue.

I ran this suggested test code:

Code: Select all

pi@raspberrypi:~/TOYS $ cat test.f90
program i8size
    implicit none
    integer(kind=8)::i
    write(0,*) huge(i)
    stop
end program i8size
Here is the version of gfortran I used:

Code: Select all

pi@raspberrypi:~/TOYS $ gfortran --version
GNU Fortran (Raspbian 8.3.0-6+rpi1) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Here is the output:

Code: Select all

pi@raspberrypi:~/TOYS $ gfortran test.f90 
pi@raspberrypi:~/TOYS $ ./a.out
  9223372036854775807
The specifics of my machine:

Code: Select all

pi@raspberrypi:~/TOYS $ lsb_release -a
No LSB modules are available.
Distributor ID:	Raspbian
Description:	Raspbian GNU/Linux 10 (buster)
Release:	10
Codename:	buster
Per Michael's suggestion, to build WPS, I tried:

Code: Select all

git diff arch/configure.defaults
diff --git a/arch/configure.defaults b/arch/configure.defaults
index 2c66a47..9bad4a3 100644
--- a/arch/configure.defaults
+++ b/arch/configure.defaults
@@ -145,7 +145,7 @@ ARFLAGS             =
 CC_TOOLS            = 
 
 ########################################################################################################################
-#ARCH    Linux i486 i586 i686, gfortran   # serial serial_NO_GRIB2 dmpar dmpar_NO_GRIB2
+#ARCH    Linux i486 i586 i686 armv7l, gfortran   # serial serial_NO_GRIB2 dmpar dmpar_NO_GRIB2
 #
 COMPRESSION_LIBS    = CONFIGURE_COMP_L
 COMPRESSION_INC     = CONFIGURE_COMP_I

pi@raspberrypi:/mnt/mydisk/gill/WPS $ git diff ungrib/src/ngl/g2/intmath.f
diff --git a/ungrib/src/ngl/g2/intmath.f b/ungrib/src/ngl/g2/intmath.f
index f9fc6eb..d571eaa 100644
--- a/ungrib/src/ngl/g2/intmath.f
+++ b/ungrib/src/ngl/g2/intmath.f
@@ -88,7 +88,7 @@
          !write(0,*) 'iand i-1'
          ilog2_8=1
       endif
-      if(iand(i,Z'FFFFFFFF00000000')/=0) then
+      if(iand(i,int(Z'7FFFFFFF00000000',kind=8))/=0) then
          ilog2_8=ilog2_8+32
          i=ishft(i,-32)
          !write(0,*) 'iand ffffffff',i,ilog2_8
With the first mod, I can get the configure step to recognize the architecture. With the second mod, ungrib builds OK. I have run ungrib all of the way through WRF, so the decoded data must be OK.

SPiltz
Posts: 5
Joined: Wed Dec 02, 2020 4:22 am

Re: Ungrib compilation failed: Integer too big for integer kind 8

Post by SPiltz » Wed Dec 02, 2020 10:55 pm

I ran the test program and got...

pi@node1://home/pi/software/ncar-wrf_3.8.1/build/WPS $ gfortran -ffree-form i8size.f90
pi@node1://home/pi/software/ncar-wrf_3.8.1/build/WPS $ ./a.out
9223372036854775807

I am looking at your follow-up posts now Dave

mgduda
Posts: 422
Joined: Mon Feb 26, 2018 7:35 pm

Re: Ungrib compilation failed: Integer too big for integer kind 8

Post by mgduda » Wed Dec 02, 2020 11:20 pm

Does the intmath.f code still compile if

Code: Select all

int(Z'7FFFFFFF00000000',kind=8)
is changed to

Code: Select all

int(Z'FFFFFFFF00000000',kind=8)
, which is what I originally suggested in GitHub Issue #159? Changing the MSB from "FF" to "7F" would change the semantics of the code, so it would be nice if "FF" worked.
NCAR/MMM

SPiltz
Posts: 5
Joined: Wed Dec 02, 2020 4:22 am

Re: Ungrib compilation failed: Integer too big for integer kind 8

Post by SPiltz » Wed Dec 02, 2020 11:47 pm

I changed from FF to 7F and it compiled.

Got this...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! Successful completion of ungrib. !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
pi@node1://home/pi/software/ncar-wrf_3.8.1/build/WPS $ ./metgrid.exe
Processing domain 1 of 1
Processing 2020-12-01_18
FILE
Processing 2020-12-02_00
FILE
Processing 2020-12-02_06
FILE
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! Successful completion of metgrid. !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Note: The following floating-point exceptions are signalling: IEEE_OVERFLOW_FLAG IEEE_UNDERFLOW_FLAG

mgduda
Posts: 422
Joined: Mon Feb 26, 2018 7:35 pm

Re: Ungrib compilation failed: Integer too big for integer kind 8

Post by mgduda » Thu Dec 03, 2020 12:11 am

Just to be clear, I was suggesting that davegill should try changing 7F to FF to match the semantics of the original code.
NCAR/MMM

davegill
Posts: 89
Joined: Mon Apr 23, 2018 9:03 pm

Re: Ungrib compilation failed: Integer too big for integer kind 8

Post by davegill » Thu Dec 03, 2020 5:27 am

Michael,
It had been too long ago since I changed the ungrib code, I forgot about the need to swap the leading "F" to a "7".

I decided to dig in more now to see why this is happening on the pi. It turns out that this is one of the places where the standard allows a processor to choose what to do.

Here is the 2008 standard:
https://j3-fortran.org/doc/year/10/10-007r1.pdf

Section 4.7 paragraph 2 (the leading values are the line numbers), focus on lines 25-28.

Code: Select all

19 The hex-digits A through F represent the numbers ten through fifteen, respectively; they may be represented
20 by their lower-case equivalents. Each digit of a boz-literal-constant represents a sequence of bits, according to
21 its numerical interpretation, using the model of 13.3, with z equal to one for binary constants, three for octal
22 constants or four for hexadecimal constants. A boz-literal-constant represents a sequence of bits that consists of
23 the concatenation of the sequences of bits represented by its digits, in the order the digits are specified. The
24 positions of bits in the sequence are numbered from right to left, with the position of the rightmost bit being zero.
25 The length of a sequence of bits is the number of bits in the sequence. The processor shall allow the position
26 of the leftmost nonzero bit to be at least z − 1, where z is the maximum value that could result from invoking
27 the intrinsic function STORAGE SIZE (13.7.160) with an argument that is a real or integer scalar of any kind
28 supported by the processor.
Here is your modified example code using the storage_size function:

Code: Select all

program i8size
    implicit none
    integer(kind=8)::i
    write(0,*) huge(i)
    write(0,*) storage_size(i)
    stop
end program i8size
Here is the build and run, again, where we care about the output from storage_size:

Code: Select all

pi@raspberrypi:~/TOYS $ gfortran -std=f2008 test.f90
pi@raspberrypi:~/TOYS $ ./a.out
  9223372036854775807
          64
Line 25-27 above says "The processor shall allow the position of the leftmost nonzero bit to be at least z − 1, where z is the maximum value that could result from invoking the intrinsic function STORAGE SIZE...".

So in Fortran 2008, it is processor dependent what happens at bit position 64 of an integer(kind=8).

This is stated in a couple of other places that I usually trust:
https://gcc.gnu.org/onlinedocs/gfortran ... tants.html
"The Fortran standard states that the treatment of the sign bit is processor dependent."

https://stackoverflow.com/questions/471 ... constants
"The value is processor dependent because the most significant bit in the resulting value is one."

davegill
Posts: 89
Joined: Mon Apr 23, 2018 9:03 pm

Re: Ungrib compilation failed: Integer too big for integer kind 8

Post by davegill » Thu Dec 03, 2020 5:52 am

For more completeness, there are compilation flags to ignore the range on the integer values.

Here is the code with the offending line included:

Code: Select all

pi@raspberrypi:~/TOYS $ cat /mnt/mydisk/gill/test.f90 
program i8size
    implicit none
    integer(kind=8)::i
    write(0,*) huge(i)
    write(0,*) storage_size(i)
    write(0,*) int(Z'FFFFFFFF00000000',kind=8)
    stop
end program i8size
Here are the compiler flags to ignore the processor dependent issue, and the resultant output.

Code: Select all

pi@raspberrypi:~/TOYS $ gfortran -std=f2008 -fno-range-check  test.f90
pi@raspberrypi:~/TOYS $ ./a.out
  9223372036854775807
          64
          -4294967296
Using this new compiler flag, the ungrib program shows no diffs with cmp. However, this "no diffs found" statement cannot be said to cover every possible grib file.

SPiltz
Posts: 5
Joined: Wed Dec 02, 2020 4:22 am

Re: Ungrib compilation failed: Integer too big for integer kind 8

Post by SPiltz » Mon Dec 14, 2020 10:00 pm

FYI... after making the modification noted above, ungrib compiles, and the WRF runs. I only get this minor error upon completion of metgrid...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! Successful completion of metgrid. !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Note: The following floating-point exceptions are signalling: IEEE_OVERFLOW_FLAG IEEE_UNDERFLOW_FLAG

kwerner
Posts: 2070
Joined: Wed Feb 14, 2018 9:21 pm

Re: Ungrib compilation failed: Integer too big for integer kind 8

Post by kwerner » Mon Dec 14, 2020 10:45 pm

That is great news! The error message you are seeing is no concern. It was put in the code for testing purposes and should not cause any problems. Thank you for updating the post.
NCAR/MMM

Post Reply

Return to “WPS Compiling/Installation”