Wrf on a cluster (Raspberry Pi)

[rowland]

Hello,

I'm pretty new to WRF and I have attempted to install it on a Rapberry Pi Cluster.
Initially I could not get WRF to run a the cluster. it would only run locally and looking at the rsl.errer files on a Pi in the cluster, I found I had to copy wrfinput_d01 wrfbdy_d01 and namelist.input to every Pi in the cluster. Only then would wrf.exe run on any of the Pis on the cluster.

I'm using the command

mpiexec -iface wlan0 -f /home/pi/build_WRF/config/wrfhosts -n 16 ./wrf.exe

to run wrf.exe

Should I have to copy those files (wrfinput_d01 wrfbdy_d01 and namelist.input) to every Pi in the cluster?

Many thanks

 

[kwerner]

Hi,
I want to let you know that your question is not being ignored. I was on travel for a bit and after returning last week, I reached out to one of our software engineers who may have a bit more experience with what you're doing. Hopefully I'll hear back from them soon with a response. Thank you for your patience!

 

[davegill]

There are a couple of quick points to make here.

• The default real and WRF format is serial NetCDF. Using this io_form=2 format means we should expect that only the master process for MPI ever attempts to open, read, write, or close a gridded WRF-system file.

• Similarly, the namelist.input file (and the various physics look-up tables) are only processed by the master node.

Based on these, I would have expected that the wrfinput_d01 (etc) files would only have been required on the master MPI process, so I am surprised. Could you send your namelist.input file along?

Since this is specifically a raspberry pi question ... I use just a single Pi 4 model B, so I have not run into this issue. I mounted an external disk that the RPi sees, but there are 4 processors together in this single box, which means that I can't really replicate your problem.

pi@raspberrypi:~ $ cat /proc/cpuinfo
processor       : 0
model name      : ARMv7 Processor rev 3 (v7l)
BogoMIPS        : 162.00
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

processor       : 1
model name      : ARMv7 Processor rev 3 (v7l)
BogoMIPS        : 162.00
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

processor       : 2
model name      : ARMv7 Processor rev 3 (v7l)
BogoMIPS        : 162.00
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

processor       : 3
model name      : ARMv7 Processor rev 3 (v7l)
BogoMIPS        : 162.00
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3

Hardware        : BCM2711
Revision        : d03114
Serial          : 10000000848c9a43
Model           : Raspberry Pi 4 Model B Rev 1.4

 

[rowley]

I too am new to WRF and faced the same problem with my Raspberry pi cluster
Not sure if there is a way to prevent the need to copy the files to each node on a cluster but I went ahead and did that.
I also net boot my cluster from the master PI that runs the WRF program.
This is a segment of the sgript I use to run wrf.

I identify each PI by it's serial number. and each PI has an area reserved for it's OS on the master PI under it's serial numver directory

for serial in "${!serial_numbers[@]}"; do
server=${serial_numbers[serial]}
echo copying files to server ${server}
sudo cp $WRFHOMEDIR/build_WRF/WRF/run/wrfbdy* /srv/raspi64Wrf/rootfs/${server}$WRFHOMEDIR/build_WRF/WRF/run
sudo cp $WRFHOMEDIR/build_WRF/WRF/run/wrfinput* /srv/raspi64Wrf/rootfs/${server}$WRFHOMEDIR/build_WRF/WRF/run
sudo cp $WRFHOMEDIR/build_WRF/WRF/run/namelist.input /srv/raspi64Wrf/rootfs/${server}$WRFHOMEDIR/build_WRF/WRF/run
done

Because I network boot them and each PI has it's own OS on the master PI's SSD I can simply copy the files accross without doing any network IO.
If you want more info on how to network boot I'm happy to oblidge.

Dave Gil states "Since this is specifically a raspberry pi question ". I don't beleive this is specifically a Raspberry pi question, as the problem would exist on any cluster irrespective of the hardware.

I've only just got wrf.exe to work in a sensible manor. I'm now wondering how to create an animation of precipitation, clouds over the specified time period.

 

[everdaniel]

Hi!

cool to read other people are running WRF on a Raspberry Pi, I built a cluster of 22 Raspberry Pi's a couple of years ago, it had 20 Pi's as compute nodes, 1 as "head" node and 1 as "post-processing" node.

What I did was, I came up with an image with everything I needed which I then use as the base image for all nodes, I used ansible to manage all of the nodes from the head node.

I had to setup NFS to export the "wrf" main folder and then setup this folder as "shared" folder in each node in order for WRF to run on the cluster.

I should still have my notes around so happy to share more information is needed, it was a fun project to work on!

- Ever Daniel