WRF error 9 Killed: KISSVEC SEED GENERATOR REQUIRES PMID FROM BOTTOM FOUR LAYERS

[bparazin]

Hello, I have almost got my WRF model up and running, it runs without error through all WPS and real.exe, but when I run Hello, I have almost got my WRF model up and running, it runs without error through all WPS and real.exe, but when I run

mpirun -np 4 ./wrf.exe,

it faults with error 9 Killed and one of the 4 rsl.error files has the following explanation: "STOP MCICA_SUBCOL: KISSVEC SEED GENERATOR REQUIRES PMID FROM BOTTOM FOUR LAYERS" which I cannot figure out how to solve. The rsl.error file with this is is rsl.error.0002 in the attached files, and if any of you have recommendations for how to modify my data/namelist to avoid this, I would be greatly appreciative.

I was doing some research on this, and the only forum post I can find that references this error is here crashes when running with noah lsm but their solution is to simply comment out the check causing that error, and if I do that the run instead stops with error 11 segfault, so that doesn't work. I'm using NOAA 20th century reanalysis V3 as my input data, using jupiter-opensource / wrf-preproc-intermediate-writer · GitLab in place of ungrib to convert it to a file readable by metgrid, but given that runs without issue, I think that is working well. My namelist.input and the rs.error.0002 are attached, and let me know if I can provide anything else to help. I've been trying to find my way around this error for a week now and would greatly appreciate any help you all have

 

[kwerner]

Hi,
The fact that the model stops immediately typically indicates something is wrong with the input data. I don't see anything crazy in your namelist that would be causing the issue. I would recommend taking a close look at the input data, making sure there is nothing missing or bad data (start with the surface level - many times that is the cause of the problem), and make sure you're using all the fields that are mandatory to run wrf.

 

[bparazin]

Hi,
The fact that the model stops immediately typically indicates something is wrong with the input data. I don't see anything crazy in your namelist that would be causing the issue. I would recommend taking a close look at the input data, making sure there is nothing missing or bad data (start with the surface level - many times that is the cause of the problem), and make sure you're using all the fields that are mandatory to run wrf.

Hi, looking at my input data, I think it's an issue of the WRF and input data landmasks not lining up, leading to the wrf.exe forcing seeing some NaN soil temps and snow depths around coastlines, but I'm having trouble correcting it to use the input landmask I've got. Following here: If my input data have a landmask that is different from the one generated by the geogrid program, which should be used?, I have made a new file "FILE:LANDSEA" (attached here as a zip) which is now being read in by metgrid by adding constants_name to the namelist.wps. I haven't changed the metgrid.tbl (also attached here) because it already looks like it fits the example give, but it appears to still be using the old landmask since the issue is persisting. Given that post is 4 years old, should I be doing something different to use my input land mask?

 

[kwerner]

Hi,
After you run metgrid, can you tell whether the met_em* files are using the new or old LANDSEA variable? In your rsl files, do you happen to see any message stating "grid%tsk unreasonable?" If so, and maybe even if not, take a look at this post that addresses an issue that may be related to yours.

 

[bparazin]

Hi, looking at it, it looks like it uses the new LANDSEA variable and the problem actually lay with the interpolation during metgrid, like you suspected (plotting out the metgrid data for soil temp showed it was different from either mask). Following the fix there, I updated interpolation method to nearest neighbor at that fixed the interpolation errors. I've gone through all the metgrid files and checked that there are no NaN values being exposed, which there aren't.

This, however did not fix the over-arching error of the runs still being killed with an error 9 Kissvec seed generators etc. like I've been getting the whole tiem so I don't think it was the cause of the error after all. I'm at such a loss

 

[kwerner]

Hi,
Take a look at this post. It's referring to the same issue, but with the MPAS model. However, MPAS and WRF use the same physics routines. They suggest that the code for RRTMG sw and lw is likely unnecessary and you can simply comment out that section of the code. You'll need to do it in both the
phys/module_ra_rrtmg_lw.F and phys/module_ra_rrtmg_sw.F files. Once you do that, recompile the code to capture those changes. You will not need to clean the code, or to reconfigure. Just simply recompile, and it should be much quicker than your original compile. After that, try to run it again to see if it gives you another error somewhere.

 

[bparazin]

That's the first thing I tried; doing that just causes it to segfault every time, as described in my original post

 

[kwerner]

I apologize for that oversight. However, now that you've sorted out the LANDSEA issue, have you tried commenting out that code since then, to see if you still get a segmentation fault?

 

[bparazin]

Yep, still segfaults

 

[kwerner]

Ok, thanks. I'd like to try to run this to see if I'm able to repeat the issue. Can you send me the wrfbdy_d01 and wrfinput_d01 file so I can test it? It's likely those files will be too large to attach. If you don't have another method for sharing the files, take a look at the home page of this forum for details on sharing large files. Thanks!

 

[bparazin]

Ok, thanks. I'd like to try to run this to see if I'm able to repeat the issue. Can you send me the wrfbdy_d01 and wrfinput_d01 file so I can test it? It's likely those files will be too large to attach. If you don't have another method for sharing the files, take a look at the home page of this forum for details on sharing large files. Thanks!

Thanks for all the help; it's uploaded as bparazin_pmid_error.rar

 

[kwerner]

I hate to cause more trouble, but do you mind uploading it as a .tar instead of .rar? We have no way of opening .rar files here. Thanks!

 

[bparazin]

No worries, uploaded as a tar of the same name

 

[kwerner]

Thanks for sending those. Okay, I was able to repeat the issue, and I also discussed that error (KISSVEC SEED GENERATOR REQUIRES PMID FROM BOTTOM FOUR LAYERS) with our physics specialist, who says that the code IS legit and should remain in the code. The code is checking to ensure that each level/layer has a higher pressure than the level/layer above it, which does make sense. I checked your wrfinput file and if you look at the "P" variable (perturbation pressure) the range is WAY outside the normal range. For e.g., your range for P is -97046.8 to 3.41615e+10 Pa, while for another generic test simulation, the range is -0.0566406 to 3161.06 Pa. To find the actual pressure, you would use the equation ( P + PB ) for a value in Pa, or ( P + PB )*0.01 for the hPa value. If you do that for the first model level (level 0) and then for level 1, you can see that pressure for level 1 is greater than for level 0, which is problematic. This means something is definitely wrong with the input data. You'll need to figure out whether it's the converter you're using, or whether the data, itself, is bad.

 

[bparazin]

Ok thanks for all the help. Looking at the intermediate steps, my input data is isobaric in hPa between 1000 and 10 hPa (and also given it's a NOAA product, I hope the issue isn't there), looking at the metgrid output netcdf, everything seems normal except for the relative humidity being in the range [-6, 105.5], which seems Bad for a value that is supposed to be a percent, would that weirdness be sufficient to cause such an insane P range? I'm not providing perturbation pressure, so what fields does WRF use to calculate it? Also looking at the input, I think this might be some kind of interpolation error happening during metgrid? Because using debug tools on the program that is used to produce the FILE:[date] metgrid input shows it is in the range [0, 100] as one would expect

 

[kwerner]

Can you send me a couple of your data input files you are using to run metgrid? I'd like to take a look at them and maybe run a test starting from that step. Can you also attach your namelist.wps file? Thanks!

 

[bparazin]

Hello, sorry about the delay in getting back to you! I was on holiday break, and then took a little bit more than I would like working through my email backlog. I've attached my namelist here and then I've uploaded 3 of the data input files as b_parazin_input_files.tar to the nextcloud storage

 

[kwerner]

Hi, Thanks for sending that, and now it's my turn to apologize for the delay. I was out of the office for a week and have been conducting our biannual tutorial this week, so I've been slow to catch up with other things.

With your files, I am, again, able to repeat the issue. Using the default METGRID.TBL, I see that variables like soil moisture, soil temp, and LANDSEA are missing (similarly to what you mentioned). When I modify all those variables to use "nearest_neighbor" interpolation, they look somewhat better after metgrid, but it doesn't fix the error message. Since the input data appears on pressure levels, it's difficult to compare to the pressure of the layers after real.exe does the vertical interpolation.

I was actually able to get it to work out (wrf.exe runs to completion) by using the same data, but from NCAR's RDA repository, instead of from NCEP. To use these data, I had to obtain both the 3 hr and 6 hr data because, for some reason, only certain atmospheric variables are available with each; however, the data are already in gribbed format and I was able to just run them through ungrib. When doing that, I didn't use the additional LANDSEA input because it wasn't necessary. Would you be willing to try that dataset, instead? Do the data you're using come in grib format, and if so, did you try to run ungrib instead of the intermediate writer you mentioned, just to see if that makes any difference?

 

[bparazin]

Thank you for all the help. Sadly, using the NCAR RDA repository doesn't work for I want--my research goal was to compare the weather surrounding the great flood of 1844 to future events on a finer scale and with more variables than the 20th century reanalysis applies, hence the decision to use this dataset since its the only one I've been able to find that goes back that far. I'm working with 1993 right now because it's a similarly large flood, and I had hoped to verify the pipeline I used by comparing simulation results to other data sets & real world observations so I know the 1844 simulations would be trustworthy. It also doesn't come in grib format, just netcdf as far as I have been able to see.

I feel like the problem is probably somewhere in the intermediate writer considering that's the only "non-standard" piece of this puzzle, but at this point, after talking to my advisor, we're calling this project a dead end and moving on to other stuff. Thanks again for all the help with this!

 

[kwerner]

Oh no! I'm really sorry to hear that. I hope you find another project that is much more successful and less frustrating!