Subroutine DATINT: Error Data not found: 1998-01-01_00:00:00

[cwrenn]

I have had two apparently interrelated issues associated with ungribbing files using wrf4.3:

1) I have been unable to ungrib more than two months of files at a time (on Cheyenne), and
2) intermittently I have been unable to ungrib at all b/c of an error stating that the data is not found (on the UH HPC).

In the first case 1), I have not been able to ungrib more than two months of NCEP R2 data at a time, so for a six month runs, I have to ungrib in three stages instead only once.

In the second case 2), I have tried every possible variation to get the NCEP R2 files to ungrib by using tar files and grib files, for four months, two months and one month. Nothing seems to work. The strange this is though: it seems to work sometimes. About a month ago, I ungribbed the same NCEP R2 dataset (Nov 1997 to Apr 1998) and it ungribbed 6-months of tarred files without any problem, but now for some reason, it doesn't seem to ungrib anything, tar files or grib files.

Any suggestions as to what the problem might be and what else I could try would be appreciated.

 

[Ming Chen]

Hi,
For case 1, did you get any error message?
For case 2, I am not sure whether the environmental variables have been changed. Please recompile WRF/WPS, then try again.

 

[cwrenn]

Hi, Ming Chen!

Yes, in case 1), when I tried to ungrib more than two months the error message said: Error: hdate = 1998-02-01_06, fulldate = 1998-02-01_00, . . . 1998-04-01, Field = PMSL

In case 2), there is another matter--that may be related--and that is a problem that I have long had: dealing with num_metgrid_soil_levels. I have only recently realized that when diag_print = 1, that wrf requires num_metgrid_soils_levels = 0, and when diag_print = 0, it requires num_metgrid_soil_levels = 2. I find this rather glitchy b/c I would like to run the first couple of days of a new job with diag_print = 1, so that I can get spin up related output.

But then when, I no longer need the output, I set diag_print = 0, forcing num_metgrid_soil_levels to be 2. The reason that I mention this other matter is because when I looked at the met_em files, they look quite different in the two cases: When print_diag = 1 and num_metgrid_soil_levels = 0, there is no entry for SOIL LEVELS in the met_em file at all. Whereas, when print_diag = 0 and num_metgrid_soil_levels = 2, there is an entry SOIL LEVELS with K 1:2, etc.

I'm not sure if it is a good idea to recompile WPS in the middle of a run. I have only done 2 months of a 6 month run and restart = .true.
What specific problem might recompiling WRF/WPS solve? Not sure I understand the thought process.

 

[cwrenn]

Ming Chen--
This post is a follow up on my earlier Data not found error post.

When I inspected the files (after ungribbed failed) I noticed that when I linked with tarred or gribbed NCEP R2 files, ungribbing left two FILEs incomplete: 1) FILE: 1998-01-01_00 had a reduced file size (941512) and 2) FILE: 1998-01-01_06 was empty (0) (then jumped to ungribbing 1998-02-01_00, etc.)

The first (normal) syntax that I used to link (which generated the Error message) was:
./link_grib.csh /.../flx.ft06.199712*.grb /.../pgb.anl.199712*.grb /.../flx.ft06.199802*.grb /.../pgb.anl.199802*.grb.

Afterwards, I decided to link a single month using the second syntax:
./link_grib.csh /.../flx.ft06.199801.tar /.../pgb.anl.199801.tar,
and it ungribbed successfully--albeit it was only one month.

I have never had a problem with the first syntax (whether using tarred or gribbed NCEP R2 files).

I was wondering if you had any idea why the first syntax (using either gribbed or tarred files) would generate errors, such as Error: hdate = 1998-01-01-06, fuldates = . . ., Field PMSL, while the second syntax did not? Is there any way I could use a similar syntax (as in the second case) in which I could ungrib more than one month at a time? Alternatively, could I just ungrib one month at a time using the second syntax until I ungribbed six months and then run ./metgid.exe using those incrementally ungribbed FILEs?

Thanks for your help and suggestions!

 

[Ming Chen]

Hi,
I apologize for getting back to you late, but this is because I was out of office in the last two weeks.
If you have trouble to ungrib more than two months of data, then please try to do month by month. I am sorry that this will add extra repetitive work, yet it is acceptable and hope it won't cause you too many troubles.

As for the syntax to link file, I would suggest that you put all files in the same directory, i.e., you can put the following files in ./data199712:
flx.ft06.199712*.grb, pgb.anl.199712*.grb,

Then you can run the scrip by:
./link_grib.csh ./data199712/*

This will link all flx and pgb files to your WPS directory. Then you can run ungrib to process all of data for 19912.

Please try and let me know if you have any problem.

 

[cwrenn]

Ming Chen--

Hope you had a good vacation. I have been doing the runs month by month but I recently realized that I can just ungrib (the users manual says "degrib") month by month and after I have six months, just metgrid all six months, etc. I will try the new syntax that you suggest.

I am still curious to know what is the source of the problem. I have the same problem on Cheyenne and on the UH HPC. When you suggested that I rebuild WRF/WPS was that because you had a particular idea of where the problem might be coming from? What do you think of just rebuilding WPS and then seeing what happens? My numerical experiment will be completed soon, so it will be a lot less painful to rebuild. Thanks for your help!

 

[Ming Chen]

HI,
Sometimes when the system is updated, or a library is updated, the previously compiled code will no longer works. This happens in cheyenne before, and we have to recompile all the codes. I don't know exactly why this happens and we didn't explore the reasons.

If you can ungrib data month by month, it indicates that the precompiled codes work just fine and you don't need to recompile. I am not sure yet why ungrib cannot process more than one month data, while metgrid is able to process data over a longer period.

 

[cwrenn]

Ming Chen--

I've recently been ungribbing CFSR reanalysis data and the disk space issue shows up promptly on Cheyenne since the 6-hr Intermediate pressure files are 220 M each. I was only able to ungrib about 8 days before the "Error: Data not found" message appeared. But at least it did ungrib about a week's worth so I could do a quick wrf run. But unfortunately, I cannot continue ungribbing from 1997-11-09 (even though I moved all the Intermediate Files to my scratch drive).

I plan to rebuild wrf4.3 in my scratch drive as soon as I have some time. Do you know of any way to recompile wrf so that it returns MPI related information?

Thanks for sharing the experience of not being able to run wrf/wps sometimes when the system is updated. It would help explain a number of weird problems that I have had over the years. Is there a log somewhere, where I can find all of the dates in which Cheyenne was updated?

 

[cwrenn]

Ming Chen--

I wanted to give you an update on my disk file space or possible corrupted wrf4.3 problem.

As I kept getting disk space related errors along with degrading WPS performance (noticeably when I ungrib CFSR reanalysis data, ds093.0), I copied all of my WPS files into my scratch drive on Cheyenne and now everything works fine. I just ungribbed 6 months of CFSR data from tar files without incident. So it seems that wrf4.3 has not been noticeably degraded by the machine going down and being brought back up (so far!) as long as I know.

I'm still perplexing over how to eliminate spin-up--and whether or not it is even possible--from wrf runs. I've been encouraged somewhat by a sentence that I recently read in Saha et al. (2010) on CFSR: "[T]here is a full 1-yr overlap between the streams to address spinup issues concerning the deep ocean.." (1018). Though I don't know what this overlap procedure is that addresses spin-up, it definitely seems suggestive.

Thanks again for your help!