init.nc file generated by 8.0.0 hangs because of bad nominalMinDc

[nickheavens-cgg]

Recently, I needed to re-generate a static file and initialisation condition for a simulation I originally had done with 7.3. I did this with 8.0.0. I then encountered a problem where the log file progressed to:

Reading initial state from 'input' stream
WARNING: Variable qi not in input file.
WARNING: Variable qs not in input file.
WARNING: Variable qg not in input file.

At the same time, I noticed on top that most of the CPU were detaching from MPAS-A, i.e., going from active to 'D' (uninterruptible sleep) status. I had to kill the simulation after 15-20 minutes, because it wasn't going anywhere.

I realised by looking at earlier simulations that there was a missing warning:

WARNING: Variable nominalMinDc not in input file.

In other words, the new init.nc file contained nominalminDc, which if omitted, is then set by MPAS-A based on the config_len_disp option in namelist.atmosphere.

As an experiment, I removed nominalMinDc from the init.nc file and then ran with the modified version.

MPAS-A properly initialised and started integrating. No cpu detachment was observed.

nominalMinDc seems fine in the init.nc file. ncdump -h reports its existence:

double nominalMinDc ;
nominalMinDc:units = "m" ;
nominalMinDc:long_name = "Nominal minimum dcEdge value where meshDensity == 1.0" ;

ncdump -v reports its value as 24000 as expected.

ncmpidump treats it similarly.

It has no dimension, but so does isice_lu and iswater_lu, cf1, cf2, and cf3.

I am not sure why the streams manager would have trouble with this, but I conclude from where the model gets stuck that it must have some trouble reading nominalMinDc is its current formatting.

This bug may be difficult to track down, so I'm mainly posting this just in case anyone else notices a problem like this when they start using the version 8 version of init_atmosphere.

 

[mgduda]

As you pointed out, we have other variables in initial conditions files with no dimensions (i.e., scalar variables): cf1, cf2, cf3. So it is interesting that nominalMinDc seems to be a problem. Did this issue occur when using PIO or SMIOL (or is it reproducible with both)?

 

[nickheavens-cgg]

I've so far tested it cleanly with PIO2. My test with SMIOL failed prior to the point that the hanging behaviour would have started. However, I am in the process of doing a pure SMIOL static-input-simulation test at present and will let you know what happens.

 

[nickheavens-cgg]

SMIOL has no problem using an init file with nominalMinDc.