Starting Cost Function Gradient is NaN

Post Reply
danny7kd
Posts: 9
Joined: Thu Dec 13, 2018 7:40 pm

Starting Cost Function Gradient is NaN

Post by danny7kd » Thu Jan 10, 2019 4:14 am

Hi all,

I've finally got WRFDA running in 4DVAR mode, but it's failing to converge because of a cost function calculation issue. After the first round I get:
Loop Iter Cost Function Gradient Step
minimize_cg 1 0 2.059757428430507D+03 NaN 0.000000000000000D+00

After the second round, it's
minimize_cg 1 1 NaN NaN 0.000000000000000D
+00

I've attached the full rsl.error and rs.out files, as well as the namelist, in case those are helpful.

Any thoughts about what would cause the cost function calculation to blow up?

Thanks!
Dan
Attachments
namelist.input
(2.05 KiB) Downloaded 38 times
rsl.error.0000.txt
(293.43 KiB) Downloaded 35 times
rsl.out.0000.txt
(297.49 KiB) Downloaded 36 times

kwthomas
Posts: 73
Joined: Tue Nov 06, 2018 10:25 pm

Re: Starting Cost Function Gradient is NaN

Post by kwthomas » Thu Jan 10, 2019 10:17 pm

Hi Dan...

I'm not a WRFDA user, however, I did notice something. If you look at the timestep times in rsl.out.0000,
there seems to be a lot of bouncing around, especially to higher numbers, then fall back to lower numbers.

This suggests to me that there may be a numerical instability somewhere. If something is marginally unstable,
it *might* be a reason for the NaN problem.

Check all the rsl.out* and rsl.error* files for complaints.

You didn't way what version you are using. If it isn't current, you might try upgrading to see if that helps.
Kevin W. Thomas
Center for Analysis and Prediction of Storms
University of Oklahoma
Norman, Oklahoma

danny7kd
Posts: 9
Joined: Thu Dec 13, 2018 7:40 pm

Re: Starting Cost Function Gradient is NaN

Post by danny7kd » Fri Jan 11, 2019 6:20 pm

HI Kevin,

Thanks, that's worth checking- I could drop the timestep down to 10 or 15 seconds (though it'll cost me...). This is version 4.0.1.

Dan

Post Reply

Return to “Operational Implementation”