Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

WRFDA 4Dvar run fails to converge

J

JFL

New member
Hello everyone！
I have some troubles to run my WRFDA 4Dvar case.
And I'm looking for someone who has more experience in running this module.

I want to use 4Dvar (single domain) to assimilate precipitation data, but da_wrfvar.exe has not finished running for more than 48 hours, seems to be looping infinitely.
I want to know if there is a problem with my namelist parameter setting? And why assimilation cannot end.
Specifically, the precipitation data is grid data (spatial resolution of 4km), dx=dy=12km, time_step=60s, e_we=e_sn=80.
And the other parameters set in namelist.input are shown in the attachment:

Records in rsl.error, repeat the following process：
···
wrf: back from adjoint integrate
wrf: calling tangent linear integrate
INPUT LandUse = "MODIFIED_IGBP_MODIS_NOAH"
LANDUSE TYPE = "MODIFIED_IGBP_MODIS_NOAH" FOUND 41 CATEGORIES 2 SEASONS WATER CATEGORY = 17 SNOW CATEGORY = 15
Timing for main: time 2016-05-28_12:01:00 on domain 1: 1.89321 elapsed seconds
Timing for main: time 2016-05-28_12:02:00 on domain 1: 1.85460 elapsed seconds
Timing for main: time 2016-05-28_12:03:00 on domain 1: 1.76859 elapsed seconds
···
wrf: back from tangent linear integrate
Swap time: <2016-05-28_12:00:00>and: <2016-05-28_18:00:00>
Swap time: <2016-05-28_13:00:00>and: <2016-05-28_17:00:00>
Swap time: <2016-05-28_14:00:00>and: <2016-05-28_16:00:00>
wrf: calling adjoint integrate
Timing for main: time 2016-05-28_17:59:00 on domain 1: 6.22956 elapsed seconds
Timing for main: time 2016-05-28_17:58:00 on domain 1: 5.67272 elapsed seconds
···


Is it because there are too few processes (with only 1 process, WRFDA runs for more than 48h), or is it a problem with the namelist parameter setting?
I would be very grateful for any ideas how to solve this problem.

Best regards.
 

Attachments

  • namelist.input
    4.6 KB · Views: 10
Hello JFL.

When I run 4dvar it seems to be running WRF from start time to end time and end time to start time.

Looking at your log it takes about 2 seconds per step from start time to end time and about 7 seconds per step from end time to start time.

The attached namelist.input shows NTMAX=100, which tells you to repeat the WRF from start time to end time and end time to start time 100 times.

So in total, it is expected to take more than 90 hours.

Sincerely yours
 
KiyoTom said:
Hello JFL.

When I run 4dvar it seems to be running WRF from start time to end time and end time to start time.

Looking at your log it takes about 2 seconds per step from start time to end time and about 7 seconds per step from end time to start time.

The attached namelist.input shows NTMAX=100, which tells you to repeat the WRF from start time to end time and end time to start time 100 times.

So in total, it is expected to take more than 90 hours.

Sincerely yours
Click to expand...
Hi Tom,

Thanks for your helpful reply！
This is indeed time expensive, I have reset the 16-process run, and waited for it to finish running to make sure it was not an issue with the namelist parameter setting or data set.

Best regards.
 
Hello,

Something I learned is that WRF4DVAR uses a lot of memory so make sure you have adequate RAM in your system otherwise it'll freeze up and stop running.

Cheers,
JeremyB
 
JeremyB said:
Hello,

Something I learned is that WRF4DVAR uses a lot of memory so make sure you have adequate RAM in your system otherwise it'll freeze up and stop running.

Cheers,
JeremyB
Click to expand...
Hi JeremyB,

It's true that 4Dvar is computationally expensive, my wrfda finished running after about two days.

However, I encountered a new problem:

During the test at a higher grid resolution (e.g., dx=4km, 16-process run), I found that WRFDA would stay at a certain period, but no error was reported in the rsl.error. file, the program seems to be stalled.

The details set in namelist.input are shown in the attachment.

Anyone know what's going on?

Best regards.
 

Attachments

  • namelist.input
    4.6 KB · Views: 7
JFL said:
Hi JeremyB,

It's true that 4Dvar is computationally expensive, my wrfda finished running after about two days.

However, I encountered a new problem:

During the test at a higher grid resolution (e.g., dx=4km, 16-process run), I found that WRFDA would stay at a certain period, but no error was reported in the rsl.error. file, the program seems to be stalled.

The details set in namelist.input are shown in the attachment.

Anyone know what's going on?

Best regards.
Click to expand...
Hello JFL,

Well for one thing you have ntmax = 100 so it will take quite a while (100 attempts to converge 6 hours). So that's why it takes 2 days. As for why it's staying at a certain period, you're likely running into a memory issue. I had that same issue, and I resolved it by reducing the number of hours I'm running da_wrfvar.exe for (3 hours instead of 6 hours) and increasing the RAM. That fixed it. What I would do to check it is run da_wrfvar.exe in a screen (screen -S WRFDA), then, while it's running, exit the screen (Ctrl + A + D). When you're back in the original screen, type in the command "free -m" That will tell you how much memory you're using and how much is left. I hope that helps.

Cheers,
JeremyB
 
JeremyB said:
Hello JFL,

Well for one thing you have ntmax = 100 so it will take quite a while (100 attempts to converge 6 hours). So that's why it takes 2 days. As for why it's staying at a certain period, you're likely running into a memory issue. I had that same issue, and I resolved it by reducing the number of hours I'm running da_wrfvar.exe for (3 hours instead of 6 hours) and increasing the RAM. That fixed it. What I would do to check it is run da_wrfvar.exe in a screen (screen -S WRFDA), then, while it's running, exit the screen (Ctrl + A + D). When you're back in the original screen, type in the command "free -m" That will tell you how much memory you're using and how much is left. I hope that helps.

Cheers,
JeremyB
Click to expand...
Hi JeremyB,

Thank you very much for your patience and prompt reply! I will try according to your suggestion.

Yours sincerely,

JFL
 
@JFL and @JeremyB

Do you have experience in converting precipitation data in GeoTIFF for WRFDA precipitation data assimilation? I have rainfall products in .tif format that I want to use with WRFDA
 
You must log in or register to reply here.
Top