Scheduled Downtime
On Friday 21 April 2023 @ 5pm MT, this website will be down for maintenance and expected to return online the morning of 24 April 2023 at the latest

dawrfvar crashes with double free or corruption

This post was from a previous version of the WRF&MPAS-A Support Forum. New replies have been disabled and if you have follow up questions related to this post, then please start a new thread from the forum home page.

Feng Liu

Member
Hi,
I am going to run WRFDA 4DVAR (WRF v4.1.2) using ds337.0 (NCEP ADP Global Upper Air and Surface Weather Observations (PREPBUFR format). The program da_wrfvar.exe always crashes with an error as below:
I am struggling to figure out the issue but in vain. I would like to get your help because I know you are an outstanding expert in WRFDA.

*** Error in `./da_wrfvar.exe': double free or corruption (!prev): 0x000000008c132040 ***
Sometimes it becomes

*** Error in `./da_wrfvar.exe': corrupted double-linked list: 0x0000000074453170 ***

More detail screen output can be found in the wa_wrfvar.log attached.

The rsl.error and rsl.out (see attached) do not show any error messages even though a high debug level is set in namelist.input (attached).

I also used compile option, Mbounds, to check compilation, and the program Valgrind ( (http://valgrind.org/) to detect memory management while running da_wrfvar.exe. No any errors and problems were founded. I do not know if we need to concern the WARNING like

--------------------------- WARNING ---------------------------
WARNING FROM FILE: da_read_obs_bufrgpsro.inc LINE: 115
Error 209 opening PREPBUFR obs file gpsro.bufr
---------------------------------------------------------------


Please also see the namelist.input file attached.

What I did in my work directory WRFDA/workdir are
1. Copy entire WRFDA/run as new directory workdir.
2. Link the ../var/run/be.dat.CV3 as be.dat at the workdir.
3. Link wrfbdy_d01 and wrfinput_d01 at the same directory.
4. Link wrfinput_d01 as fg.
5. Link prepbufr.gdas.20131228.t00z.nr and prepbufr.gdas.20131228.t06z.nr as ob01.bufr and ob02.bufr, respectively.
6. I also link or changed three files’ name
a. ln –sf RRTM_DATA_DBL RRTM_DATA
b. ln –sf RRTMG_LW_DATA_DBL RRTMG_LW_DATA
c. ln –sf RRTMG_SW_DATA_DBL RRTMG_SW_DATA
Because if I do not change above files' names the following error, for example if not change RRTMG_LW_DATA_DBL into RRTMG_LW_DATA
FATAL CALLED FROM FILE: <stdin> LINE: 12932
module_ra_rrtmg_lw: error reading RRTMG_LW_DATA on unit 10


Thank you very much for taking the time to help me.
 

Attachments

  • logfile.tar
    1.2 MB · Views: 77
Hi,
For more details, please find debug information on this issue

(gdb) info source
Current source file is ../nptl/sysdeps/unix/sysv/linux/raise.c
Compilation directory is /usr/src/debug/glibc-2.17-c758a686/signal
Located in /usr/src/debug/glibc-2.17-c758a686/nptl/sysdeps/unix/sysv/linux/raise.c
Contains 58 lines.
Source language is c.
Compiled with DWARF 2 debugging format.
Does not include preprocessor macro info.
(gdb) bt
#0 0x00002aaaaee302c7 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55
#1 0x00002aaaaee319b8 in __GI_abort () at abort.c:90
#2 0x00002aaaaee72e17 in __libc_message (do_abort=do_abort@entry=2,
fmt=fmt@entry=0x2aaaaef847f8 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/unix/sysv/linux/libc_fatal.c:196
#3 0x00002aaaaee7b609 in malloc_printerr (ar_ptr=0x2aaaaf1c0760 <main_arena>, ptr=<optimized out>,
str=0x2aaaaef84920 "double free or corruption (!prev)", action=3) at malloc.c:5004
#4 _int_free (av=0x2aaaaf1c0760 <main_arena>, p=<optimized out>, have_lock=0) at malloc.c:3843
#5 0x000000000273aab4 in a_module_em::a_rk_update_scalar () at module_em_ad.f90:3472
#6 0x00000000025926cf in solve_em_ad () at solve_em_ad.f90:6451
#7 0x00000000021953fb in solve_interface () at solve_interface.f90:221
#8 0x000000000127628a in module_integrate::integrate () at module_integrate.f90:365
#9 0x000000000041466d in module_wrf_top::wrf_run_ad () at ../main/module_wrf_top.f90:1655
#10 0x000000000047fa9c in da_4dvar::da_ad_model () at da_4dvar.f:518
#11 0x0000000000ab224c in da_minimisation::da_transform_vtoy_adj () at da_minimisation.f:3436
#12 0x0000000000a96edb in da_minimisation::da_calculate_gradj () at da_minimisation.f:596
#13 0x0000000000aa5f6c in da_minimisation::da_minimise_cg () at da_minimisation.f:2641
#14 0x00000000004621ca in da_wrfvar_top::da_solve () at da_wrfvar_top.f:3752
#15 0x000000000043a5e0 in da_wrfvar_top::da_wrfvar_interface () at da_wrfvar_top.f:2841
#16 0x000000000043a533 in da_wrfvar_top::da_wrfvar_run () at da_wrfvar_top.f:2811
#17 0x000000000040a204 in da_wrfvar_main () at da_wrfvar_main.f:35
#18 0x000000000040a176 in main ()
#19 0x00002aaaaee1c495 in __libc_start_main (main=0x40a140 <main>, argc=1, argv=0x7fffffffde28, init=<optimized out>,
fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffde18) at ../csu/libc-start.c:266
---Type <return> to continue, or q <return> to quit---
#20 0x000000000040a079 in _start ()
 
Top