Error in WRF-Chem runs at the termination

196 views
Skip to first unread message

Sreyashi Debnath

unread,
Apr 23, 2021, 3:29:41 AM4/23/21
to wrf-chem
Dear members,

I have run two sets of WRF-Chem simulations, and it is expected that on completion of the simulations one must get the  message 'success complete WRF'. However, in my cases, although the simulations have executed for the entire period with proper generation of wrfout* and wrfrst* files,  each time the wrf execution is ending with the following error message:

*** Error in `./wrf.exe': double free or corruption (out): 0x000000000ed8acd0 ***

======= Backtrace: =========

/lib64/libc.so.6(+0x721af)[0x2aaaacb861af]

/lib64/libc.so.6(+0x77706)[0x2aaaacb8b706]

/lib64/libc.so.6(+0x78453)[0x2aaaacb8c453]

/opt/cray/pe/lib64/libnetcdf_intel.so.11(NC_hashmapDelete+0x13)[0x2aaaab205e33]

/opt/cray/pe/lib64/libnetcdf_intel.so.11(free_NC_vararrayV+0x1e)[0x2aaaab1ea58e]

/opt/cray/pe/lib64/libnetcdf_intel.so.11(NC3_close+0xe4)[0x2aaaab1e8c34]

/opt/cray/pe/lib64/libnetcdf_intel.so.11(nc_close+0x2e)[0x2aaaab19ffce]

./wrf.exe[0x4485de2]

./wrf.exe[0x1dbed33]

./wrf.exe[0x1b2483a]

./wrf.exe[0x1c8237c]

./wrf.exe[0x40d3c8]

./wrf.exe[0x40d34b]

./wrf.exe[0x40d2de]

/lib64/libc.so.6(__libc_start_main+0xf5)[0x2aaaacb346d5]

./wrf.exe[0x40d1e9]

======= Memory map: ========

00400000-049b1000 r-xp 00000000 00:06 107307931                          /var/opt/cray/alps/spool/169339168/wrf.exe

04bb1000-04bb2000 r--p 045b1000 00:06 107307931                          /var/opt/cray/alps/spool/169339168/wrf.exe

04bb2000-04da8000 rw-p 045b2000 00:06 107307931                          /var/opt/cray/alps/spool/169339168/wrf.exe

04da8000-0c221000 rw-p 00000000 00:00 0                                  [heap]

0c221000-13f0e000 rw-p 00000000 00:00 0                                  [heap]

2aaaaaaab000-2aaaaaacc000 r-xp 00000000 00:02 50128                      /lib64/ld-2.22.so

2aaaaaacc000-2aaaaaacf000 r--p 00000000 00:00 0                          [vvar]

2aaaaaacf000-2aaaaaad1000 r-xp 00000000 00:00 0                          [vdso]

2aaaaaad1000-2aaaaaad2000 rw-p 00000000 00:00 0

2aaaaaad2000-2aaaaaad3000 rw-p 20144a6000 00:06 1877                     /dev/kgni0

2aaaaaad3000-2aaaaaad5000 rw-p 2010b82000 00:06 1877                     /dev/kgni0

2aaaaaad5000-2aaaaaad6000 r--p 619134000 00:06 1877                      /dev/kgni0

2aaaaaad6000-2aaaaaad8000 rw-p d6710a000 00:06 1877                      /dev/kgni0

.

.

.

.

.

.

.

.

.

2aaadccb6000-2aaadccd4000 rw-s 00000000 00:00 0                          /dev/xpmem

2aaae0000000-2aaae0021000 rw-p 00000000 00:00 0

2aaae0021000-2aaae4000000 ---p 00000000 00:00 0

7ffffbe42000-7fffffff6000 rwxp 00000000 00:00 0                          [stack]

7fffffff6000-7ffffffff000 rw-p 00000000 00:00 0

ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

forrtl: error (76): Abort trap signal

Image              PC                Routine            Line        Source

wrf.exe            00000000044E635A  for__signal_handl     Unknown  Unknown

libpthread-2.22.s  00002AAAAC60AB20  Unknown               Unknown  Unknown

libc-2.22.so       00002AAAACB488C7  gsignal               Unknown  Unknown

libc-2.22.so       00002AAAACB49C9A  abort                 Unknown  Unknown

libc-2.22.so       00002AAAACB861B4  Unknown               Unknown  Unknown

libc-2.22.so       00002AAAACB8B706  Unknown               Unknown  Unknown

libc-2.22.so       00002AAAACB8C453  Unknown               Unknown  Unknown

libnetcdf_intel.s  00002AAAAB205E33  NC_hashmapDelete      Unknown  Unknown

libnetcdf_intel.s  00002AAAAB1EA58E  free_NC_vararrayV     Unknown  Unknown

libnetcdf_intel.s  00002AAAAB1E8C34  NC3_close             Unknown  Unknown

libnetcdf_intel.s  00002AAAAB19FFCE  nc_close              Unknown  Unknown

wrf.exe            0000000004485DE2  Unknown               Unknown  Unknown

wrf.exe            0000000001DBED33  Unknown               Unknown  Unknown

wrf.exe            0000000001B2483A  Unknown               Unknown  Unknown

wrf.exe            0000000001C8237C  Unknown               Unknown  Unknown

wrf.exe            000000000040D3C8  Unknown               Unknown  Unknown

wrf.exe            000000000040D34B  Unknown               Unknown  Unknown

wrf.exe            000000000040D2DE  Unknown               Unknown  Unknown

libc-2.22.so       00002AAAACB346D5  __libc_start_main     Unknown  Unknown

wrf.exe            000000000040D1E9  Unknown               Unknown  Unknown

I am getting this error for the first time while performing WRF-Chem simulations in my system and do not have much idea about it.

I would be obliged if somebody can help me in this regard.

Regards
Sreyashi Debnath
IITM, Pune, India

Stacy Walters

unread,
Apr 23, 2021, 7:40:06 AM4/23/21
to Sreyashi Debnath, wrf-chem
Sreyashi,

Looks like WRF is deallocating the same variable twice.  What WRF version
are you running?  Debugging this would require more time and resources
than ACOM has available.

Stacy

Sreyashi Debnath

unread,
Apr 23, 2021, 7:44:40 AM4/23/21
to Stacy Walters, wrf-chem
Dear Stacy,

I am using v3.9.1. Is this something to worry about when the output files for the entire simulation period have got generated? 

Regards,
Sreyashi

Stacy Walters

unread,
Apr 23, 2021, 10:18:09 AM4/23/21
to Sreyashi Debnath, wrf-chem
Sreyashi,

If you can generate a restart file as well at the end of the run then you can
use your executable as is.  

Stacy

Sreyashi Debnath

unread,
Apr 23, 2021, 10:57:12 AM4/23/21
to Stacy Walters, wrf-chem
Dear Stacy,

Thank you for your comment on this. Yes, restart file has got generated at the end of the run as well.

Thanks and regards,
Sreyashi
Reply all
Reply to author
Forward
0 new messages