intermittent failures of vmem_mix_allocations/TEST1

[Jeff Moyer]

Hi,

When running make check, I see failures of this one test. If I run the
test by hand, sometimes it fails, sometimes it succeeds.

vmem_mix_allocations/TEST1: START: vmem_mix_allocations
vmem_mix_allocations/TEST1 crashed (signal 6). err1.log below.
vmem_mix_allocations/TEST1:

vmem_mix_allocations/TEST1: Signal 6, backtrace:
vmem_mix_allocations/TEST1: 0: ./vmem_mix_allocations() [0x404063]
vmem_mix_allocations/TEST1: 1: ./vmem_mix_allocations() [0x404158]
vmem_mix_allocations/TEST1: 2: /lib64/libc.so.6(+0x35650) [0x7fe0f919c650]
vmem_mix_allocations/TEST1: 3: /lib64/libc.so.6(gsignal+0x37) [0x7fe0f919c5d7]
vmem_mix_allocations/TEST1: 4: /lib64/libc.so.6(abort+0x148) [0x7fe0f919dcc8]
vmem_mix_allocations/TEST1: 5: ../../debug/libvmem.so.1(+0x12a06) [0x7fe0f995ba0
6]
vmem_mix_allocations/TEST1: 6: ../../debug/libvmem.so.1(+0x13457) [0x7fe0f995c45
7]
vmem_mix_allocations/TEST1: 7: ../../debug/libvmem.so.1(+0x7f42) [0x7fe0f9950f42
]
vmem_mix_allocations/TEST1: 8: ../../debug/libvmem.so.1(+0xe38f) [0x7fe0f995738f
]
vmem_mix_allocations/TEST1: 9: ../../debug/libvmem.so.1(+0xef19) [0x7fe0f9957f19
]
vmem_mix_allocations/TEST1: 10: ../../debug/libvmem.so.1(vmem_free+0x65) [0x7fe0
f994d69a]
vmem_mix_allocations/TEST1: 11: ./vmem_mix_allocations() [0x401c10]
vmem_mix_allocations/TEST1: 12: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fe0
f9188af5]
vmem_mix_allocations/TEST1: 13: ./vmem_mix_allocations() [0x401899]
vmem_mix_allocations/TEST1:

RUNTESTS: stopping: vmem_mix_allocations/TEST1 failed, TEST=check FS=pmem BUILD=debug

It doesn't actually matter what file system I run the test against, they
all fail in the same way. I've attached a backtrace in case it's
useful.

I'm running the latest code for the github repo. HEAD is 3c8b19800e8d3.

-Jeff

$ gdb ./vmem_mix_allocations.static-debug
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-64.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/phro/nvml/src/test/vmem_mix_allocations/vmem_mix_allocations.static-debug...done.
(gdb) r /tmp
Starting program: /home/phro/nvml/src/test/vmem_mix_allocations/./vmem_mix_allocations.static-debug /tmp
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
START: vmem_mix_allocations
/home/phro/nvml/src/test/vmem_mix_allocations/./vmem_mix_allocations.static-debug /tmp

Program received signal SIGABRT, Aborted.
0x00007ffff762e5d7 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-78.el7.x86_64 libuuid-2.23.2-26.el7.x86_64
(gdb) bt
#0 0x00007ffff762e5d7 in raise () from /lib64/libc.so.6
#1 0x00007ffff762fcc8 in abort () from /lib64/libc.so.6
#2 0x00000000004116ee in je_vmem_je_arena_ptr_small_binind_get (
ptr=0x7ffff705536c, mapbits=1)
at /home/phro/nvml/src/jemalloc/include/jemalloc/internal/arena.h:949
#3 0x000000000041213f in je_vmem_je_arena_salloc (ptr=0x7ffff705536c,
demote=false)
at /home/phro/nvml/src/jemalloc/include/jemalloc/internal/arena.h:1145
#4 0x0000000000406c2a in je_vmem_je_isalloc (ptr=0x7ffff705536c, demote=false)
at ../debug/jemalloc/include/jemalloc/internal/jemalloc_internal.h:787
#5 0x000000000040d077 in pool_ifree (pool=0x7ffff6602000, ptr=0x7ffff705536c)
at /home/phro/nvml/src/jemalloc/src/jemalloc.c:2128
#6 0x000000000040dc01 in je_vmem_pool_free (pool=0x7ffff6602000,
ptr=0x7ffff705536c) at /home/phro/nvml/src/jemalloc/src/jemalloc.c:2336
#7 0x0000000000403382 in vmem_free (vmp=0x7ffff6600000, ptr=0x7ffff705536c)
at vmem.c:308
#8 0x0000000000402910 in main (argc=2, argv=0x7fffffffe428)
at vmem_mix_allocations.c:97

[Krzysztof Czurylo]

Hi Jeff,

Thanks for reporting this.

We have seen this problem a few days ago and have been working on identifying the root cause.

Seems like we eventually have a fix. The good news is that there is no problem in libvmem - the bug is in the unit test.

We'll merge the patch ASAP.

Krzysztof