sanitizer_posix.cc:131 "(("unable to unmap" && 0)) != (0)" (0x0, 0x0)

[Martin Quinson]

Hello,

we are great fans of the sanitizers, they catched quite a lot of bugs
in our codebase. We are really thankful for that.

I come to you because they are however getting mad at our code, and
most of the remaining errors appearing seem to be somewhere in your
code, because we trigger an assertion failure in sanitizer_posix.

(gcc 6.3.0, Debian testing)

+==20715==ERROR: AddressSanitizer failed to deallocate 0x2000 (8192) bytes at address 0x7f1096171a20
+==20715==AddressSanitizer CHECK failed: ../../../../src/libsanitizer/sanitizer_common/sanitizer_posix.cc:131 "(("unable to unmap" && 0)) != (0)" (0x0, 0x0)
+ #0 0x7f1096243ebd (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xcaebd)
+ #1 0x7f1096249b03 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xd0b03)
+ #2 0x7f109625378f (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xda78f)
+ #3 0x7f1096254430 (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xdb430)
+ #4 0x7f1096247168 (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xce168)
+ #5 0x7f10946cf4e8 in __nptl_deallocate_tsd.part.4 (/lib/x86_64-linux-gnu/libpthread.so.0+0x64e8)
+ #6 0x7f10946d05d7 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x75d7)
+ #7 0x7f10944139be in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xe89be)

Here is an example:
https://ci.inria.fr/simgrid/view/Tous/job/SimGrid-DynamicAnalysis-Sanitizer-Address/label=simgrid-debian8-64-dynamic-analysis-2/lastCompletedBuild/testReport/%28root%29/TestSuite/msg_plugin_hostload_thread/

The configure and build log (gcc 6.3.0, Debian 8):
https://ci.inria.fr/simgrid/view/Tous/job/SimGrid-DynamicAnalysis-Sanitizer-Address/label=simgrid-debian8-64-dynamic-analysis-2/lastCompletedBuild/console

The source code is available:
https://github.com/simgrid/simgrid

The script used to build on the CI server is also available:
https://github.com/simgrid/simgrid/blob/master/tools/jenkins/Sanitizers.sh

In short:
cmake -Denable_documentation=OFF -Denable_lua=ON -Denable_java=OFF \
-Denable_compile_optimizations=ON -Denable_compile_warnings=ON \
-Denable_jedule=ON -Denable_mallocators=OFF \
-Denable_smpi=ON -Denable_smpi_MPICH3_testsuite=ON -Denable_model-checking=OFF \
-Denable_memcheck=OFF -Denable_memcheck_xml=OFF -Denable_smpi_ISP_testsuite=ON -Denable_coverage=OFF\
-Denable_fortran=OFF -Denable_address_sanitizer=ON \
-Denable_undefined_sanitizer=OFF -Denable_thread_sanitizer=OFF
make -j4
examples/msg/plugin-hostload/plugin-hostload examples/platforms/energy_platform.xml --cfg=contexts/factory:thread

Attached: strace output of the offending command. Note that the strace
was captured on my laptop, not on the server. I tried to reproduce the
same conditions, but I'm not 100% sure.

If you need more info, please just ask.

Thanks again for this magical tool,
Mt.

--
Essentially, all models are wrong, but some are useful.
-- George E. P. Box

[chet...@gmail.com]

Hi Martin, 

I am having the same issue too, it seems to be a real issue in the address sanitizer: https://github.com/apache/trafficserver/issues/1498.

In my case I am running a C++ component on RHEL 6.7, compiled with GCC 6.2.1 and using address sanitizer (-fsanitize=address -fno-omit-frame-pointer -O1 -g)

I get an error stack very similar to yours: 

==14657==ERROR: AddressSanitizer failed to deallocate 0x4000 (16384) bytes at address 0x629001220200
==14657==AddressSanitizer CHECK failed: ../../../../libsanitizer/sanitizer_common/sanitizer_posix.cc:131 "(("unable to unmap" && 0)) != (0)" (0x0, 0x0)
==14657==ERROR: AddressSanitizer failed to deallocate 0x4000 (16384) bytes at address 0x62900121b200
==14657==AddressSanitizer CHECK failed: ../../../../libsanitizer/sanitizer_common/sanitizer_posix.cc:131 "(("unable to unmap" && 0)) != (0)" (0x0, 0x0)
    #0 0x33b18cb2cd  (/usr/lib64/libasan.so.3+0x33b18cb2cd)
    #1 0x33b18d0f13 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) (/usr/lib64/libasan.so.3+0x33b18d0f13)
    #2 0x33b18dacaf  (/usr/lib64/libasan.so.3+0x33b18dacaf)
    #3 0x33b18db970  (/usr/lib64/libasan.so.3+0x33b18db970)
    #0 0x33b18cb2cd  (/usr/lib64/libasan.so.3+0x33b18cb2cd)
    #4 0x33b18ce578  (/usr/lib64/libasan.so.3+0x33b18ce578)
    #5 0x3c8ae07ae8 in start_thread (/lib64/libpthread.so.0+0x3c8ae07ae8)
    #6 0x3c8aae896c in __clone (/lib64/libc.so.6+0x3c8aae896c)

Let me know if you have any clue on how to overcome this issue. 

Also, agree with your sentiment about the sanitizers, they are awesome!

Marco

[Dmitry Vyukov]

Hi,

Somehow this was forgotten. Sorry.
This seems to be all about AddressSanitizer, so +address-sanitizer@
mailing list.
Unmaps at 0x629001220200 seems to be allocator returning memory back
to OS. +Kostya

[Kostya Serebryany]

I've never seen anything like this before :(

Please file an issue at https://github.com/google/sanitizers/issues

The smaller the reproducer and the more details you provide the more likely someone will be able to help. 

--kcc

[alek...@google.com]

To test if it is returning memory back to OS, set allocator_release_to_os_interval_ms=-1 flag and see if it is still reproducible. -1 is a default value, so if you're not setting it explicitly, it's not causing this problem.