False positive or a real problem?

[pavel.ce...@appeartv.com]

Hi, I have tried to compile my source code with the thread sanitizer and got an error which I did not really understand. I was able to narrow the problem down to a short example (see attached file). To me this seems like some sort of false positive. Can you please take a look at it? And if it is false positive, can it be avoided somehow? Blacklisted?

Thank you for your time, the thread sanitizer is a great tool (I have already fixed some real bugs thanks to it! And maybe this is my bug as well, but I just do not know C++ enough?).

Pavel Cernohorsky

---

$ clang++ --version
clang version 3.8.1-24 (tags/RELEASE_381/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

- but we reported by newer releases as well

$ clang++ --std=c++1z -ggdb3 -fsanitize=thread thread_sanitizer_problem.cpp

$ while ./a.out ; do :; done
... data race in between the future's get and promise's delete? why?

[Kuba Mracek]

Can you try a newer clang? Can you try using an instrumented libcxx? I saw some false positives with std::future last year on macOS, and we fixed that with <https://reviews.llvm.org/D27797>. Maybe it's not the same issue, though.

Kuba

-- 
You received this message because you are subscribed to the Google Groups "thread-sanitizer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to thread-sanitiz...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
<thread_sanitizer_problem.cpp>

[pavel.ce...@appeartv.com]

Hello Kuba,

Thank you for your answer.

First of all sorry for double posting. I wasn’t member of the group, so I became one and posted again, but my previous post appeared later.

I have tried newer clang (clang version 5.0.0-+rc2-1 (tags/RELEASE_500/rc2)) and the problem is still reported. The reported message differs (it points to the ~promise against touching the exception data in main, not the future.get as before), but something is still reported.

How do I get instrumented libcxx? Unfortunately, llvm.org seems to be down for several hours already and I am not able to find any instructions for building it. And I even remember that TSan documentation mentioned that steps tend to change a lot, so they were not written there on the TSan web pages.

I tried your test code from the commit and nothing is reported even on my older clang, so the problem will be something bit different, but I bet very close…

Kind regards,
Pavel

[Kuba Mracek]

There are folks on this mailing list who know better (I'm not a Linux guy), but getting the instrumented libcxx means building clang+llvm+compiler-rt+libcxx using CMake and that will also produce "libcxx_tsan_x86_64" which you can then build and link against.

Kuba

[Dmitry Vyukov]

Here is the report I got with clang version 5.0.0 (trunk 303084):

==================
WARNING: ThreadSanitizer: data race (pid=28075)
Write of size 8 at 0x7b08000016d8 by thread T2:
#0 operator delete(void*)
/home/dvyukov/src/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:73
(a.out+0x4b5971)
#1 std::runtime_error::~runtime_error() <null> (libstdc++.so.6+0x72947)
#2 std::__future_base::_Result<void>::_M_destroy()
include/c++/4.8/future:544:27 (a.out+0x4bab53)
#3 std::__future_base::_State_base::~_State_base() <null>
(libstdc++.so.6+0xb0ab9)
#4 _ZNSt16allocator_traitsISaINSt13__future_base11_State_baseEEE10_S_destroyIS1_EENSt9enable_ifIXsr16__destroy_helperIT_EE5valueEvE4typeERS2_PS6_
include/c++/4.8/bits/alloc_traits.h:281:8 (a.out+0x4ba9ae)
#5 void std::allocator_traits<std::allocator<std::__future_base::_State_base>
>::destroy<std::__future_base::_State_base>(std::allocator<std::__future_base::_State_base>&,
std::__future_base::_State_base*)
include/c++/4.8/bits/alloc_traits.h:405:4 (a.out+0x4ba96e)
#6 std::_Sp_counted_ptr_inplace<std::__future_base::_State_base,
std::allocator<std::__future_base::_State_base>,
(__gnu_cxx::_Lock_policy)2>::_M_dispose()
include/c++/4.8/bits/shared_ptr_base.h:407:9 (a.out+0x4ba5e8)
#7 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()
include/c++/4.8/bits/shared_ptr_base.h:144:6 (a.out+0x4b88bc)
#8 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count()
include/c++/4.8/bits/shared_ptr_base.h:546:11 (a.out+0x4b8882)
#9 std::__shared_ptr<std::__future_base::_State_base,
(__gnu_cxx::_Lock_policy)2>::~__shared_ptr()
include/c++/4.8/bits/shared_ptr_base.h:781:31 (a.out+0x4b66f9)
#10 std::promise<void>::~promise() include/c++/4.8/future:1145:7
(a.out+0x4bade5)
#11 ConnectionContext::~ConnectionContext()
thread_sanitizer_problem.cpp:8:8 (a.out+0x4bad65)
#12 void __gnu_cxx::new_allocator<ConnectionContext>::destroy<ConnectionContext>(ConnectionContext*)
include/c++/4.8/ext/new_allocator.h:124:35 (a.out+0x4bad45)
#13 _ZNSt16allocator_traitsISaI17ConnectionContextEE10_S_destroyIS0_EENSt9enable_ifIXsr16__destroy_helperIT_EE5valueEvE4typeERS1_PS5_
include/c++/4.8/bits/alloc_traits.h:281:8 (a.out+0x4bad1e)
#14 void std::allocator_traits<std::allocator<ConnectionContext>
>::destroy<ConnectionContext>(std::allocator<ConnectionContext>&,
ConnectionContext*) include/c++/4.8/bits/alloc_traits.h:405:4
(a.out+0x4bacde)
#15 std::_Sp_counted_ptr_inplace<ConnectionContext,
std::allocator<ConnectionContext>,
(__gnu_cxx::_Lock_policy)2>::_M_dispose()
include/c++/4.8/bits/shared_ptr_base.h:407:9 (a.out+0x4b9b98)
#16 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()
include/c++/4.8/bits/shared_ptr_base.h:144:6 (a.out+0x4b88bc)
#17 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count()
include/c++/4.8/bits/shared_ptr_base.h:546:11 (a.out+0x4b8882)
#18 std::__shared_ptr<ConnectionContext,
(__gnu_cxx::_Lock_policy)2>::~__shared_ptr()
include/c++/4.8/bits/shared_ptr_base.h:781:31 (a.out+0x4b63c9)
#19 void std::_Bind_simple<void
(*(std::shared_ptr<ConnectionContext>))(std::shared_ptr<ConnectionContext>)>::_M_invoke<0ul>(std::_Index_tuple<0ul>)
include/c++/4.8/functional:1731:11 (a.out+0x4bc5ef)
#20 std::_Bind_simple<void
(*(std::shared_ptr<ConnectionContext>))(std::shared_ptr<ConnectionContext>)>::operator()()
include/c++/4.8/functional:1720:16 (a.out+0x4bc575)
#21 std::thread::_Impl<std::_Bind_simple<void
(*(std::shared_ptr<ConnectionContext>))(std::shared_ptr<ConnectionContext>)>
>::_M_run() include/c++/4.8/thread:115:13 (a.out+0x4bc109)
#22 <null> <null> (libstdc++.so.6+0xb1a5f)

Previous read of size 1 at 0x7b08000016db by main thread:
#0 strlen projects/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:299
(a.out+0x473cbf)
#1 main thread_sanitizer_problem.cpp:46:11 (a.out+0x4b5da0)

Thread T2 (tid=28550, running) created by main thread at:
#0 pthread_create
/home/dvyukov/src/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:890
(a.out+0x449a16)
#1 std::thread::_M_start_thread(std::shared_ptr<std::thread::_Impl_base>)
<null> (libstdc++.so.6+0xb1cae)
#2 asyncGetConnection(std::thread&)
thread_sanitizer_problem.cpp:29:22 (a.out+0x4b5cab)
#3 main thread_sanitizer_problem.cpp:38:42 (a.out+0x4b5d42)

SUMMARY: ThreadSanitizer: data race
/home/dvyukov/src/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:73
in operator delete(void*)
==================


Unfortunately, it includes frames from libstdc++.so, so I guess you
need an instrumented libc++.

[pavel.ce...@appeartv.com]

Hello Kuba and Dmitry,

Thank you for your answers! The documentation "https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual" mentions that "...to rebuild libstdc++ (libc++) with ThreadSanitizer (this might be tricky, and the process changes periodically; contact us for details)" and indeed, I was not able to find any detailed instructions, but by combining several sources I was able to get to the instrumented libc++ and indeed the reported problem disappeared afterwards.

Again, thank you for your time.

If somebody gets into the same situation in the future, below you can find steps to repeat the test and get instrumented libc++ inside controlled environment in docker.

Kind regards,
Pavel

# mostly based on https://clang.llvm.org/get_started.html, https://github.com/google/sanitizers/wiki/MemorySanitizerLibcxxHowTo and https://groups.google.com/forum/#!topic/thread-sanitizer/yyC7L1FYW5c

docker run --rm -it debian:buster bash
apt-get update
apt-get install cmake g++ python subversion

cd ~
svn co http://llvm.org/svn/llvm-project/llvm/tags/RELEASE_500/final llvm
cd llvm/tools
svn co http://llvm.org/svn/llvm-project/cfe/tags/RELEASE_500/final clang
cd clang/tools
svn co http://llvm.org/svn/llvm-project/clang-tools-extra/tags/RELEASE_500/final extra
cd ../../../projects/
svn co http://llvm.org/svn/llvm-project/compiler-rt/tags/RELEASE_500/final compiler-rt
svn co http://llvm.org/svn/llvm-project/libcxx/tags/RELEASE_500/final libcxx
svn co http://llvm.org/svn/llvm-project/libcxxabi/tags/RELEASE_500/final libcxxabi
cd ../..

mkdir build.normal
cd build.normal
cmake ../llvm -DCMAKE_BUILD_TYPE=Release
make -j8
make install
cd ..
ldconfig

mkdir build.tsan
cd build.tsan
cmake ../llvm -DCMAKE_BUILD_TYPE=Release -DLLVM_USE_SANITIZER=Thread -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
make -j8
cd ..

# get the application into the docker
# build the application
clang++ thread_sanitizer_problem.cpp --std=c++1z -ggdb3 -stdlib=libc++ -fsanitize=thread
# run the application
LD_LIBRARY_PATH=build.tsan/lib ./a.out

[Dmitry Vyukov]

Thanks for providing the updated instructions!