stack-buffer-overflow on addressable memory

[Martin Ri]

Hi,

has anyone seen reports like the one below? A stack-buffer-overflow is reported, but the memory is addressable.
Notes: I have seen similar instances reported as "unknown-crash". Looking at the code in asan-report.cc, this is the initial error code, and the real error is determined by looking at the shadow. So it seems this could be some data race: When the shadow is initially checked as part of the instrumented code, the memory is marked as invalid and error reporting is triggered. But shortly afterwards it gets marked as valid. However, I can't imagine how that could happen: The object lives on the stack of thread 0, this thread is still running, and the frame should still be valid (which is also what ASan reports). The reported offset 44 also matches the accessed object in BinarySemaphore::open().

The involved code uses futexes, and I suspect this somehow causes the problem. But I'm not sure whether it is a problem of the codebase (at least we haven't seen anything like it in practice), or a limitation/bug of ASan.

==39533==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fff656ad0ec at pc 0x7f1a103a4d6b bp 0x7f17e0181e10 sp 0x7f17e0181e08
READ of size 4 at 0x7fff656ad0ec thread T182 (Pool)
    #0 0x7f1a103a4d6a in Synchronization::BinarySemaphore::open(Execution::Context&)
    #1 0x7f1a5fc64e25 in TrexSync::Event::signal()
    #2 0x7f1a47ecbdf3 in TrexSync::EventGuard::signal()
    #3 0x7f1a47ec83c2 in TrexThreads::PoolThread::run()
    #4 0x7f1a47ec6a12 in TrexThreads::PoolThread::run(void*&)
    #5 0x7f1a100a8a80 in Execution::Thread::staticMainImp(void**)
    #6 0x7f1a100b37ff in Execution::Thread::staticMain(void*)

Address 0x7fff656ad0ec is located in stack of thread T0 at offset 44 in frame
    #0 0x7f1a47ed09df in TrexThreads::ThreadManager::create(TrexThreads::Thread*, void*, TrexThreads::_CreateFlags, int)

  This frame has 1 object(s):
    [32, 160) 'aCreationInfo' <== Memory access at offset 44 is inside this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow
Shadow bytes around the buggy address:
  0x10006cacd9c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10006cacd9d0: 00 00 00 00 f1 f1 f1 f1 04 f2 04 f2 04 f2 04 f2
  0x10006cacd9e0: 04 f2 04 f2 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2
  0x10006cacd9f0: 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2
  0x10006cacda00: 04 f2 04 f2 00 f3 f3 f3 00 00 00 00 00 00 00 00
=>0x10006cacda10: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00[00]00 00
  0x10006cacda20: 00 00 00 00 00 00 00 00 00 00 00 00 f3 f3 f3 f3
  0x10006cacda30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10006cacda40: 00 00 00 00 f1 f1 f1 f1 04 f2 04 f2 04 f2 04 f2
  0x10006cacda50: 04 f2 00 00 f2 f2 00 00 f3 f3 f3 f3 00 00 00 00
  0x10006cacda60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00

Best regards,
Martin

[Yury Gribov]

On 03/02/2015 12:44 PM, Martin Ri wrote:
> Hi,
>
> has anyone seen reports like the one below? A stack-buffer-overflow is
> reported, but the memory is addressable.
> Notes: I have seen similar instances reported as "unknown-crash". Looking
> at the code in asan-report.cc, this is the initial error code, and the real
> error is determined by looking at the shadow. So it seems this could be
> some data race: When the shadow is initially checked as part of the
> instrumented code, the memory is marked as invalid and error reporting is
> triggered. But shortly afterwards it gets marked as valid.

Hi Martin,

We have seen something like this in KASan. But there it was clearly
caused by use-after-return which is not obvious in your case.

-Y

[Dmitry Vyukov]

It can be a racy use-after-return. When asan detects the bug the
shadow is poisoned, but when it reports it, another thread already
unpoisoned the memory.
Since the stack trace contains words like Thread and BinarySemaphore,
I would look for a data race first. Try to run under ThreadSanitizer
(-fsanitize=thread).

[Martin Ri]

Hi,

thanks for the responses. Unfortunately I cannot use ThreadSanitizer since my application uses C++ exceptions.

Best regards,

Martin