Issue 193 in address-sanitizer: Abort "attempting to call malloc_usable_size() for pointer which is not owned" with posix_memalign and malloc_usable_size

[address-...@googlecode.com]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 193 by decoder...@googlemail.com: Abort "attempting to call
malloc_usable_size() for pointer which is not owned" with posix_memalign
and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

What steps will reproduce the problem?

1. Compile the following program with ASan:

#include <stdio.h>
#include <stdlib.h>
#include <malloc.h>

int main() {
void* ptr;
if (!posix_memalign(&ptr, 16, 36)) {
return malloc_usable_size(ptr);
}
return 0;
}


2. Run and observe error.

What is the expected output? What do you see instead?

Expected behavior is to not crash, however, we get this:

==32439==ERROR: AddressSanitizer: attempting to call malloc_usable_size()
for pointer which is not owned: 0x60500000efc0

What version of the product are you using? On what operating system?

Tested on LLVM r183459 and LLVM 3.3 branch, both on Ubuntu 12.04 LTS (64
bit).


Please provide any additional information below.

The issue does not reproduce for other values of alignment and size to
posix_memalign. I got these values from one of our failing Firefox tests.
If you could fix this, then we would highly appreciate if a fix could be
backported to the 3.3 branch because we plan to update to this branch soon.
Thanks! :)


--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[address-...@googlecode.com]

Comment #1 on issue 193 by justin.l...@gmail.com: Abort "attempting to call

malloc_usable_size() for pointer which is not owned" with posix_memalign
and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

It's not clear to me that calling malloc_usable_size on a posix_memalign'ed
pointer is safe in general. I'm pretty sure it's safe with our allocator,
however.

The right thing might be to make this behavior configurable.

[address-...@googlecode.com]

Updates:
Status: Accepted
Owner: samso...@google.com

Comment #2 on issue 193 by samso...@google.com: Abort "attempting to call

malloc_usable_size() for pointer which is not owned" with posix_memalign
and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

I see no reason why ASan should report an error for malloc_usable_size() on
posix_memalign(). Looks like it's just a bug.

[address-...@googlecode.com]

Updates:
Owner: konstant...@gmail.com

Comment #3 on issue 193 by samso...@google.com: Abort "attempting to call

malloc_usable_size() for pointer which is not owned" with posix_memalign
and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

The problem is:
when we're looking for allocation metadata (aka AsanChunk header), we don't
know the redzone size in advance, so we're trying to calculate redzone size
from the actual number of bytes we allocated. This doesn't work, as we
calculate redzone size before respecting alignment.

Quick fix for this may be: calculate redzone size after we increase the
user-requested size because of alignment. However, this may blow up the
redzone sizes for allocations with huge alignments.

[address-...@googlecode.com]

Updates:
Status: Started

Comment #4 on issue 193 by konstant...@gmail.com: Abort "attempting to call

malloc_usable_size() for pointer which is not owned" with posix_memalign
and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

I am on it, will take closer look on Monday. Thanks for the report.

[address-...@googlecode.com]

Comment #5 on issue 193 by konstant...@gmail.com: Abort "attempting to call

malloc_usable_size() for pointer which is not owned" with posix_memalign
and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

BTW, you may use ASAN_OPTIONS=check_malloc_usable_size=0 as a temporary
workaround.

[address-...@googlecode.com]

Updates:
Status: Fixed

Comment #6 on issue 193 by konstant...@gmail.com: Abort "attempting to call

malloc_usable_size() for pointer which is not owned" with posix_memalign
and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

Should be fixed by http://llvm.org/viewvc/llvm-project?rev=183647&view=rev

[address-...@googlecode.com]

Comment #7 on issue 193 by decoder...@googlemail.com: Abort "attempting to

call malloc_usable_size() for pointer which is not owned" with
posix_memalign and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

Fix confirmed, thanks. We will try and see if we can backport this to the
3.3 branch.

[address-...@googlecode.com]

Comment #8 on issue 193 by decoder...@googlemail.com: Abort "attempting to

call malloc_usable_size() for pointer which is not owned" with
posix_memalign and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

kcc, we have tried to port this to the 3.3 branch but we get a startup
crash with our patch. Would you mind helping us to port this to 3.3? It
could also be landed in a maintenance branch then once it has been opened.

Not sure what is easier for you, taking a look at our patch or just
creating the backport yourself.

[address-...@googlecode.com]

Updates:
Cc: samso...@google.com

Comment #9 on issue 193 by konstant...@gmail.com: Abort "attempting to call

malloc_usable_size() for pointer which is not owned" with posix_memalign
and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

Please let us see your start-up crash (stack trace)

Also, Alexey just added another small fix in the same general area.
http://llvm.org/viewvc/llvm-project?rev=184404&view=rev
W/o this change we had a crash in LSan, but theoretically it could cause a
crash in ASan too.

[address-...@googlecode.com]

Comment #10 on issue 193 by decoder...@googlemail.com: Abort "attempting to

call malloc_usable_size() for pointer which is not owned" with
posix_memalign and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

I figured that the trace wouldn't be very helpful to you. It's related to
memalign:

==2677==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x602000033450 at pc 0x7fa1925865ea bp 0x7fffc3c349c0 sp 0x7fffc3c349b8
READ of size 4 at 0x602000033450 thread T0
#0 0x7fa1925865e9 in NS_IsMainThread()
/builds/slave/try-l64-0000000000000000000000/build/obj-firefox/xpcom/ds/../../dist/include/nsThreadUtils.h:104
#1 0x7fa18ee71923 in nsPrefBranch
/builds/slave/try-l64-0000000000000000000000/build/modules/libpref/src/nsPrefBranch.cpp:82
#2 0x7fa18ee6126b in mozilla::Preferences::GetInstanceForService()
/builds/slave/try-l64-0000000000000000000000/build/modules/libpref/src/Preferences.cpp:237
#3 0x7fa18ee7d908 in PreferencesConstructor(nsISupports*, nsID
const&, void**)
/builds/slave/try-l64-0000000000000000000000/build/modules/libpref/src/nsPrefsFactory.cpp:13
#4 0x7fa1925fe5db in
nsComponentManagerImpl::CreateInstanceByContractID(char const*,
nsISupports*, nsID const&, void**)
/builds/slave/try-l64-0000000000000000000000/build/xpcom/components/nsComponentManager.cpp:10
93
#5 0x7fa1925f59e9 in
nsComponentManagerImpl::GetServiceByContractID(char const*, nsID const&,
void**)
/builds/slave/try-l64-0000000000000000000000/build/xpcom/components/nsComponentManager.cpp:1449
#6 0x7fa19254718c in CallGetService(char const*, nsID const&,
void**)
/builds/slave/try-l64-0000000000000000000000/build/obj-firefox/xpcom/build/nsComponentManagerUtils.cpp:60
#7 0x7fa192542d8b in
nsCOMPtr_base::assign_from_gs_contractid(nsGetServiceByContractID, nsID
const&)
/builds/slave/try-l64-0000000000000000000000/build/obj-firefox/xpcom/build/nsCOMPtr.cpp:92
#8 0x7fa19256a080 in nsCOMPtr
/builds/slave/try-l64-0000000000000000000000/build/obj-firefox/chrome/src/../../dist/include/nsCOMPtr.h:640
#9 0x7fa19256912b in nsChromeRegistry::GetSingleton()
/builds/slave/try-l64-0000000000000000000000/build/chrome/src/nsChromeRegistry.cpp:645
#10 0x7fa192560408 in nsChromeRegistryConstructor(nsISupports*,
nsID const&, void**)
/builds/slave/try-l64-0000000000000000000000/build/xpcom/build/nsXPComInit.cpp:252
#11 0x7fa1925fe5db in
nsComponentManagerImpl::CreateInstanceByContractID(char const*,
nsISupports*, nsID const&, void**)
/builds/slave/try-l64-0000000000000000000000/build/xpcom/components/nsComponentManager.cpp:1093
#12 0x7fa1925f59e9 in
nsComponentManagerImpl::GetServiceByContractID(char const*, nsID const&,
void**)
/builds/slave/try-l64-0000000000000000000000/build/xpcom/components/nsComponentManager.cpp:1449
#13 0x7fa19254718c in CallGetService(char const*, nsID const&,
void**)
/builds/slave/try-l64-0000000000000000000000/build/obj-firefox/xpcom/build/nsComponentManagerUtils.cpp:60
#14 0x7fa192542d8b in
nsCOMPtr_base::assign_from_gs_contractid(nsGetServiceByContractID, nsID
const&)
/builds/slave/try-l64-0000000000000000000000/build/obj-firefox/xpcom/build/nsCOMPtr.cpp:92
#15 0x7fa192558bfe in nsCOMPtr
/builds/slave/try-l64-0000000000000000000000/build/xpcom/build/../glue/nsCOMPtr.h:640
#16 0x7fa1925eca73 in ParseManifest(NSLocationType,
mozilla::FileLocation&, char*, bool)
/builds/slave/try-l64-0000000000000000000000/build/xpcom/components/ManifestParser.cpp:626
#17 0x7fa1925fa185 in
nsComponentManagerImpl::RegisterManifest(NSLocationType,
mozilla::FileLocation&, bool)
/builds/slave/try-l64-0000000000000000000000/build/xpcom/components/nsComponentManager.cpp:556
#18 0x7fa1925fa44f in
nsComponentManagerImpl::ManifestManifest(nsComponentManagerImpl::ManifestProcessingContext&,
int, char* const*)
/builds/slave/try-l64-0000000000000000000000/build/xpcom/components/nsComponentManager.cpp:569
#19 0x7fa1925eccbf in ParseManifest(NSLocationType,
mozilla::FileLocation&, char*, bool)
/builds/slave/try-l64-0000000000000000000000/build/xpcom/components/ManifestParser.cpp:645
#20 0x7fa1925fa185 in
nsComponentManagerImpl::RegisterManifest(NSLocationType,
mozilla::FileLocation&, bool)
/builds/slave/try-l64-0000000000000000000000/build/xpcom/components/nsComponentManager.cpp:556
#21 0x7fa1925f800e in
nsComponentManagerImpl::RereadChromeManifests(bool)
/builds/slave/try-l64-0000000000000000000000/build/xpcom/components/nsComponentManager.cpp:732
#22 0x7fa19255c4fb in NS_InitXPCOM2
/builds/slave/try-l64-0000000000000000000000/build/xpcom/build/nsXPComInit.cpp:467
#23 0x7fa18e6c9c42 in ScopedXPCOMStartup::Initialize()
/builds/slave/try-l64-0000000000000000000000/build/toolkit/xre/nsAppRunner.cpp:1189
#24 0x7fa18e6cabc5 in XRE_main
/builds/slave/try-l64-0000000000000000000000/build/toolkit/xre/nsAppRunner.cpp:4126
#25 0x43177e in do_main(int, char**, nsIFile*)
/builds/slave/try-l64-0000000000000000000000/build/browser/app/nsBrowserApp.cpp:272
#26 0x7fa19a0f976c in __libc_start_main
/build/buildd/eglibc-2.15/csu/libc-start.c:226
#27 0x4309ec in _start ??:0
0x602000033450 is located 0 bytes inside of 4-byte region
[0x602000033450,0x602000033454)
allocated by thread T0 here:
#0 0x422ae9 in memalign ??:0
#1 0x7fa19b2d1364 in allocate_and_init
/build/buildd/eglibc-2.15/elf/dl-tls.c:526
Shadow bytes around the buggy address:
0x0c047fffe630: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fffe640: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c047fffe650: fa fa 00 01 fa fa 00 06 fa fa fd fd fa fa 00 fa
0x0c047fffe660: fa fa fd fd fa fa 00 fa fa fa 00 02 fa fa 00 01
0x0c047fffe670: fa fa 05 fa fa fa 00 fa fa fa fd fd fa fa 00 fa
=>0x0c047fffe680: fa fa 00 04 fa fa 00 04 fa fa[04]fa fa fa 00 fa
0x0c047fffe690: fa fa 00 fa fa fa 00 00 fa fa 00 00 fa fa 00 00
0x0c047fffe6a0: fa fa 00 00 fa fa 00 00 fa fa 00 fa fa fa fd fa
0x0c047fffe6b0: fa fa fd fd fa fa fd fd fa fa 00 00 fa fa 00 00
0x0c047fffe6c0: fa fa fd fa fa fa fd fd fa fa fd fa fa fa fd fd
0x0c047fffe6d0: fa fa 00 00 fa fa fd fd fa fa fd fd fa fa fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
ASan internal: fe
==2677==ABORTING

[address-...@googlecode.com]

Comment #11 on issue 193 by konstant...@gmail.com: Abort "attempting to

call malloc_usable_size() for pointer which is not owned" with
posix_memalign and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

extremely weird crash. Are you saying that you don't see this crash with
3.3 (w/o the patch) and with trunk?

[address-...@googlecode.com]

Comment #12 on issue 193 by decoder...@googlemail.com: Abort "attempting to

call malloc_usable_size() for pointer which is not owned" with
posix_memalign and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

Yep, I tried with trunk yesterday and I did not see it. I also did not see
this with the last 3.3 branch I tried, but I will try again now with and
without the patch to exactly confirm it's just the patch.

[address-...@googlecode.com]

Comment #13 on issue 193 by decoder...@googlemail.com: Abort "attempting to

call malloc_usable_size() for pointer which is not owned" with
posix_memalign and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

I tried reproducing locally now and didn't succeed so far :( maybe there's
a libc difference to the build machines. I'm still pretty sure though that
it's the patch because the last build (without the patch) worked fine and
we didn't change anything else. Also the memalign in the trace is
suspicious.

would it help you to see the backported patch? Thanks for any help :)

[address-...@googlecode.com]

Comment #14 on issue 193 by konstant...@gmail.com: Abort "attempting to

call malloc_usable_size() for pointer which is not owned" with
posix_memalign and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

>> would it help you to see the backported patch?

sure

[address-...@googlecode.com]

Comment #15 on issue 193 by decoder...@googlemail.com: Abort "attempting to

call malloc_usable_size() for pointer which is not owned" with
posix_memalign and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

Okay :) We're tracking our Clang 3.3 update here:

https://bugzilla.mozilla.org/show_bug.cgi?id=870173

The backport patch is in the first attachment (it's a patch creating this
and another patch in our tree, you can probably just ignore the rest and
only look at the actual patch).

The second attachment is the manifest which contains the revision number
for the latest build used (including the patch).

Let me know if you need anything else :)

[address-...@googlecode.com]

Comment #16 on issue 193 by konstant...@gmail.com: Abort "attempting to

call malloc_usable_size() for pointer which is not owned" with
posix_memalign and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

The patch looks ok, it's hard to tell why it crashes when applied to 3.3
(while not crashing in trunk).

Could you remind us why do you have to use 3.3 for asan?
For Chrome we always use recent trunk and have no such problems.

[address-...@googlecode.com]

Comment #17 on issue 193 by jruder...@gmail.com: Abort "attempting to call

malloc_usable_size() for pointer which is not owned" with posix_memalign
and malloc_usable_size
http://code.google.com/p/address-sanitizer/issues/detail?id=193

I'm hitting the NS_IsMainThread problem (comment 10) with LLVM trunk. I
filed http://code.google.com/p/address-sanitizer/issues/detail?id=204.