Issue 130987 in chromium: HEAP_PROFILE_MMAP causes Segmentation fault

[chro...@googlecode.com]

Status: Untriaged
Owner: ----
CC: j...@chromium.org
Labels: Type-Bug Pri-2 Area-Internals

New issue 130987 by dmikur...@chromium.org: HEAP_PROFILE_MMAP causes
Segmentation fault
http://code.google.com/p/chromium/issues/detail?id=130987

Version: r137672 or later
OS: Linux


What steps will reproduce the problem?
1. Build a Chromium Debug build.
2. Run chrome with --no-sandbox and the following envs:
--- HEAPPROFILE=/path/to/tmpdir/some-prefix-for-dumpfiles
--- HEAP_PROFILE_MMAP=true


What is the expected output? What do you see instead?
It frequently fails with Segmentation fault just after starting.


jam@, what do you think about it? This failure looks started from r137672
(from my git bisect and many rebuilds and retries of r137671 and r137672).

It looks strange that it is failing in TCMalloc's spinlock, but your change
r137672 (http://crrev.com/137672) looks not related to TCMalloc...

(Actually, r137671 fails, too. But it's a different failure and I know the
reason.)

Stacktraces:
#0 0x00007fb778566da0 in SpinLock::Lock (this=0x0) at
third_party/tcmalloc/chromium/src/base/spinlock.h:72
#1 0x00007fb778577707 in (anonymous namespace)::ArenaLock::ArenaLock
(this=0x7fb76be4e130, arena=0x7fb7772c0020)
at third_party/tcmalloc/chromium/src/base/low_level_alloc.cc:233
#2 0x00007fb778578042 in DoAllocWithArena (request=320,
arena=0x7fb7772c0020)
at third_party/tcmalloc/chromium/src/base/low_level_alloc.cc:438
#3 0x00007fb77cdc852d in LowLevelAlloc::AllocWithArena (request=320,
arena=0x7fb7772c0020)
at third_party/tcmalloc/chromium/src/base/low_level_alloc.cc:511
#4 0x00007fb77856d105 in MemoryRegionMap::MyAllocator::Allocate (n=320)
at third_party/tcmalloc/chromium/src/memory_region_map.h:229
#5 0x00007fb77856e3ff in
STL_Allocator<std::_Rb_tree_node<MemoryRegionMap::Region>,
MemoryRegionMap::MyAllocator>::allocate (
this=0x7fb780e28780, n=1) at
third_party/tcmalloc/chromium/src/base/stl_allocator.h:83
#6 0x00007fb77856e326 in std::_Rb_tree<MemoryRegionMap::Region,
MemoryRegionMap::Region, std::_Identity<MemoryRegionMap::Region>,
MemoryRegionMap::RegionCmp, STL_Allocator<MemoryRegionMap::Region,
MemoryRegionMap::MyAllocator> >::_M_get_node (
this=0x7fb780e28780) at /usr/include/c++/4.4/bits/stl_tree.h:359
#7 0x00007fb77856e207 in std::_Rb_tree<MemoryRegionMap::Region,
MemoryRegionMap::Region, std::_Identity<MemoryRegionMap::Region>,
MemoryRegionMap::RegionCmp, STL_Allocator<MemoryRegionMap::Region,
MemoryRegionMap::MyAllocator> >::_M_create_node (
this=0x7fb780e28780, __x=...) at
/usr/include/c++/4.4/bits/stl_tree.h:369
#8 0x00007fb77856e007 in std::_Rb_tree<MemoryRegionMap::Region,
MemoryRegionMap::Region, std::_Identity<MemoryRegionMap::Region>,
MemoryRegionMap::RegionCmp, STL_Allocator<MemoryRegionMap::Region,
MemoryRegionMap::MyAllocator> >::_M_insert_ (
this=0x7fb780e28780, __x=0x0, __p=0x7fb7772b0b20, __v=...) at
/usr/include/c++/4.4/bits/stl_tree.h:881
#9 0x00007fb77856dbb6 in std::_Rb_tree<MemoryRegionMap::Region,
MemoryRegionMap::Region, std::_Identity<MemoryRegionMap::Region>,
MemoryRegionMap::RegionCmp, STL_Allocator<MemoryRegionMap::Region,
MemoryRegionMap::MyAllocator> >::_M_insert_unique (
this=0x7fb780e28780, __v=...) at
/usr/include/c++/4.4/bits/stl_tree.h:1177
#10 0x00007fb77856d85b in std::set<MemoryRegionMap::Region,
MemoryRegionMap::RegionCmp, STL_Allocator<MemoryRegionMap::Region,
MemoryRegionMap::MyAllocator> >::insert (this=0x7fb780e28780, __x=...) at
/usr/include/c++/4.4/bits/stl_set.h:411
#11 0x00007fb77856d41b in MemoryRegionMap::DoInsertRegionLocked (region=...)
at third_party/tcmalloc/chromium/src/memory_region_map.cc:375
#12 0x00007fb77856d6c0 in MemoryRegionMap::InsertRegionLocked (region=...)
at third_party/tcmalloc/chromium/src/memory_region_map.cc:436
#13 0x00007fb77856c3d5 in MemoryRegionMap::RecordRegionAddition
(start=0x7fb768e9a000, size=1048576)
at third_party/tcmalloc/chromium/src/memory_region_map.cc:468
#14 0x00007fb77856cad8 in MemoryRegionMap::MmapHook (result=0x7fb768e9a000,
start=0x0, size=1048576, prot=3, flags=34, fd=-1,
offset=0) at third_party/tcmalloc/chromium/src/memory_region_map.cc:595
#15 0x00007fb778568887 in MallocHook::InvokeMmapHookSlow
(result=0x7fb768e9a000, start=0x0, size=1048576, protection=3, flags=34,
fd=-1, offset=0) at third_party/tcmalloc/chromium/src/malloc_hook.cc:549
#16 0x00007fb778569147 in MallocHook::InvokeMmapHook
(result=0x7fb768e9a000, start=0x0, size=1048576, protection=3, flags=34,
fd=-1, offset=0) at
third_party/tcmalloc/chromium/src/malloc_hook-inl.h:219
#17 0x00007fb77cdc813b in mmap64 (start=0x0, length=1048576, prot=3,
flags=34, fd=-1, offset=0)
at third_party/tcmalloc/chromium/src/malloc_hook_mmap_linux.h:164
#18 0x00007fb7785914d2 in MmapSysAllocator::Alloc (this=0x7fb780e448f0,
size=1048576, actual_size=0x7fb76be4ebb0, alignment=4096)
at third_party/tcmalloc/chromium/src/system-alloc.cc:310
#19 0x00007fb7785918a4 in DefaultSysAllocator::Alloc (this=0x7fb780e44900,
size=1048576, actual_size=0x7fb76be4ebb0,
alignment=4096) at third_party/tcmalloc/chromium/src/system-alloc.cc:428
#20 0x00007fb778591a9d in TCMalloc_SystemAlloc (size=1048576,
actual_size=0x7fb76be4ebb0, alignment=4096)
at third_party/tcmalloc/chromium/src/system-alloc.cc:481
#21 0x00007fb77858aada in tcmalloc::PageHeap::GrowHeap
(this=0x7fb777408000, n=2)
at third_party/tcmalloc/chromium/src/page_heap.cc:471
#22 0x00007fb77858807a in tcmalloc::PageHeap::New (this=0x7fb777408000,
n=2) at third_party/tcmalloc/chromium/src/page_heap.cc:102
#23 0x00007fb77857b80c in tcmalloc::CentralFreeList::Populate
(this=0x7fb780e3b960)
at third_party/tcmalloc/chromium/src/central_freelist.cc:315
#24 0x00007fb77857b635 in tcmalloc::CentralFreeList::FetchFromSpansSafe
(this=0x7fb780e3b960)
at third_party/tcmalloc/chromium/src/central_freelist.cc:283
#25 0x00007fb77857b563 in tcmalloc::CentralFreeList::RemoveRange
(this=0x7fb780e3b960, start=0x7fb76be4f128, end=0x7fb76be4f120,
N=1) at third_party/tcmalloc/chromium/src/central_freelist.cc:262
#26 0x00007fb7785757fe in tcmalloc::ThreadCache::FetchFromCentralCache
(this=0x7fb7773e2500, cl=23, byte_size=576)
at third_party/tcmalloc/chromium/src/thread_cache.cc:165
#27 0x00007fb778573010 in tcmalloc::ThreadCache::Allocate
(this=0x7fb7773e2500, size=576, cl=23)
at third_party/tcmalloc/chromium/src/thread_cache.h:368
#28 0x00007fb778570fb1 in (anonymous namespace)::do_malloc (size=576) at
third_party/tcmalloc/chromium/src/tcmalloc.cc:1099
#29 0x00007fb778571f9f in (anonymous namespace)::cpp_alloc (size=512,
nothrow=false)
at third_party/tcmalloc/chromium/src/tcmalloc.cc:1394
#30 0x00007fb77cdc86bc in tc_new (size=512) at
third_party/tcmalloc/chromium/src/tcmalloc.cc:1577
#31 0x00007fb7794844fb in __gnu_cxx::new_allocator<tracked_objects::Births
const*>::allocate (this=0x7fb76be4f600, __n=64)
at /usr/include/c++/4.4/ext/new_allocator.h:89
#32 0x00007fb779483def in std::_Deque_base<tracked_objects::Births const*,
std::allocator<tracked_objects::Births const*> >::_M_allocate_node
(this=0x7fb76be4f600) at /usr/include/c++/4.4/bits/stl_deque.h:444
#33 0x00007fb779482bdc in std::_Deque_base<tracked_objects::Births const*,
std::allocator<tracked_objects::Births const*> >::_M_create_nodes
(this=0x7fb76be4f600, __nstart=0x7fb7773deb08, __nfinish=0x7fb7773deb10) at
/usr/include/c++/4.4/bits/stl_deque.h:538
#34 0x00007fb7794811af in std::_Deque_base<tracked_objects::Births const*,
std::allocator<tracked_objects::Births const*> >::_M_initialize_map
(this=0x7fb76be4f600, __num_elements=0) at
/usr/include/c++/4.4/bits/stl_deque.h:512
#35 0x00007fb77947fbf7 in std::_Deque_base<tracked_objects::Births const*,
std::allocator<tracked_objects::Births const*> >::_Deque_base
(this=0x7fb76be4f600) at /usr/include/c++/4.4/bits/stl_deque.h:375
#36 0x00007fb77947f056 in std::deque<tracked_objects::Births const*,
std::allocator<tracked_objects::Births const*> >::deque (
this=0x7fb76be4f600) at /usr/include/c++/4.4/bits/stl_deque.h:691
#37 0x00007fb77947c8c9 in tracked_objects::ThreadData::ThreadData
(this=0x7fb777366780, suggested_name="CrShutdownDetector")
at base/tracked_objects.cc:236
#38 0x00007fb77947d04f in
tracked_objects::ThreadData::InitializeThreadContext
(suggested_name="CrShutdownDetector")
at base/tracked_objects.cc:284
#39 0x00007fb77946b9db in base::PlatformThread::SetName
(name=0x7fb77ce575d4 "CrShutdownDetector")
at base/threading/platform_thread_posix.cc:203
#40 0x00007fb778b48e3d in (anonymous
namespace)::ShutdownDetector::ThreadMain (this=0x7fb7773be6c0)
at chrome/browser/chrome_browser_main_posix.cc:123
#41 0x00007fb77946b5f7 in base::(anonymous namespace)::ThreadFunc
(params=0x7fb7773be520)
at base/threading/platform_thread_posix.cc:65
#42 0x00007fb772a779ca in start_thread (arg=<optimized out>) at
pthread_create.c:300
#43 0x00007fb76fe77cdd in clone ()
at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#44 0x0000000000000000 in ?? ()

[chro...@googlecode.com]

Issue 130987: HEAP_PROFILE_MMAP causes Segmentation fault
http://code.google.com/p/chromium/issues/detail?id=130987

This issue is no longer blocking issue 123750.
See http://code.google.com/p/chromium/issues/detail?id=123750
--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings

[chro...@googlecode.com]

Comment #1 on issue 130987 by dmikur...@chromium.org: HEAP_PROFILE_MMAP

causes Segmentation fault
http://code.google.com/p/chromium/issues/detail?id=130987

fyi,

But, the latest Chromium + reverting r137672 doesn't fix this issue. Are
there related changes around this remove?

[chro...@googlecode.com]

Updates:
Cc: joc...@chromium.org bau...@chromium.org jhawk...@chromium.org

Comment #2 on issue 130987 by dmikur...@chromium.org: HEAP_PROFILE_MMAP

causes Segmentation fault
http://code.google.com/p/chromium/issues/detail?id=130987

+bauerb, jochen, jhawkins

I found how to remove this Segmentation fault. Could you try reproducing
it in your environment, the authors and reviewers for the changes?


Reverting these two changes :

1. Remove all the unused automation IPCs.
http://crrev.com/137672
2. Reland 138502 - Move version metadata from PluginGroup into
PluginInstaller.
http://crrev.com/138779


HEAP_PROFILE_MMAP is, actually, a necessary feature for our memory
profiling tool. We strongly want the profiling tool during the next week.

In fact, I wonder if we could revert both these changes (temporarily for
the next week). But, 138779 looks like a kind of required change, maybe?

As the second option, I wonder if we could
- revert 137672, and
- try finding the main cause in 138779, and fix it.

What do you think, jam and bauerb?

We'd be happy if you could co-operate on this issue. The problem is that
we don't understand why these changes cause such Segmentation fault yet...

[chro...@googlecode.com]

Comment #3 on issue 130987 by joc...@chromium.org: HEAP_PROFILE_MMAP causes
Segmentation fault
http://code.google.com/p/chromium/issues/detail?id=130987

I doubt that this crash is related to any of the changes you listed

Maybe it's a race or something? The last two frames of the stack trace
don't make much sense (in frame #1, arena is 0x7fb7772c0020) and in frame
#0, it's suddenly NULL

you could try running this with tsan or asan.

If we rely on these features to work, maybe there should be a test for it?

[chro...@googlecode.com]

Comment #4 on issue 130987 by j...@chromium.org: HEAP_PROFILE_MMAP causes
Segmentation fault
http://code.google.com/p/chromium/issues/detail?id=130987

r137672 deleted unused IPC classes. If that's causing your tool to fail,
then something is wrong with the tool and that should be debugged instead.

[chro...@googlecode.com]

Comment #5 on issue 130987 by dmikur...@chromium.org: HEAP_PROFILE_MMAP

causes Segmentation fault
http://code.google.com/p/chromium/issues/detail?id=130987

Yes, race or memory corruption is the most suspicious. Ok, I'll try asan
and tsan for them.
I started to write a test for it.


The failing feature is not my tool, it's a long-time existing feature in
tcmalloc.

[chro...@googlecode.com]

Comment #6 on issue 130987 by dmikur...@chromium.org: HEAP_PROFILE_MMAP

causes Segmentation fault
http://code.google.com/p/chromium/issues/detail?id=130987

But, before discussing, please try reproducing it...

[chro...@googlecode.com]

Comment #7 on issue 130987 by dmikur...@chromium.org: HEAP_PROFILE_MMAP

causes Segmentation fault
http://code.google.com/p/chromium/issues/detail?id=130987

Ok, I found that it may be a bug in the latest TCMalloc. When I revert
TCMalloc to the old version, the failure stopped.

(TCMalloc was updated in March, but it didn't come to a head so far...)

[chro...@googlecode.com]

Comment #8 on issue 130987 by dmikur...@chromium.org: HEAP_PROFILE_MMAP

causes Segmentation fault
http://code.google.com/p/chromium/issues/detail?id=130987

Ah, no, again. Sorry, and thank you for your trouble. It's not on the new
TCMalloc itself, but it looks like Chromium's original tuning in TCMalloc.
(It's mysterious that it wasn't failing until now.)

I'll try fixing it.

[chro...@googlecode.com]

Updates:
Status: started
Owner: dmikur...@chromium.org
Cc: j...@chromium.org

Comment #9 on issue 130987 by j...@chromium.org: HEAP_PROFILE_MMAP causes
Segmentation fault
http://code.google.com/p/chromium/issues/detail?id=130987

dmikurube: Looking at the proposed CL... I can believe it is better (closer
to what we had before in terms of tuning, prior to your upgrade)... but I'm
not at all clear as to why the current code induced a crash.

Given that you changed the number of size-classes (in the proposed CL),
perhaps the count was just plain wrong, and we didn't have the required set
of sizes to cover the range of possible allocation requests. It would be
good to dump the TCMalloc sizes-for-class, as well as the number of
pages-allocated-for-each-class, and make sure it covered the range 0-32K
properly.

[chro...@googlecode.com]

Comment #11 on issue 130987 by kaiw...@chromium.org: HEAP_PROFILE_MMAP

causes Segmentation fault
http://code.google.com/p/chromium/issues/detail?id=130987

Hi Dai, I synced my code to 137672 but could not reproduce the crash.
Are there more detailed steps to reproduce it?

[chro...@googlecode.com]

Updates:
Status: WontFix

Comment #18 on issue 130987 by brat...@opera.com: HEAP_PROFILE_MMAP causes
Segmentation fault
https://code.google.com/p/chromium/issues/detail?id=130987

This is part of Deep Memory Profiler which is discontinued and deleted from
the source. The new tool that will help us here is memory-infra.

--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings