Incomplete chrome backtrace in GDB

[Wu-Cheng Li]

I used GDB by remote debugging on Samsung ARM Chromebook (daisy). I attached to renderer process. When I ran "thread apply all bt", many backtraces seemed incomplete (see below). The error message was "Cannot access memory at address 0x0". But breakpoints worked and the backtrace of the currently running thread looked correct. The stacktraces of other threads were usually incomplete. It seemed the backtrace would not be shown if the thread was waiting. Any idea?

This issue made debugging deadlock difficult. I tried adding -funwind-tables to cflags and '-rdynamic' to ldflags, which didn't work. Building release or debug chrome had the same problem.

Regards,

Wu-Cheng

(gdb) thread apply all bt

[New Thread 28161]

[New Thread 28162]

[New Thread 28163]

[New Thread 28164]

[New Thread 28165]

[New Thread 28169]

[New Thread 28173]

[New Thread 28179]

[New Thread 28180]

[New Thread 28181]

[New Thread 28182]

[New Thread 28183]

Thread 19 (Thread 28183):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  ?? ()

Thread 18 (Thread 28182):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  do_futex_wait (isem=...) at ../nptl/sysdeps/unix/sysv/linux/sem_wait.c:51

#2  ?? ()

Cannot access memory at address 0x0

#3  ?? ()

Cannot access memory at address 0x0

Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 17 (Thread 28181):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  do_futex_wait (isem=...) at ../nptl/sysdeps/unix/sysv/linux/sem_wait.c:51

#2  ?? ()

Cannot access memory at address 0x0

#3  ?? ()

Cannot access memory at address 0x0

Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 16 (Thread 28180):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  do_futex_wait (isem=...) at ../nptl/sysdeps/unix/sysv/linux/sem_wait.c:51

#2  ?? ()

Cannot access memory at address 0x0

#3  ?? ()

Cannot access memory at address 0x0

Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 15 (Thread 28179):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  do_futex_wait (isem=...) at ../nptl/sysdeps/unix/sysv/linux/sem_wait.c:51

#2  ?? ()

Cannot access memory at address 0x0

#3  ?? ()

Cannot access memory at address 0x0

Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 14 (Thread 28173):

#0  read () from /build/daisy/lib/libpthread.so.0

#1  file_util::ReadFromFD (fd=..., buffer=..., bytes=...) at ../../base/file_util_posix.cc:362

#2  base::SyncSocket::Receive (this=..., buffer=..., length=...) at ../../base/sync_socket_posix.cc:106

#3  media::AudioDeviceThread::Thread::Run (this=...) at ../../media/audio/audio_device_thread.cc:164

#4  media::AudioDeviceThread::Thread::ThreadMain (this=...) at ../../media/audio/audio_device_thread.cc:154

#5  base::(anonymous namespace)::ThreadFunc (params=...) at ../../base/threading/platform_thread_posix.cc:100

#6  start_thread (arg=...) at pthread_create.c:307

#7  _ioperm (from=..., num=..., turn_on=...) at ../ports/sysdeps/unix/sysv/linux/arm/ioperm.c:189

#8  ?? ()

Thread 13 (Thread 28169):

#0  read () from /build/daisy/lib/libpthread.so.0

#1  file_util::ReadFromFD (fd=..., buffer=..., bytes=...) at ../../base/file_util_posix.cc:362

#2  base::SyncSocket::Receive (this=..., buffer=..., length=...) at ../../base/sync_socket_posix.cc:106

#3  media::AudioDeviceThread::Thread::Run (this=...) at ../../media/audio/audio_device_thread.cc:164

#4  media::AudioDeviceThread::Thread::ThreadMain (this=...) at ../../media/audio/audio_device_thread.cc:154

#5  base::(anonymous namespace)::ThreadFunc (params=...) at ../../base/threading/platform_thread_posix.cc:100

#6  start_thread (arg=...) at pthread_create.c:307

#7  _ioperm (from=..., num=..., turn_on=...) at ../ports/sysdeps/unix/sysv/linux/arm/ioperm.c:189

#8  ?? ()

Thread 12 (Thread 28165):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  ?? ()

---Type <return> to continue, or q <return> to quit---

Thread 11 (Thread 28164):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  ?? ()

Thread 10 (Thread 28163):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  ?? ()

Thread 9 (Thread 28162):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  __pthread_cond_wait (cond=..., mutex=...) at pthread_cond_wait.c:156

#2  ?? ()

Cannot access memory at address 0x0

#3  ?? ()

Cannot access memory at address 0x0

Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 8 (Thread 28161):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  __pthread_cond_wait (cond=..., mutex=...) at pthread_cond_wait.c:156

#2  ?? ()

Cannot access memory at address 0x0

#3  ?? ()

Cannot access memory at address 0x0

Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 7 (Thread 27939):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  __pthread_cond_wait (cond=..., mutex=...) at pthread_cond_wait.c:156

#2  ?? ()

Cannot access memory at address 0x1

#3  ?? ()

Cannot access memory at address 0x1

Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 6 (Thread 27936):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  __pthread_cond_wait (cond=..., mutex=...) at pthread_cond_wait.c:156

#2  ?? ()

Cannot access memory at address 0x0

#3  ?? ()

Cannot access memory at address 0x0

Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 5 (Thread 27935):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  do_futex_wait (isem=...) at ../nptl/sysdeps/unix/sysv/linux/sem_wait.c:51

#2  ?? ()

Cannot access memory at address 0x0

#3  ?? ()

Cannot access memory at address 0x0

Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 4 (Thread 27934):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  do_futex_wait (isem=...) at ../nptl/sysdeps/unix/sysv/linux/sem_wait.c:51

#2  ?? ()

Cannot access memory at address 0x0

#3  ?? ()

Cannot access memory at address 0x0

Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 3 (Thread 27932):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  __pthread_cond_wait (cond=..., mutex=...) at pthread_cond_wait.c:156

#2  ?? ()

Cannot access memory at address 0x0

#3  ?? ()

Cannot access memory at address 0x0

Backtrace stopped: previous frame identical to this frame (corrupt stack?)

---Type <return> to continue, or q <return> to quit---

Thread 2 (Thread 27931):

#0  __syslog_chk (pri=..., flag=..., fmt=...) at ../misc/syslog.c:133

#1  ?? ()

Cannot access memory at address 0x0

#2  ?? ()

Cannot access memory at address 0x0

Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 1 (Thread 27930):

#0  __pthread_mutex_cond_lock (mutex=...) at ../nptl/pthread_mutex_lock.c:47

#1  __pthread_cond_wait (cond=..., mutex=...) at pthread_cond_wait.c:156

#2  ?? ()

Cannot access memory at address 0x0

#3  ?? ()

Cannot access memory at address 0x0

Backtrace stopped: previous frame identical to this frame (corrupt stack?)

[Simon Que]

It's probably because some parts of Chrome are built without frame pointer, so backtraces are hard to obtain.  I ran into this problem while attempting to get similar data.

I have not tried to do backtraces with Chrome in gdb, so I don't know if what you're seeing is new.

--
--
Chromium OS Developers mailing list: chromiu...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-os-dev?hl=en
 
 
 

[Pawel Osciak]

+Doug, who ran into this as well a few months ago iirc.

[Doug Anderson]

I remember being unable to get things setup until I used the
"gdb_remote" script in the chroot. ...and then I think things all
magically started working without any other tricks.

Can you try that and see if it works?

-Doug

[Wu-Cheng Li (李務誠)]

I found it was a stupid mistake that my chrome OS image did not match with chroot. After flashing the image built in chroot, backtrace in gdb worked.

gdb_remote is very handy! I needed to change a few lines because my chrome was built outside of chroot.

Not sure about frame pointer. If you mean chrome doesn't generate stack trace on ARM chromebook (base::debug::StackTrace not working), adding -funwind-tables and -rdynamic can fix that. Backtrace in gdb doesn't need those flags.