Tombstone behavior in Lollipop

[Martin Siegumfeldt]

Hi,

I was wondering if the Lollipop behavior of the tombstones have changed somewhere around Jellybean/Lollipop? With reference to http://www.kandroid.org/online-pdk/guide/debugging_native.html (yes, I know it is quite outdated) the tombstone contains the value of the PC iterating the individual libraries for easy backtrace through the 'stack' script:

I/DEBUG: pid: 373, tid: 401  >>> android.content.providers.pim <<<
I/DEBUG: signal 11 (SIGSEGV), fault addr 00000000
I/DEBUG:  r0 ffffffff  r1 00000000  r2 00000454  r3 002136d4
I/DEBUG:  r4 002136c0  r5 40804810  r6 0022dc70  r7 00000010
I/DEBUG:  r8 0020a258  r9 00000014  10 6b039074  fp 109ffcf8
I/DEBUG:  ip 6b039e90  sp 109ffc0c  lr 580239f0  pc 6b0156a0
I/DEBUG:          #01  pc 6b0156a0  /android/lib/libjamvm.so
I/DEBUG:          #01  lr 580239f0  /android/lib/libandroid_runtime.so
I/DEBUG:          #02  pc 6b01481c  /android/lib/libjamvm.so
I/DEBUG:          #03  pc 6b0148a4  /android/lib/libjamvm.so
I/DEBUG:          #04  pc 6b00ebc0  /android/lib/libjamvm.so
I/DEBUG:          #05  pc 6b02166c  /android/lib/libjamvm.so
I/DEBUG:          #06  pc 6b01657c  /android/lib/libjamvm.so
I/DEBUG:          #07  pc 6b01481c  /android/lib/libjamvm.so
I/DEBUG:          #08  pc 6b0148a4  /android/lib/libjamvm.so

I am currently debugging a segfaulting Lollipop based system but do not see the convenient trace from above:

pid: 10195, tid: 10195, name: mediaserver  >>> /system/bin/mediaserver <<<

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0

    eax 00000000  ebx 74582cbc  ecx 74582cbc  edx 44274c7c

    esi 745840d8  edi 00000000

    xcs 00000073  xds 0000007b  xes 0000007b  xfs 00000000  xss 0000007b

    eip 00000000  ebp 745840e0  esp 7ff22acc  flags 00010283

backtrace:

    #00 pc 00000000  <unknown>

    #01 pc 0006ffff  <unknown>

stack:

         7ff22a8c  44274c7c

         7ff22a90  758031f0  [anon:libc_malloc]

         7ff22a94  00000000

         7ff22a98  7745d679  /system/lib/libcutils.so (__android_log_vprint+9)

         7ff22a9c  74582cbc  /system/lib/mylib.so

         7ff22aa0  74584080  /system/lib/mylib.so

         7ff22aa4  00000000

         7ff22aa8  745840e0  /system/lib/mylib.so

         7ff22aac  7452f6a6  /system/lib/mylib.so (lib_ilog_print+70)

         7ff22ab0  00000004

         7ff22ab4  74554b18  /system/lib/mylib.so

         7ff22ab8  745548b4  /system/lib/mylib.so

         7ff22abc  7ff22ad4  [stack]

         7ff22ac0  00000000

         7ff22ac4  7452f666  /system/lib/mylib.so (lib_ilog_print+6)

         7ff22ac8  74508b15  /system/lib/mylib.so (lib_silent_poweron_init+5)

    #00  7ff22acc  74501735  /system/lib/mylib.so (lib_modules_init+53)

         7ff22ad0  00000000

         7ff22ad4  7ff22aef  [stack]

         7ff22ad8  00000002

         7ff22adc  758031e0  [anon:libc_malloc]

         7ff22ae0  00000000

         7ff22ae4  00000000

         7ff22ae8  74501709  /system/lib/mylib.so (lib_modules_init+9)

         7ff22aec  74582cbc  /system/lib/mylib.so

         7ff22af0  00000040

         7ff22af4  00000000

         7ff22af8  7458ba10

         7ff22afc  744f6b15  /system/lib/mylib.so (lib_set_driver_mode+949)

         7ff22b00  00000000

         7ff22b04  7ff22b3c  [stack]

         7ff22b08  00000040

         ........  ........

    #01  745840e8  00000000

         745840ec  00000000

         745840f0  00000000

         745840f4  00000000

         745840f8  00000000

         745840fc  00000000

         74584100  ffffffff

         74584104  ffffffff

         74584108  0000ffff

         7458410c  00000000

         74584110  00000000

         74584114  00000000

         74584118  00000000

         7458411c  00000000

         74584120  000b0000

         74584124  00000b00

As you can see the PC value is not present for the library iteration and I thus have no means for doing the source code line-mapping. The output from the 'stack' tool is as a consequence not very helpful

Stack Trace:

  RELADDR   FUNCTION  FILE:LINE

  00000000    <unknown>

  0006ffff    <unknown>

Stack Data:

  ADDR      VALUE     FUNCTION                   FILE:LINE

  7ff22a8c  44274c7c

  7ff22a90  758031f0  <unknown>                  [anon:libc_malloc]

  7ff22a94  00000000

  7ff22a98  7745d679  __android_log_vprint+9     /system/lib/libcutils.so

  7ff22a9c  74582cbc  <unknown>                  /system/lib/mylib.so

  7ff22aa0  74584080  <unknown>                  /system/lib/mylib.so

  7ff22aa4  00000000

  7ff22aa8  745840e0  <unknown>                  /system/lib/mylib.so

  7ff22aac  7452f6a6  lib_ilog_print+70          /system/lib/mylib.so

  7ff22ab0  00000004

  7ff22ab4  74554b18  <unknown>                  /system/lib/mylib.so

  7ff22ab8  745548b4  <unknown>                  /system/lib/mylib.so

  7ff22abc  7ff22ad4                             [stack]

  7ff22ac0  00000000

  7ff22ac4  7452f666  lib_ilog_print+6           /system/lib/mylib.so

  7ff22ac8  74508b15  lib_silent_poweron_init+5  /system/lib/mylib.so

  7ff22acc  74501735  lib_modules_init+53        /system/lib/mylib.so

  7ff22ad0  00000000

  7ff22ad4  7ff22aef                             [stack]

  7ff22ad8  00000002

  7ff22adc  758031e0  <unknown>                  [anon:libc_malloc]

  7ff22ae0  00000000

  7ff22ae4  00000000

  7ff22ae8  74501709  lib_modules_init+9         /system/lib/mylib.so

  7ff22aec  74582cbc  <unknown>                  /system/lib/mylib.so

  7ff22af0  00000040

  7ff22af4  00000000

  7ff22af8  7458ba10

  7ff22afc  744f6b15  lib_set_driver_mode+949    /system/lib/mylib.so

  7ff22b00  00000000

  7ff22b04  7ff22b3c                             [stack]

  7ff22b08  00000040

There are additional threads running of the particular process, and it is not clear to me which one causing the segfault. 

Has the tombstone generating changed in 'recent' Android versions or is something broken in my toolchain?

Thanks,

Martin

[Zoltan Kuscsik]

Hi Martin,

your binary must have an unwind table compiled in, otherwise you don't necessary

get a backtrace. Platform binaries normally have that, but if construct your own 

toolchain calls, you must have unwind-tables and debug symbols enabled.

If you still have a problem, you can try to get GDB running or use libunwind on Android,

but I think it is the lack of unwind tables that causes the issue.

Br,

Zoltan

[Martin Siegumfeldt]

Thanks Zoltan,

It turned out to be a function pointer being NULL, which apparently seems to mess up the call stack. In case of a data pointer being NULL we do see the PC values of the call stack.

I hope you are well :)

Br,

Martin