debugging DVM mods

[Barry Hayes]

I'm trying to add some code to the garbage collector to measure some
lifetime stats for possible generational GC for apps where it's
appropriate.

What are the best ways to test and debug vm changes?

I made my changes and they compile successfully.
I modified the Android.mk file in .../dalvik to have the debug flags
they way I expect they should be.

I'm now trying to find some way to run a test load on my debugging
version of the DVM; say the Browser Functional Test Runner from the
SDK. And I'd like to debug it with gdb, and have all the niceties like
my symbols ..

I realize this is a pretty dark corner, but is this process documented
anywhere? I mean, I've looked the build files and the output from make
-n and have some guesses, but docs trump guesswork ..

[By the way, I found out off-list that the "TARGET_BUILD_TYPE=debug"
pretty much shouldn't be used. If lots of the build system is
vestigial, is someone going to houseclean some day and make sure
people don't get suckered into trying "TARGET_BUILD_TYPE=debug"? Or
fix it or document it ...]

[Mattaku Betsujin]

You can get gdbserver from here

http://groups.google.com/group/android-framework/browse_thread/thread/7070c72171f087bb

Then, use this command to forward a server port on the emulator to your PC

    adb forward tcp:127.0.0.1:12345 127.0.0.1:12345

Then, you can run gdbserver inside the emulator, attach to the process that you want to debug, and listen on port 12345.

On the PC, start prebuilt/linux-x86/toolchain/arm-eabi-4.2.1/bin/arm-eabi-gdb, then

   target remote localhost:12345

[Barry Hayes]

Thanks!

To be more clear ..
I am running on linux, and would be happy enough, fro now, to just
debug in the emulator. I'm not looking to actually debug on an actual
Android phone.

[Dan Bornstein]

On Mon, Mar 9, 2009 at 11:12 AM, Barry Hayes <bha...@gmail.com> wrote:
> I am running on linux, and would be happy enough, fro now, to just
> debug in the emulator. I'm not looking to actually debug on an actual
> Android phone.

Mattaku's instructions were with respect to the emulator, though you
can debug on-device in exactly the same way.
Adb transparently handles the difference in connection setup (that is,
it knows how to find devices both on local sockets and over USB).

-dan

[Barry Hayes]

Mattaku's instructions are indeed a huge help.

I can get the gdb instance hooked to the app in the emulator.
However, I can't figure out how to get symbolic information into gdb
to let me set breakpoints and so on.

[Patrick Heeb]

the build process builds unstripped libs and binaries under
out/target/product/generic/symbols/system/lib
use the following commands in gdb

set sysroot out/target/product/generic/symbols/
set solib-search-path out/target/product/generic/symbols/system/lib

[David Turner]

There is a "gdbclient" command defined in the envsetup.mk script that does all the nasty setup for you. 

[Barry Hayes]

Thanks to everyone who has helped me on this question. Here is the
answer *I* needed as best I have it. I am running on a linux system,
and as always YMMV. Someone mentioned a wiki for things like this; I
don't know where it is or if it is.

To make and test mods to the DVM:

Edit the Android.mk file to include the flags you need;
TARGET_BUILD_TYPE=debug isn't really supported; you should continue to
use TARGET_BUILD_TYPE=release.
Edit your files and get a clean build using "make" at the top. [That
seems to take a long time. I don't know if there is poor dependency
management on the changes I am making, or if this is always going to
be a long operation.]

In shell 1 run:
$ ANDROID_PRODUCT_OUT=/home/bhayes/android/out/target/product/generic
out/host/linux-x86/bin/emulator
[ANDROID_PRODUCT_OUT does not work if it is a relative path:
"out/target/product/generic"]
The emulator window comes up.

The emulator now needs to be sent some commands. In shell 2:
$ out/host/linux-x86/bin/adb forward tcp:12345 tcp:12345
[This forwards data sent via TCP to port 12345 by any program running
on the emulator to the emulator's hosting machine at port 12345.]
$ out/host/linux-x86/bin/adb shell
[This logs me in to the emulator directly. The prompt there is "#".]
# ps
[...]
app_0 125 22 90164 12768 ffffffff afe0d4b4 S com.android.alarmclock
[...]
# gdbserver 10.0.2.2:12345 --attach 125
[This runs the program gdbserver. It's a server for remote gdb
debugging. It will communicate on port 10.0.2.2:12345. I have no idea
what 10.0.2.2 designates in the emulator. 12345 is the port I named
before, and I'm telling it to attach to running process 125, as seen
in the ps. There is an alternative comman-line form for gdbserver
which will fire up a new job to debug rather than attach to an
existing job.]
[Several sources suggested that after the "adb forward" command, I
should also issue:
$ out/host/linux-x86/bin/adb push
out/target/product/generic/system/bin/gdbserver /data/gdbserver
to copy the newly-built version of gdbserver to the emulator. In that
case, the command immediately above would become:
# ./data/gdbserver 10.0.2.2:12345 --attach 125
I don't understand when I would need the "push" version unless I was
modifying the gdbserver code itself and wanted to test those changes.]

In shell 3:
[The script build/envsetup.sh has a definition of a gdbclient command.
That command does not seem to work correctly for me, but if it works
for you is a very useful shortcut around many of the steps below.]
$ ./prebuilt/linux-x86/toolchain/arm-eabi-4.2.1/bin/arm-eabi-gdb
out/target/product/generic/symbols/system/bin/app_process
[This brings up a version of gdb for cross-debugging the arm
instructions from linux. It says:
This GDB was configured as "--host=i386 --target=arm-elf-linux"...
Its prompt is "(gdb)"
I don't know how PC users should do this.]
Give gdb the following commands:
(gdb) set solib-absolute-prefix
/home/bhayes/android/out/target/product/generic/symbols
(gdb) set solib-search-path
/home/bhayes/android/out/target/product/generic/symbols/system/lib
[It seems absolute paths are needed here.]
(gdb) target remote localhost:12345
[Recall that you told gdbserver to use port 10.0.2.2:12345, and you
told the emulator to forward that traffic to localhost:12345. Now
you're telling gdb to look at port localhost:12345 for its remote
debugging server's traffic.]
[This generates a large number of warnings, which it seems I may ignore.]
[...]
warning: .dynamic section for
"/home/bhayes/android/out/target/product/generic/symbols/system/lib/libm.so"
is not at the expected address
warning: difference appears to be caused by prelink, adjusting expectations
[...]
warning: .dynamic section for
"/home/bhayes/android/out/target/product/generic/symbols/system/lib/libicui18n.so"
is not at the expected address (wrong library or version mismatch?)
[...]
[and then a lot of encouraging messages]
[...]
Loaded symbols for
/home/bhayes/android/out/target/product/generic/symbols/system/lib/libdvm.so
[...]
warning: Unable to find dynamic linker breakpoint function.
GDB will be unable to debug shared library initializers
and track explicitly loaded dynamic code.
__futex_wait () at bionic/libc/arch-arm/bionic/atomics_arm.S:150
150 ldmia sp!, {r4, r7}
Current language: auto; currently asm
(gdb)

[And at long last, there I am, debugging my mods. Yay.]

[joel]

Barry, are you able to step from that point through all your code? I
can get to the point you described (in fact that exact instruction
shown in your example) but soon after I start stepping through, I get
a SIGSEGV.

[Barry Hayes]

Well, first off, it doesn't start in "my code". I would set a
breakpoint where I wanted and
(gdb) cont

And to answer you.
The first time I tried:

warning: Unable to find dynamic linker breakpoint function.
GDB will be unable to debug shared library initializers
and track explicitly loaded dynamic code.
__futex_wait () at bionic/libc/arch-arm/bionic/atomics_arm.S:150
150 ldmia sp!, {r4, r7}
Current language: auto; currently asm

(gdb) step
[New Thread 129]
[Switching to Thread 129]
__futex_wait () at bionic/libc/arch-arm/bionic/atomics_arm.S:151
151 bx lr
(gdb) step
__pthread_cond_timedwait_relative (cond=<value optimized out>, mutex=0x119498,
reltime=0x0) at bionic/libc/bionic/pthread.c:1151
1151 pthread_mutex_lock(mutex);
Current language: auto; currently c
(gdb) step
pthread_mutex_lock (mutex=0x119498) at bionic/libc/bionic/pthread.c:936
936 {
(gdb) step
937 if (__likely(mutex != NULL))
(gdb) step
936 {

and so on for a while until:
(gdb)
79 add pc, r3, #(kernel_cmpxchg - kernel_atomic_base)
Could not insert single-step breakpoint at 0xffff0fc0
(gdb)
79 add pc, r3, #(kernel_cmpxchg - kernel_atomic_base)
Could not insert single-step breakpoint at 0xffff0fc0
(gdb) where
#0 __atomic_cmpxchg () at bionic/libc/arch-arm/bionic/atomics_arm.S:79
#1 0xafe0ea2c in pthread_mutex_lock (mutex=0x119498)
at bionic/libc/bionic/pthread.c:842
#2 0xafe0f0ac in __pthread_cond_timedwait_relative (
cond=<value optimized out>, mutex=0x119498, reltime=0x0)
at bionic/libc/bionic/pthread.c:1151
#3 0xafe0f128 in __pthread_cond_timedwait (cond=0x11949c, mutex=0x119498,
abstime=0x0, clock=-61441) at bionic/libc/bionic/pthread.c:1181
#4 0xad072e6c in waitMonitor (self=<value optimized out>, mon=0x119480,
msec=<value optimized out>, nsec=48184, interruptShouldThrow=true)
at dalvik/vm/Sync.c:584
#5 0xad0733c4 in dvmObjectWait (self=0xad00e500, obj=<value optimized out>,
msec=-4716827917345047496, nsec=0, interruptShouldThrow=true)
at dalvik/vm/Sync.c:993

But I tried again, and "step" seems to be hanging.
But no SIGSEGV.