Debugging software without debug instrumentation explained by means of Android

[AchimNohl]

Hi all,

It is not unusual that you need to debug or trace code that has no
debug information that would allow connecting a source level debugger.
This can have various reasons such as the software module is not owned
by you so you cannot re-build it, or your defect scenario would not
allow for a debug build as the fault disappears or the profiling data
is fudged. Not only in well layered software stacks, embedded software
modules are delivered in object code only (e.g. libraries) to
development teams that have to build their piece of the software on-
top of it. Still, as a developer it might be of high importance to
gain more insight into the workflow of those libraries to trace
defects in your code, or to proof that the problem is not on your
side. I want to share some ideas where this visibility into software
can be enabled by each software developer using a Virtual Platform for
the software execution. The pre-requisite for this kind of debugging
is at least access to the function symbols (addresses) and a view into
the source-code.
What I will try to explain is basically just a little extension to
what I have already introduced in “User defined software traces using
Virtual Platform scripting APIs”. Also, the explanations relate to
what I have written in my last post in “Less is more, system level
software centric power analysis”. Here, we had to trace the execution
of the Dalvik Virtual Machine (DVM) to figure out which Java process
has woken up the CPU every 500msec. In our DVM object code we want to
trace the name of the Java object that wakes up.

First of all, Virtual Platforms provide comprehensive build in tracing
capabilities that allow you to generate e.g. a process or function
trace. Using this trace (as e.g. shown here
http://virtual-platform-users.googlegroups.com/web/android_idle_analysis.png?hl=en)
we can figure out which functions need closer observation.

Let’s have a look at the source-code of the function that is called
after every wake-up:

file:dalvik/vm/Sync.c
void dvmObjectWait(Thread* self, Object *obj, s8 msec, s4 nsec, bool
interruptShouldThrow)

We are interested in the object’s name. Here, the object is the second
parameter of the function. Thus, for ARM and gcc the pointer value is
stored in the second ARM register R1 (starting with R0=first
parameter). In order to get the pointer, we simply use the e.g. the
Virtual Platform TCL Scripting API as provided by the VP in the cloud.

set obj [i_ARM926EJS_0/R get_value 1 ]

Now, this is just the pointer. But, we want to get the descriptive
Java object name instead. Therefore, let us have a look at the
definition of “Object” in “dalvik/vm/oo/Object.h”:

typedef struct Object {
ClassObject* clazz; // Byte 0
int* methodIndexArray; // Byte 4
} // Byte 8

struct ClassObject {
Object obj; // Byte 0
u4 instanceData[4]; // Byte 8
const char* descriptor; // Byte 24

Hhmm, the dscriptor string is not part of Object but of ClassObject
which is a member of Object. So, as a software developer we would
access it like this:

char* myDescriptor = obj->clazz->descriptor

The nice thing is that we can do it in a similar fashion using Virtual
Platform Scripting, thus without any change of the software. In the VP
in the cloud the TCL helper dptr and dstr allow us to dereference a
pointer or a string. Let's do it step by step:

# 1. We dereference the pointer $obj to get the value
# of the ClassObject pointer
set clazz [dptr $obj]

# 2. We need to get the address of descriptor in ClassObject
set descriptor_ptr [expr $clazz + 24 ]

This is the place where we need to spend some brain power. As no debug
info is available, we need to compute the offsets manually. Here,
descriptor is at byte offset 24 inside ClassObject.

# 3. We need to dereference the descriptor_ptr to get the address
# where the string is stored
set descriptor [dptr $descriptor_ptr]

# 4. Finally, we need to read out the string
set descriptor_contents [dstr $descriptor]

You get e.g.: Landroid/os/MessageQueue;

And, now we have the string. Looks cumbersome? Yes, indeed, but only
for the purpose of the explanations. You can make this very efficient
by putting the code into a generic, highly reusable helper procedure
such as:

proc get_descriptor { reg } {
return [dstr [dptr [expr ([dptr [i_ARM926EJS_0/R get_value $reg ] ]
+24)]]]
}

Now you just attach it to any function where you want to trace the
object name:

[create_breakpoint dvmObjectWait] set_callback "get_descriptor 1"

And you get a nice trace:
Ljava/lang/VMThread;
Landroid/os/MessageQueue;
Landroid/os/HandlerThread;
Landroid/os/MessageQueue;
Landroid/os/HandlerThread;

This principle can be extended to any property of the embedded
software. At the end you can create an extensive software specific
debug and analysis kit that can really boost the productivity. Of
course scripting gets a little slow if traces happen frequently, but
each script API can be also realized in the native simulation code of
the virtual platform. This will allow the analysis to run very fast.
We have done this for some aspects of Linux in out VP in the cloud as
you can see in the OS-Kits under the Analysis configuration.

Best regards,
Achim

PS: You need to have the libdvm symbols loaded. Either via menu or
via:
load_symbols [list [list $symbolsAndroidDir/system/lib/libdvm.so
0xad000000]]