breakpoints ignored when debugging native code with eclipse

[Piotr Morgwai Kotarbiński]

Hey all,
I'm trying to debug native code with eclipse ndk plugin, but the breakpoints I set don't work. On the gdb console I can see the following output:

warning: Could not load shared library symbols for 52 libraries, e.g. /system/bin/linker.
Use the "info sharedlibrary" command to see the complete listing.
Do you need "set solib-search-path" or "set sysroot"?
warning: Unable to find dynamic linker breakpoint function.
GDB will retry eventurally.  Meanwhile, it is likely
that GDB is unable to debug shared library initializers
or resolve pending breakpoints after dlopen().
No symbol table is loaded.  Use the "file" command.
No symbol table is loaded.  Use the "file" command.

The output on "gdb traces" console is attached as a gdb.log file.

When I use ndk-gdb script everything works fine.

Here is my environment config:
host platform: Linux Mint 15 on x86_64
Eclipse Version: Juno Service Release 2
NDK version: r8e
SDK version: 22.0.1
CDT version: 8.1.2
I have android:debuggable set to true in my AndroidManifest.xml and NDK_DEBUG=1 added as argument to ndk-build in "C/C++ Build".

Any ideas where the problem is?

Thanks!

[ash.fa...@gmail.com]

Make sure you have set the path in the debug configurations to the gdb command file.
Are u sure u r getting the gdb.setup and server files being created in libs/{platform}/ folder???
Even after that if you are not getting it, then try to run ndk-gdb in the command line. Doing this sets up some paths that are not done ...
Don't worry about the warnings, they do not matter.

[Piotr Morgwai Kotarbiński]

Hey,
Thanks for the reply! Please see inline below:

On Friday, July 19, 2013 4:55:07 PM UTC+7, ash.fa...@gmail.com wrote:

Make sure you have set the path in the debug configurations to the gdb command file.

Could you please elaborate on this? In the "debugger" tab of my configuration eclipse automatically filled "Debugger" field with ${NdkGdb} and left "GDB command file" empty. Should I put anything there? (it also added ${NdkProject}/obj/local/${NdkCompatAbi}/ to "Shared libraries" list and set port to 5039)

Are u sure u r getting the gdb.setup and server files being created in libs/{platform}/ folder???

yes, I do: I have gdbserver, gdb.setup and all my native libs there.


 

Even after that if you are not getting it, then try to run ndk-gdb in the command line. Doing this sets up some paths that are not done ...

I'm doing it all the time as this is the only way debugging works for me. It doesn't help debugging in eclipse in any way though...


 

Don't worry about the warnings, they do not matter.

Well, the warning says "...it is likely that GDB is unable to (...) resolve pending breakpoints" and that's exactly what my problem is. So it seems to me that figuring out why GDB is "unable to find dynamic linker breakpoint function" will probably help to solve my issue...


Thanks!

[John McFarlane]

Just checking: do you have -g specified on compiler command line?

[Piotr Morgwai Kotarbiński]

Hey,
I don't exactly understand what you mean: as I wrote before I'm building with `ndk-build NDK_DEBUG=1` command as described here: http://tools.android.com/recent/usingthendkplugin
ndk-build does not have -g flag...

Thanks!

[John McFarlane]

ndk-build is not the compiler, it's the builder. This in turn causes the compiler to be executed for each source file. 

You should be able to find the compiler command line (or options which affect it) under the "C/C++ Build -> Settings" category. Alternatively, you may be able to alter compiler flags if you can find a LOCAL_CPPFLAGS in the Android.mk file associated with your source files.

Sorry to be so vague but I haven't used Eclipse in a while.

[Piotr Morgwai Kotarbiński]

Sorry, I should have emphasized again that I verified that my binaries do have debugging symbols and are generally debuggable: as I wrote in the initial post I can debug them without problems from the command line with ndk-gdb after they are built by eclipse and deployed to the emulator.
My problem seems to be related to the eclipse embedded ndk-gdb configuration (the warning saying that gdb is "unable to find dynamic linker breakpoint function" and thus is "unable to (...) resolve pending breakpoints" seems to be the key)

Thanks!

[Radovan Chytracek]

Since the native shared libraries are loaded explicitly the pending
breakpoints do not work well with Android and stay pending forever.
What you can to is to store the breakpoints in a gdb breakpoints file
and load them once you are sure your debug symbols are loaded for the
native library you are debugging.
Verify with "info shared" if the shared library is loaded ok.
Check with "thread apply all bt" is you can see correctly the stack
traces of all of your native threads and also the symbols and source
code.
Then load the breakpoints you saved before and if all your gdb
settings are OK and you use the right libraries you should be able to
set them.

> --
> You received this message because you are subscribed to the Google Groups
> "android-ndk" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to android-ndk...@googlegroups.com.
> To post to this group, send email to andro...@googlegroups.com.
> Visit this group at http://groups.google.com/group/android-ndk.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Piotr Morgwai Kotarbiński]

On Saturday, July 20, 2013 4:55:27 PM UTC+7, Radovan Chytracek wrote:

Since the native shared libraries are loaded explicitly the pending
breakpoints do not work well with Android and stay pending forever.
What you can to is to store the breakpoints in a gdb breakpoints file
and load them once you are sure your debug symbols are loaded for the
native library you are debugging.

That would make debugging via Eclipse much less useful... :(
I was also trying to set a breakpoint after the gdb has connected to my process, but it didn't help: the breakpoint was ignored anyway.


 

Verify with "info shared" if the shared library is loaded ok.
Check with "thread apply all bt" is you can see correctly the stack
traces of all of your native threads and also the symbols and source
code.

Hmm, I don't know how to issue a gdb command when debugging via eclipse: is it possible at all?

Thanks!





 

[Piotr Morgwai Kotarbiński]

I have just reproduced the problem with a fresh install of eclipse Kepler (4.3)

Does anyone have similar problem actually or is it only me? I'm trying to figure out if it's a bug or just some problem with configuration of my environment.

Thanks!

[Radovan Chytracek]

I basically gave up on trying using ndk-gdb with eclipse and use the
gdbserver launched by hand on the target and connect with armeabi GDB
client from Eclipse and use my explicit .gdbinit

[Piotr Morgwai Kotarbiński]

it seems that this is a known problem:

http://code.google.com/p/android/issues/detail?id=35274

while the title says "if gdbserver connection happens after the JNI code executes" many people reported that breakpoints doesn't work for them at all just like for me.

Cheers!

[ash.fa...@gmail.com]

Hey, sorry for the delay... There you go... the gdb command file should not be empty.. It should point to the gdb.setup in your libs/ architecture (armeabi or mips or x86 or armeabi-v7)/ folder..

Reply if that works coz it will help many....

Regards,

Ash.

[Piotr Morgwai Kotarbiński]

Hey,
I've just given it a try, unfortunately it hasn't helped: everything is as before :(

Thanks!

[Piotr Morgwai Kotarbiński]

great news:
on eclipse Kepler running ndk-gdb once solves the problem just as you suggested before! Now I'm able to debug my app fully from eclipse :)))
It doesn't even require anything to be put in "GDB command file" field (I guess it picks gdb.setup from libs/<arch> automatically by default).

Could you please explain what paths exactly are set when running ndk-gdb from a command line and where are these settings stored? I'd like to understand what's actually happening here. You can put this info directly to http://code.google.com/p/android/issues/detail?id=57865 that I created (as it is a different situation than the original report in 35274 as described here: http://code.google.com/p/android/issues/detail?id=35274#c18).

Many many thanks for your help!!!

On Saturday, July 20, 2013 11:42:39 PM UTC+7, ash.fa...@gmail.com wrote:

[Piotr Morgwai Kotarbiński]

On Sunday, July 21, 2013 10:36:48 PM UTC+7, Piotr Morgwai Kotarbiński wrote:

Could you please explain what paths exactly are set when running ndk-gdb from a command line and where are these settings stored? I'd like to understand what's actually happening here.

I managed to figure it out myself: http://code.google.com/p/android/issues/detail?id=57865#c3

Once again many thanks for your help, Ash!

[Jeffrey Walton]

Hi Morg. Could you cat both 'gdb.setup' and 'linker'? (Are both text files?).

I've had problems with ndk-gdb too. I don't use typically Eclipse
since the libraries I test/approve ave command line drivers.

Jeff

[Piotr Morgwai Kotarbiński]

Hey Jeff :)
So  I myself never have had any problems with ndk-gdb: it has always been working fine for me. I'm guessing from the other threads that you might be building your stuff with standalone toolchain and your own Makefiles rather than with ndk-build script and Android.mk files. In such case ndk-gdb might get confused as far as I understand...

Anyway, the files in question:
'linker' is a binary file: I have just figured out that it is /system/bin/linker file extracted from the image of the android platform you are working with. You can get it from your emulator/device with 'adb pull' easily.
'gdb.setup' contents is pasted below:

---begin gdb.setup----
set solib-search-path ./obj/local/armeabi
directory /var/home/morgwai/opt/android-ndk/platforms/android-8/arch-arm/usr/include jni/cryptopp/include /var/home/morgwai/opt/android-ndk/sources/cxx-stl/stlport/stlport /var/home/morgwai/opt/android-ndk/sources/cxx-stl//gabi++/include jni/crypt_user
file ./obj/local/armeabi/app_process
target remote :5039
---end---

Hope it will help you.

Cheers!

[Steve IllLetYouKnow]

This was the most useful post I found on this subject.
To summarize what I found in my two days of head-pounding:
     The newest ADT/SDK/NDK (as of now march29,2014) allow native debugging.
     The manual run-from-console IS still required as explained in this thread.
     It takes about 5 seconds from the time you LoadLibrary in java until the first breakpoint is actually set.
          (2.5 GHz,Xeon quad core 64 bit)
     Any C/C++ calls before that will take place but the breakpoints will look like they aren't working.
     You have to have your debug configuration as an Android Native Application.
..... There may be more, I tried so many things in the past 2 days....
      If there is more, it's well covered in the google pages you'll find.
Thanks Piotr,  my head is feeling better already.