Problems with libc++_shared.so in LLDB, NDK27c, Android 14.

[John Dallman]

Preface: I am not producing an Android app. I work for a software
component business, creating shared libraries, compiled from C and C++
code, for use in third-party customers' apps. I test my libraries in a
command-line test harness, which I run in the ADB shell. I am only
producing software for 64-bit ARM, because none of my customers want
32-bit code.

I do not use Android Studio, at all, nor any of the supplied build
systems. The software is written in a domain-specific language, which
cannot be compiled correctly by build systems that use file timestamps
to detect changes. It's been pretty easy so far to teach our buld
system for Unix-like platforms to use the NDK C/C++ compiler, linker,
headers and libraries. I only use the NDK and the Platform Tools. I
run those entirely on Linux (Rocky 8.10) and connect to the various
development and test devices over USB. .

I'm in the process of switching from NDK23 to NDK27. I have some SEGVs
in my test results, so I want to use the debugger. Because I'm going
to have to support developers with no familiarity with Android fixing
their regressions, and I'm not much good at Python, I'm trying to set
it up using lldb, lldb-server, and shell scripts. I have that working
for C programs, but C++ programs can't find libc++_shared.so when I
run them under the debugger, thus:

(lldb) run
Process 20046 launched:
'/u/jgd/regimes/tools_android/patssy/lx86/world_11_debug' (aarch64)
CANNOT LINK EXECUTABLE "/data/local/tmp/world_11_debug": library
"libc++_shared.so" not found: needed by main executable
Process 20046 exited with status = 1 (0x00000001)
(lldb)

libc++_shared.so is in the same directory (/data/local/tmp) as my test
executable, and I set and exported LD_LIBRARY_PATH=/data/local/tmp in
the adb shell window where I subsequently started lldb-server.

I looked for other versions of libc++_shared.so and only found ones
for other architectures (32-bit arm, i686, x86-64 and risc-v). I found
libc++.so and .a libraries in directories that indicate they're for
specific Android versions. I'm curious as to what those are for, and
if I should be making any use of them?

I tried the libc++_shared.so from NDK28, but it behaves just the same.

I'll get a full repro case together later, after I've dealt with some
corporate bureaucracy, but I wondered if I was missing something
obvious?

Thanks in advance,

John

[enh]

are you sure it's actually making it through? because that error is
the exact error you'd see if it wasn't.

as a hack, stick a sleep() or getchar() in the start of your main, run
it normally, and then attach to the running process by pid.

as an alternative hack, debug a static binary.

https://stackoverflow.com/questions/33587130/why-doesnt-lldb-forward-my-environment-variable-anymore
isn't obviously relevant as a cause (because that sounds like macOS
explicitly filtering the specific environment variable they're talking
about, rather than a general inability to pass environment variables
through) but does offer another workaround.

> I looked for other versions of libc++_shared.so and only found ones
> for other architectures (32-bit arm, i686, x86-64 and risc-v). I found
> libc++.so and .a libraries in directories that indicate they're for
> specific Android versions. I'm curious as to what those are for, and
> if I should be making any use of them?
>
> I tried the libc++_shared.so from NDK28, but it behaves just the same.
>
> I'll get a full repro case together later, after I've dealt with some
> corporate bureaucracy, but I wondered if I was missing something
> obvious?
>
> Thanks in advance,
>
> John
>

> --
> You received this message because you are subscribed to the Google Groups "android-ndk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/android-ndk/CAH1xqgm9ncEjPTYbaiFAOZFv7XdWT90RmOcuoo_ricjxoR-jcw%40mail.gmail.com.

[John Dallman]

On Tue, Mar 25, 2025 at 12:41 PM 'enh' via android-ndk
<andro...@googlegroups.com> wrote:
> are you sure it's actually making it through? because that error is
> the exact error you'd see if it wasn't.

Indeed. I tried the workaround in that Stack Overflow question, by running:

(lldb) platform shell LD_LIBRARY_PATH=/data/local/tmp
(lldb) platform shell echo $LD_LIBRARY_PATH
/data/local/tmp

before saying "run" to the debugger, but I get exactly the same error.

> as a hack, stick a sleep() or getchar() in the start of your main, run
> it normally, and then attach to the running process by pid.
>
> as an alternative hack, debug a static binary.

I'll try those after pulling a complete repro case together.

I am not doing any kind of initialisation of NDK variables, just
setting PATH to include some of its directories. Could that be a
problem?

Thanks,

John

> To view this discussion visit https://groups.google.com/d/msgid/android-ndk/CAJgzZopUkU5QBXwJ_gFSPq3tW_6fgpYx8%3DBKmVBWYzSEL7QTSQ%40mail.gmail.com.

[enh]

On Tue, Mar 25, 2025 at 9:46 AM John Dallman <jgdats...@gmail.com> wrote:
>
> On Tue, Mar 25, 2025 at 12:41 PM 'enh' via android-ndk
> <andro...@googlegroups.com> wrote:
> > are you sure it's actually making it through? because that error is
> > the exact error you'd see if it wasn't.
>
> Indeed. I tried the workaround in that Stack Overflow question, by running:
>
> (lldb) platform shell LD_LIBRARY_PATH=/data/local/tmp
> (lldb) platform shell echo $LD_LIBRARY_PATH
> /data/local/tmp
>
> before saying "run" to the debugger, but I get exactly the same error.

i'm assuming there's no selinux domain transition or similar going on
here? (LD_LIBRARY_PATH is on the list of unsafe environment variables
that the dynamic linker clears if AT_SECURE is set.)

if you have https://android-review.googlesource.com/c/platform/bionic/+/3250573
(which basically means "you're running a developer preview/beta of
this year's release") you have an LD_DEBUG environment variable
similar in use to the glibc one (and use LD_DEBUG=help for more
information). anything older and you're stuck with a fairly
inscrutable integer, but you can try setting that to increasingly high
numbers until you see where it's searching for things. (note though
that LD_DEBUG is also on the AT_SECURE blacklist.)

(i didn't realize until looking for a link just now that LD_DEBUG is
undocumented. https://android-review.googlesource.com/c/platform/bionic/+/3560463
addresses that.)

> > as a hack, stick a sleep() or getchar() in the start of your main, run
> > it normally, and then attach to the running process by pid.
> >
> > as an alternative hack, debug a static binary.
>
> I'll try those after pulling a complete repro case together.
>
> I am not doing any kind of initialisation of NDK variables, just
> setting PATH to include some of its directories. Could that be a
> problem?

i'm not aware that we have any custom changes to lldb.

> To view this discussion visit https://groups.google.com/d/msgid/android-ndk/CAH1xqg%3Dywok06PRt3EOqAshaKdXr7q9pPH%2B3tBPK5yp0JzG4Pg%40mail.gmail.com.

[John Dallman]

On Tue, Mar 25, 2025 at 3:05 PM 'enh' via android-ndk
<andro...@googlegroups.com> wrote:

> i'm assuming there's no selinux domain transition or similar going on
> here? (LD_LIBRARY_PATH is on the list of unsafe environment variables
> that the dynamic linker clears if AT_SECURE is set.)

I'm not trying to do anything with selinux settings: I'm just naively
running a command-line binary in the adb shell.

$AT_SECURE is not set in the adb shell.

pineapple:/data/local/tmp $ getprop | grep -i selinux
[selinux.restorecon_recursive]: [/data/misc_ce/0]
pineapple:/data/local/tmp $ uname -a
Linux localhost 6.1.57-android14-11-maybe-dirty #1 SMP PREEMPT Thu Jan
1 00:00:00 UTC 1970 aarch64 Toybox

> if you have https://android-review.googlesource.com/c/platform/bionic/+/3250573
> (which basically means "you're running a developer preview/beta of
> this year's release") you have an LD_DEBUG environment variable
> similar in use to the glibc one (and use LD_DEBUG=help for more
> information). anything older and you're stuck with a fairly
> inscrutable integer, but you can try setting that to increasingly high
> numbers until you see where it's searching for things. (note though
> that LD_DEBUG is also on the AT_SECURE blacklist.)

See above, it's Android 14. I'll try LD_DEBUG integers. If that
doesn't help, I have a bug report nearly ready to zip up. Where should
I send it?

Thanks very much,

> To view this discussion visit https://groups.google.com/d/msgid/android-ndk/CAJgzZoosvGA%2BgrDwzZGs0-pvOK4C%3DQK8iiNGDUYDBTxYAzmHFQ%40mail.gmail.com.

[John Dallman]

Here's my bug report, which I got down to a C++ source file and a
shell script that provides instructions when you run it.

It certainly looks as if $LD_LIBRARY_PATH is ignored by the
lldb-server in NDK27c.

If there's a bug-tracker I should submit this to, please let me know
and I'll do it. This is the first apparent bug I;ve reported against
Android or the NDK after five years on it, which is pretty good. I've
reported about a hundred against Windows down the years.

I'll start on workarounds now.

Thanks again,

John

[enh]

https://github.com/android/ndk/issues, same as always :-)

> To view this discussion visit https://groups.google.com/d/msgid/android-ndk/CAH1xqgkJBERg%2BE8FP0Qta6Ov-%3D-d3EJGS3JiPrBOmurZ%3DEtDkQ%40mail.gmail.com.

[Dan Albert]

That's the right link assuming it's an NDK bug, anyway. I haven't had time to digest the thread, but I see discussion of selinux. I can't fix platform issues in the NDK if that's the problem. Those would be "System" under http://b.android.com, I think. If you're not sure one way or the other, feel free to start it in the NDK at https://github.com/android/ndk/issues.

To view this discussion visit https://groups.google.com/d/msgid/android-ndk/CAJgzZop1MJrVcsSWiJk6RbVgZS0RG7Uqnk%3DpfPcvT8cH6Pzmpw%40mail.gmail.com.

[Florian Mayer]

Just confirming: How are you running lldb-server? I think the LD_LIBRARY_PATH should go into the program to launch, not in front of the lldb-server.

So not "LD_LIBRARY_PATH lldb-server [...] program" but "lldb-server [...] LD_LIBRARY_PATH program".

I don't have a phone to confirm right now, but wanted to throw that in.

[John Dallman]

L$ adb shell
A$ cd /data/local/tmp
A$ export LD_LIBRARY_PATH=$(pwd)
A$ ./lldb-server platform --listen '*:5039' --server

Unlike gdb-server, you don't need to start lldb-server for a
particular program, AFAICS.

Thanks, John

Using "platform shell echo $LD_LIBRARY_PATH" shows the setting above.

> To view this discussion visit https://groups.google.com/d/msgid/android-ndk/62a0035e-0b46-4a73-b134-b34de0b540cdn%40googlegroups.com.

[John Dallman]

I am not attempting to interact with selinux in any way.

> if you have https://android-review.googlesource.com/c/platform/bionic/+/3250573
> (which basically means "you're running a developer preview/beta of
> this year's release") you have an LD_DEBUG environment variable
> similar in use to the glibc one (and use LD_DEBUG=help for more
> information). anything older and you're stuck with a fairly
> inscrutable integer, but you can try setting that to increasingly high
> numbers until you see where it's searching for things. (note though
> that LD_DEBUG is also on the AT_SECURE blacklist.)

It appears that LD_DEBUG is not compiled into the Android on my test
device. I've set integers from 1 to 20, and used values to set the
button 4, 8, 16 and 32 bits, and I get no output of any kind.

thanks,

John

On Tue, Mar 25, 2025 at 3:05 PM 'enh' via android-ndk

> To view this discussion visit https://groups.google.com/d/msgid/android-ndk/CAJgzZoosvGA%2BgrDwzZGs0-pvOK4C%3DQK8iiNGDUYDBTxYAzmHFQ%40mail.gmail.com.

[enh]

On Thu, Mar 27, 2025 at 7:26 AM John Dallman <jgdats...@gmail.com> wrote:
>
> I am not attempting to interact with selinux in any way.
>
> > if you have https://android-review.googlesource.com/c/platform/bionic/+/3250573
> > (which basically means "you're running a developer preview/beta of
> > this year's release") you have an LD_DEBUG environment variable
> > similar in use to the glibc one (and use LD_DEBUG=help for more
> > information). anything older and you're stuck with a fairly
> > inscrutable integer, but you can try setting that to increasingly high
> > numbers until you see where it's searching for things. (note though
> > that LD_DEBUG is also on the AT_SECURE blacklist.)
>
> It appears that LD_DEBUG is not compiled into the Android on my test
> device. I've set integers from 1 to 20, and used values to set the
> button 4, 8, 16 and 32 bits, and I get no output of any kind.

(it's certainly possible that you have a build without any LD_DEBUG at
all, but it's also worth mentioning for posterity that LD_DEBUG
historically only output to logcat. i think my change to make it go to
stderr as well is new enough that if you have that, you also have the
human-readable comma-separated options [and 'help'].)

> To view this discussion visit https://groups.google.com/d/msgid/android-ndk/CAH1xqgm8oRvpfz3O1m3_cce1WUBXPGnLG0_Pui0qM0M6EY657A%40mail.gmail.com.

[John Dallman]

On Thu, Mar 27, 2025 at 12:33 PM 'enh' via android-ndk
<andro...@googlegroups.com> wrote:
> (it's certainly possible that you have a build without any LD_DEBUG at
> all, but it's also worth mentioning for posterity that LD_DEBUG

> historically only output to logcat. I think my change to make it go to

> stderr as well is new enough that if you have that, you also have the
> human-readable comma-separated options [and 'help'].)

Well spotted. I'll look into "adb logcat", the name being all I know about it.

Thanks again,

John


On Thu, Mar 27, 2025 at 12:33 PM 'enh' via android-ndk

> To view this discussion visit https://groups.google.com/d/msgid/android-ndk/CAJgzZoqERNkDQxGURQ%2BAQQsoywKobTqZEWMYzTsWEQjxEGLY2g%40mail.gmail.com.