Advice on properly using c++_shared as middleware

[jim.b...@couchbase.com]

I am trying to consolidate information from the following two pages:

https://developer.android.com/ndk/guides/middleware-vendors

https://developer.android.com/ndk/guides/cpp-support

In short it looks like on the one hand the advice is "don't use c++_static because it could violate the one definition rule".  Sure that makes sense.  On the other hand it says "don't use c++_shared if you are distributing middleware because it could conflict with versions used by the app".  What are we supposed to do in this situation?  I'll briefly describe our setup.

We develop a native library (let's call it libnative.so) that we use in Android, and in other platforms (.NET Linux, Java Desktop linux, etc).  We build this library, but obviously since it's just a naked shared lib JNI has no knowledge of how to use it.  So enter libnativeJNI.so, which has the JNI glue and links with libnative.so.  This gets packaged up and shipped out as a maven package that app developers can then use.

Since there are two shared libs here, I am assuming the advice is to use c++_shared (the exact thing that the middleware docs say NOT to do) so that both of these libraries use the same runtime.  However, I understand why we do not want to include the libc++_shared.so in our package since the app could be including it too.  However, if the app doesn't include it then where does it come from? 

Any advice on how to handle this situation would be appreciated.  Until now I haven't been aware of this and have been using the CMake default, which means both of the libs consume libc++_static.  I only discovered this as a potential issue when enabling ASAN and seeing strange behavior.

[John Dallman]

What I do, which has worked fine so far, is use libc++_shared.so, and require my customers to supply it in their apps. I document the NDK version that I'm using so that my customers know what version is required. 

John

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/ea636703-aef2-42b5-bca7-df1788f2961dn%40googlegroups.com.

[jim.b...@couchbase.com]

That seems like an odd burden to put on a customer who may or may not know/care about the NDK.  Do you have a documentation page for your customers describing this or something?  Do you know if there is a way to signal that a maven package needs it without relying on customer knowledge to include it?

[enh]

On Mon, Jan 29, 2024 at 12:34 PM 'jim.b...@couchbase.com' via android-ndk <andro...@googlegroups.com> wrote:

That seems like an odd burden to put on a customer who may or may not know/care about the NDK. 

aye, but that's what the NDK docs are trying to say --- these are the choices, and they're inherently awkward for middleware.

(which is why, given the general lack of ABI guarantees with C++, you might prefer to only expose C types/functions to your users, not C++ ones. if you've ever wondered why Android's own ABI is all C...)

 

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/8698437a-fbbc-4c65-b519-805a9adb6b0an%40googlegroups.com.

[Dan Albert]

So enter libnativeJNI.so, which has the JNI glue and links with libnative.so. 

The best fix would be to merge these two, statically link libc++, and use a version script to ensure that the only exposed function is `JNI_OnLoad`: https://developer.android.com/ndk/guides/middleware-vendors#java_middleware_with_jni_libraries

If you for some reason can't merge those libraries, but the API surface of libnative.so is C (or at least minimal C++: no RTTI/exception across the boundary, no stdlib types used at the boundary, memory allocated by one library is not deallocated by the other, that sort of thing) rather than C++, apply that advice to libnative.so (statically link libc++, use a version script to limit the visible symbols to your known C API and nothing more), and then do the same in libnativejni.so. Both libraries would have a private copy of libc++, which avoids the worst of the ODR problems (crashes, RTTI/exception issues). If libnativejni.so can be written in C, you can avoid one of those libc++ copies.

If you can't merge the two, and the interface between libnative.so and libnativejni.so is C++ and communicates stdlib types across that boundary, you need to go talk to whomever is preventing you from merging the two libraries, because that's the only possible fix.

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAJgzZooNLJeAuYv6b_3Ab-0-M9orS7avjifpYO%2BPobJtLTGQ0A%40mail.gmail.com.

[enh]

On Mon, Jan 29, 2024 at 1:26 PM 'Dan Albert' via android-ndk <andro...@googlegroups.com> wrote:

So enter libnativeJNI.so, which has the JNI glue and links with libnative.so. 

The best fix would be to merge these two, statically link libc++, and use a version script to ensure that the only exposed function is `JNI_OnLoad`: https://developer.android.com/ndk/guides/middleware-vendors#java_middleware_with_jni_libraries

If you for some reason can't merge those libraries, but the API surface of libnative.so is C (or at least minimal C++: no RTTI/exception across the boundary, no stdlib types used at the boundary, memory allocated by one library is not deallocated by the other, that sort of thing) rather than C++, apply that advice to libnative.so (statically link libc++, use a version script to limit the visible symbols to your known C API and nothing more), and then do the same in libnativejni.so. Both libraries would have a private copy of libc++, which avoids the worst of the ODR problems (crashes, RTTI/exception issues). If libnativejni.so can be written in C, you can avoid one of those libc++ copies.

If you can't merge the two, and the interface between libnative.so and libnativejni.so is C++ and communicates stdlib types across that boundary, you need to go talk to whomever is preventing you from merging the two libraries, because that's the only possible fix.

...which is why C++ is problematic for middleware, unless you're providing source to your users. the "make sure you use the same NDK i did" option is basically the only non-full-source option for middleware[1].

___

1. note that _in practice_ you can get away with all kinds of things, much of the time. but assuming your actual question is "how can i actually be guaranteed to be safe?"...

 

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAFVaGhtkGX4NOBiTq-tmsJbtzY7ZVu3_M4-%3DLdzuuKATnMBL1A%40mail.gmail.com.

[jim.b...@couchbase.com]

Would it make a difference if I said that we don't intend to expose any C or C++ APIs to our end users at all?  The expected workflow is that the end user codes against our Java API, and our Java API makes use of the included native components.  Actually, for the purposes of .NET (which cannot easily link with C++) libnative.so has a complete and usable C API, and all C++ symbols are hidden.  That being said, when libnativeJNI.so is built it uses some header only C++ components.  I don't directly develop the JNI library, but I have been directly involved with the libnative.so development so I can't say for sure if any linker script is used (though I suspect not) to hide things. 

So if I understand correctly, libnative.so is already c++ static safe?  And JNI can be made so if everything aside from JNI_OnLoad is hidden (the other JNI calls via "native" marked Java functions will still be usable right?  The things that look like com_whatever_native_function, or should those be listed in global as well?). 

Before these very helpful insights, I have been attempting to do the following:

Modify all libnative.so and libnativeJNI.so to use c++_shared, and include libc++_shared.so in our maven package.  I don't mind if it is clobbered by the app's one, if it even includes one (which I am assuming the majority of our users don't) since we already keep a pretty old C++ API usage.  I get fuzzy here since I don't know if there have been any libc++ ABI breaks between say r18b and now.  My plan was to build an app which links with NDK r18b, and have it clobber our libc++_shared so ensure that everything still works (the support matrix of Clang indicates that clang 7 is plenty new enough for the features we use). I also wasn't sure if modern gradle would just fail the build outright with two identically named libraries, or if it just warns and clobbers.  I also wasn't sure that if the two were identical if that was still an error condition.

However with this new information, it looks like I should be working back toward c++ static again?

[enh]

On Mon, Jan 29, 2024 at 3:00 PM 'jim.b...@couchbase.com' via android-ndk <andro...@googlegroups.com> wrote:

Would it make a difference if I said that we don't intend to expose any C or C++ APIs to our end users at all?  The expected workflow is that the end user codes against our Java API, and our Java API makes use of the included native components.  Actually, for the purposes of .NET (which cannot easily link with C++) libnative.so has a complete and usable C API, and all C++ symbols are hidden.  That being said, when libnativeJNI.so is built it uses some header only C++ components.  I don't directly develop the JNI library, but I have been directly involved with the libnative.so development so I can't say for sure if any linker script is used (though I suspect not) to hide things. 

So if I understand correctly, libnative.so is already c++ static safe? 

from what you've said, yes.

 

And JNI can be made so if everything aside from JNI_OnLoad is hidden (the other JNI calls via "native" marked Java functions will still be usable right?  The things that look like com_whatever_native_function, or should those be listed in global as well?). 

your JNI_OnLoad needs to call RegisterNatives(): https://developer.android.com/training/articles/perf-jni#native-libraries

at that point, _only_ JNI_OnLoad needs to be non-hidden.

 

Before these very helpful insights, I have been attempting to do the following:

Modify all libnative.so and libnativeJNI.so to use c++_shared, and include libc++_shared.so in our maven package.  I don't mind if it is clobbered by the app's one, if it even includes one (which I am assuming the majority of our users don't) since we already keep a pretty old C++ API usage.  I get fuzzy here since I don't know if there have been any libc++ ABI breaks between say r18b and now. 

(this is the part you can't really guarantee, which is why you don't want to go this route.)

 

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/7ca68c15-0bd0-4bb6-bf6c-554835f34988n%40googlegroups.com.

[jim.b...@couchbase.com]

Thanks!  I learned quite a bit from this thread and have a clear direction forward for prod which is less painful then the one I was considering before all this information!  If I may push just a bit further, the only reason I ran into this in the first place was trying to debug a third-hand heap corruption issue using ASAN.  Is it a requirement to use c++_shared for ASAN?  The docs suggest that "exception handling" doesn't work.  Does that mean for any exceptions thrown ever?  Including ones caught and handled within the same library?  Or just ones that cross the border?  I know ASAN is deprecated but this is a crash report we received from a customer of a customer so asking them to flash a special android image or upgrade to the brand newest Android version is not really viable.  The linked issue seemed to be abruptly closed by "ASAN is bad, use HWAsan" before the conclusion to the thread regarding c++_static was reached.  I don't disagree with that conclusion, but for now it seems like it is ASAN or nothing.  Obviously we aren't pushing an ASAN version into production so we don't need many guarantees other than "could it possibly work before barfing on the first exception it came to?" but the whole chain of people involved were willing to give an ASAN enabled build a try to diagnose the issue.

[John Dallman]

Ah, we have different expectations. I'm producing libraries for customers who develop against them in C/C++. I don't provide a JNI interface, or distribute through a package manager. I do include the need for libc++_shared in documentation, along with all my compile and link options.

John

To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/8698437a-fbbc-4c65-b519-805a9adb6b0an%40googlegroups.com.

[Andrea Fiorito]

In a very much smaller scale, I'm trying to modularise the components for the R&D prototypes I'm working on (3d engine, camera module, slam module etc.) and faced the same problems mentioned above. For context, I'm building .aar packages with prefab.prefabPublishing = true on the libraries and prefab.prefab = true on the main project. I encountered the situation of 2 c++ shared libs found (OpenCV module and my 3d engine module) and solved with the not optimal solution packagingOptions.jniLibs.pickFirsts += ['**/libc++_shared.so']. The libc++_shared.so in my case should be the same cause I compile both modules using the same ndk version.
Would be enough, in the aforementioned case of middleware, to give a set of versions compiled with a range of ndk versions supported? For example, mylib_ndk_26.1.10909125.aar, mylib_ndk_25.1.8937393.aar, etc.
I see inside the aar there is a copy of libc++_shared.so, would be possible to enforce a ndk version check using some metadata, for example prefab/prefab.json? Or with a checksum or both, so to ensure there are not accidental mistakes when combininig multiple libraries?