Guidance on callbacks and memory managemnt
Miha
I'm developing an application which will use a lot of existing C++
libraries; I am therefore using the NDK. So far I've just written a
simple demo for myself testing the JNI stuff and callbacks. This
mailing list has been very helpful so far, but since I'm new to
android development and haven't touched C/C++ in 10+ years, I still
have lots to (re)learn.
The library I will be porting (and using) uses threads and is
asynchronous, which means I will be getting various events I should be
handling. I know that I must execute UI-modifying stuff on the UI
thread and so far, I've used Activity.runOnUiThread method on the main
activity instance for that purpose.
On the native side, I get a reference to a class in the JNI_OnLoad and
obtain the jmethodID of the callback methods which I invoke from the
native side-of-things. I cache the reference to the main class using
env->NewGlobalRef(clazz).
When I invoke my first method (from java to native), I store the
instance reference as a global variable in native code (again, I use
env->NewGlobalRef(instance)) and I use that global reference in the
thread procedure to invoke methods on it (callback methods).
The thread uses AttachCurrentThread to get a reference to JNIEnv and
then invokes the methods on the java object instance (the cached
reference).
Now, for some questions:
1) I did not find a way to get the instance of the main activity other
than by calling a native method from the activity. Is this OK? Or is
there an API call I'm missing?
2) Since I'm caching the reference to an _instance_ (the MainActivity
actually), I'm wondering if JNI_OnLoad will be called when this
instance is recreated? (my testing shows that if I exit the app and
then reenter it, the references are still valid and native library
still works as expected)
3) I'm creating strings via env->NewStringUTF and passing them on to
Java world. Who will free them? GC?
4) When should (MUST?) I call DeleteGlobalRef? Is there an
JNI_OnUnload equivalent to JNI_OnLoad available?
5) static functions don't work: I'm getting: No implementation found
for native ... error message. I declared the function static and also
the declaration in corresponding Java file. If I remove static,
everything works. The reason I wanted to use static is beacuse of a
JNI tip that "You should declare the methods "static" so the names
don't take up space in the symbol table on the device." I am not
explicitly registering functions though... (maybe autodiscovery
doesn't find static methods?)
6) Given that much of the codebase I'm integrating is in c++, does it
make sense to develop activities in the NDK as well? Perhaps that
would be easier because I wouldn't have to call java methods all the
time to update the UI?
7) Probably a silly question, but I'm having hard time figuring out
the whole C/C++ API that is available (I have yet to setup eclipse for
C++ dev). What are you guys using as a reference to a C/C++ API that
is available on Android? Currently, I'm between grepping the NDK
header files and googling for method signatures... Is there such a
thing as a compelte Bionic reference?
Thanks,
Miha.
Zoran Angelov
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To post to this group, send email to andro...@googlegroups.com.
To unsubscribe from this group, send email to android-ndk...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/android-ndk?hl=en.
Tim Mensch
> 1) I did not find a way to get the instance of the main activity
> other than by calling a native method from the activity. Is this OK?
> Or is there an API call I'm missing?
> 2) Since I'm caching the reference to an _instance_ (the
> MainActivity actually), I'm wondering if JNI_OnLoad will be called
> when this instance is recreated? (my testing shows that if I exit the
> app and then reenter it, the references are still valid and native
> library still works as expected)
I have my MainActivity pass in a new reference to itself, and then all of my connections between JNI and Java are rebuilt, releasing the old global references.
> 3) I'm creating strings via env->NewStringUTF and passing them on to
> Java world. Who will free them? GC?
> 4) When should (MUST?) I call DeleteGlobalRef? Is there an
> JNI_OnUnload equivalent to JNI_OnLoad available?
I'd say that you should call it in onDestroy(), but don't. onDestroy() has a bad habit of being called AFTER the onCreate() of the new instance, so there's no point in actually doing any destruction there if you've got statics in your JNI code. Just release everything when a new instance is created.
> 5) static functions don't work: I'm getting: No implementation found
> for native ... error message. I declared the function static and
> also the declaration in corresponding Java file. If I remove static,
> everything works. The reason I wanted to use static is beacuse of a
> JNI tip that "You should declare the methods "static" so the names
> don't take up space in the symbol table on the device." I am not
> explicitly registering functions though... (maybe autodiscovery
> doesn't find static methods?)
> 6) Given that much of the codebase I'm integrating is in c++, does
> it make sense to develop activities in the NDK as well? Perhaps that
> would be easier because I wouldn't have to call java methods all the
> time to update the UI?
> 7) Probably a silly question, but I'm having hard time figuring out
> the whole C/C++ API that is available (I have yet to setup eclipse
> for C++ dev). What are you guys using as a reference to a C/C++ API
> that is available on Android? Currently, I'm between grepping the
> NDK header files and googling for method signatures... Is there such
> a thing as a compelte Bionic reference?
Tim
[1] http://java.sun.com/developer/onlineTraining/Programming/JDCBook/jniref.html#call
Olivier Guilyardi
> 1) I did not find a way to get the instance of the main activity other
> than by calling a native method from the activity. Is this OK? Or is
> there an API call I'm missing?
It sounds OK to me. See ndk/docs/NATIVE-ACTIVITY.HTML for an alternative,
although your method is correct IMO if you have a normal Java activity.
> 2) Since I'm caching the reference to an _instance_ (the MainActivity
> actually), I'm wondering if JNI_OnLoad will be called when this
> instance is recreated? (my testing shows that if I exit the app and
> then reenter it, the references are still valid and native library
> still works as expected)
No, JNI_OnLoad is only called when the process which loads the library, and
hosts the Activity, is created. But a new process is not created whenever an
Activity in created. An app usually runs in a single process, which can stay
around for quite a long time, and within which activities can be created and
destroyed several times.
If your activity is only paused and resumed, then yes your activity reference is
still valid. This for example (usually) happens when the user presses Home and
then come back in your app. But if the user exits by pressing Back, then the
Activity will be destroyed, and you shouldn't hold a global reference to it in
your native code or you'll end up leaking memory, and will have certainly other
problems when working with this "zombified" activity.
> 3) I'm creating strings via env->NewStringUTF and passing them on to
> Java world. Who will free them? GC?
Yes
> 4) When should (MUST?) I call DeleteGlobalRef? Is there an
> JNI_OnUnload equivalent to JNI_OnLoad available?
There apparently is a JNI_OnUnload available:
http://download.oracle.com/javase/1.5.0/docs/guide/jni/spec/invocation.html#JNI_OnUnload
But I think you are still confusing process and activity lifecycle here. You
should delete the global references from onPause() or onDestroy().
That said, I hope that you're not relying on static variables in native code,
especially to hold your global references... Weird things may happen if you do
that. In my experience and opinion it's better to avoid any kind of native
static variable.
> 5) static functions don't work: I'm getting: No implementation found
> for native ... error message. I declared the function static and also
> the declaration in corresponding Java file. If I remove static,
> everything works. The reason I wanted to use static is beacuse of a
> JNI tip that "You should declare the methods "static" so the names
> don't take up space in the symbol table on the device." I am not
> explicitly registering functions though... (maybe autodiscovery
> doesn't find static methods?)
static in native code and static in Java are different things. Do not declare
the function static in C or C++, otherwise it is not available for dynamic linking.
>
> 6) Given that much of the codebase I'm integrating is in c++, does it
> make sense to develop activities in the NDK as well? Perhaps that
> would be easier because I wouldn't have to call java methods all the
> time to update the UI?
If you use the standard UI API, then you have no choice but to call Java methods
(from Java or from native code through JNI). What you could do is develop the
Controller part in native code, but you'll still have to use the Java API for
the View.
> 7) Probably a silly question, but I'm having hard time figuring out
> the whole C/C++ API that is available (I have yet to setup eclipse for
> C++ dev). What are you guys using as a reference to a C/C++ API that
> is available on Android? Currently, I'm between grepping the NDK
> header files and googling for method signatures... Is there such a
> thing as a compelte Bionic reference?
I grep too, and sometimes take a look at ndk/docs/STABLE-APIS.html
--
Olivier
Olivier Guilyardi
I was writing my answer and didn't see yours ;)
On 06/13/2011 07:18 PM, Tim Mensch wrote:
>> 4) When should (MUST?) I call DeleteGlobalRef? Is there an
>> JNI_OnUnload equivalent to JNI_OnLoad available?
>
> You really should call it if a new instance of the Java Application is
> created, with any global objects from the old instance. You could hold
> on to anything that you're SURE doesn't have a reference to an
> Application, an Activity, or a View-derived class.
>
> I'd say that you should call it in onDestroy(), but don't. onDestroy()
> has a bad habit of being called AFTER the onCreate() of the new
> instance, so there's no point in actually doing any destruction there if
> you've got statics in your JNI code. Just release everything when a new
> instance is created.
Yes, I've seen the same thing, onDestroy() being called after onCreate(), and
similar things with onPause() and onResume(). What this means is that several
instances of the same activity can "live" at the same time, and this can result
in bad problems with static variables in native code.
I personally use absolutely no native static variable, and rely on native
objects for encapsulation: I keep native pointers as Java long values in Java
wrappers. This way, several activities can live at the same time, it's safe.
IMO native statics are quite bad in general, and pure evil on Android, because
of the distinction between Activities/Services/etc.. and processes.
--
Olivier
Tim Mensch
> I personally use absolutely no native static variable, and rely on
> native objects for encapsulation: I keep native pointers as Java long
> values in Java wrappers. This way, several activities can live at the
> same time, it's safe.
>
> IMO native statics are quite bad in general, and pure evil on
> Android, because of the distinction between Activities/Services/etc..
> and processes.
Yes, I could do that, and it wouldn't be the end of the world, but I don't actually see enough of an advantage. Given that you can't play more than one copy of a game at a time, I find it's a reasonable trade-off to just give up on "pure OO" cleanliness in exchange for API cleanliness. Sure, if I wanted to have several copies of the game running in the same context, I might have issues -- but I don't think I've ever actually needed to do that in reality, and so I'm not highly motivated to make sure the design handles a feature I've never used. ;)
I guess I've gotten cynical in my old age: If I don't see an immediate benefit from adding typing to my daily coding, then I don't want to add that typing, even if it satisfies some criteria of cleanliness -- I've been there and done that, and now the pendulum is swinging back toward "do the minimal I need to ship the game."
Also, I use enough memory that if a full second instance of a game really did load, it would be (at least temporarily) twice the size with no real benefit, and you'd need to reload assets that were already in memory -- so it seems objectively useful to keep your asset management in static, persistent structures, that would even be useful across global context instances, if you choose to implement them.
And finally, if you're porting someone else's game, you simply don't have a choice in the matter.
YMMV. :)
Tim
Olivier Guilyardi
> On 6/13/2011 11:58 AM, Olivier Guilyardi wrote:
>> I personally use absolutely no native static variable, and rely on
>> native objects for encapsulation: I keep native pointers as Java long
>> values in Java wrappers. This way, several activities can live at the
>> same time, it's safe.
>>
>> IMO native statics are quite bad in general, and pure evil on
>> Android, because of the distinction between Activities/Services/etc..
>> and processes.
>
> Well, that is a very clean way to do it, from an OO perspective. In
> writing games, I find there are too many places that I need static
> access to information to want to be passing a context pointer into
> obscure corners of my library, though. So much of the game needs to have
> access to a global context that literally every function and/or class
> would need a pointer to its parent context passed in.
I understand. Actually, I'm very strict on OO encapsulation, and I especially
like to do that in pure C :)
> Yes, I could do that, and it wouldn't be the end of the world, but I
> don't actually see enough of an advantage. Given that you can't play
> more than one copy of a game at a time, I find it's a reasonable
> trade-off to just give up on "pure OO" cleanliness in exchange for API
> cleanliness. Sure, if I wanted to have several copies of the game
> running in the same context, I might have issues -- but I don't think
> I've ever actually needed to do that in reality, and so I'm not highly
> motivated to make sure the design handles a feature I've never used. ;)
In your case you "can't play more than one copy of a game at a time", ok. But I
think it's dangerous to extrapolate this to: anyway there's always one activity
running at one time.
Especially, if one tries to optimize an app for tablets using the new Fragments
API, what was activities may become parts of the UI. And proper encapsulation in
the design may be critical for this. And who knows what API/platform/spaceship
will come next? That's the thing with clean designs, it /helps/ to anticipate.
> I guess I've gotten cynical in my old age: If I don't see an immediate
> benefit from adding typing to my daily coding, then I don't want to add
> that typing, even if it satisfies some criteria of cleanliness -- I've
> been there and done that, and now the pendulum is swinging back toward
> "do the minimal I need to ship the game."
Well, I take some shortcuts too.. It depends on the goal.
The one thing is that I'm generally working on software which is meant to grow
slowly, evolve regularly, on a long term basis. I need as much maintainability
as possible. And such software or some /modules/ are often candidates for being
ported or reused. So it has be clean, modular and sort of abstract.
Anyway, I consider game development to be a true specialty, and I know near to
nothing about it. It's very different from audio software, which I focus on. In
my case, a little design mistake can become a big problem on the long run. And
well, it does. That said, the OP doesn't seem to mention a game.
> Also, I use enough memory that if a full second instance of a game
> really did load, it would be (at least temporarily) twice the size with
> no real benefit, and you'd need to reload assets that were already in
> memory -- so it seems objectively useful to keep your asset management
> in static, persistent structures, that would even be useful across
> global context instances, if you choose to implement them.
Memory's a pretty valid argument IMO. I would maybe look for other solutions to
respect OO encapsulation, but I understand the need to break certain rules when
working in an embedded environment. I somehow do that too with other things, and
it's absolutely required in my case on Android for performance reasons.
> And finally, if you're porting someone else's game, you simply don't
> have a choice in the matter.
Sure, alchemy's not an option yet :)
--
Olivier
Miha
Thank you for your answers. Deleting a reference when a new activity
spins up seems like a good idea. I guess I'll have a method like
register, where an activity can pass a reference to self and I can
then release existing one if it exists and reference a new one.
With regards to statics at all: I do need a way to call to java from a
different thread; so I need to have a reference somewhere. And global
reference is the only thing that works (I've tried passing a local
reference to a thread method, but all I got was invalid pointers).
The android-cpp-sdk looks interesting, one other thing similar to that
is also http://swig.org. I'm not yet sure what I will use - I only
need input and graphics -- no sound, sensors or other stuff.
If I understood you correctly (Oliver), it is possible that when the
use reenters the app, a NEW activity will be created, but the native
library will be the same (because it is statically loaded)? And since
library would be hanging on to a reference to the old activity, that
would prevent it from being garbace collected, hence using memory. But
if I was to release it in this case, it would be OK, right?
I think I'll try to release it on onPause (I'll try to end all
activities then) and reconnect it on onResume or onCreate. That's
probably safest and would allow GC to collect the activity when the
user is not running it (the app).
Oliver, you also write: "That said, I hope that you're not relying on
especially to hold your global references..."
do lookups, etc, one should create a global reference. How can I call
from a native thread to the java world if I don't cache a reference?
Thanks,
Miha.
Dmitry.Skiba
I suggest you to look at HelloJNIpp example of my JNIpp library:
https://github.com/DmitrySkiba/itoa-jnipp/tree/master/android/samples
It features activity implemented completely in native code, i.e. when
Java activity is created constructor for the native class is called
and when activity is collected by GC native instance is destroyed via
destructor. Native instance actually holds weak reference to the Java
object, so GC is able to collect activity object.
Dmitry
On Jun 14, 1:25 pm, Miha <miha.valen...@gmail.com> wrote:
> Hi guys!
>
> Thank you for your answers. Deleting a reference when a new activity
> spins up seems like a good idea. I guess I'll have a method like
> register, where an activity can pass a reference to self and I can
> then release existing one if it exists and reference a new one.
>
> With regards to statics at all: I do need a way to call to java from a
> different thread; so I need to have a reference somewhere. And global
> reference is the only thing that works (I've tried passing a local
> reference to a thread method, but all I got was invalid pointers).
>
> The android-cpp-sdk looks interesting, one other thing similar to that
Miha
Can someone explain what happens with weak references? When are they
garbage collected? When the application exits?
Miha
Tim Mensch
> Can someone explain what happens with weak references? When are they
> garbage collected? When the application exits?
from the garbage collector. So when all strong references are
eliminated, after that point it can be GC'ed. Before you use a weak
reference you need to check to see if it's still valid, and probably
temporarily turn it into a strong reference, though I don't know the
specific semantics around Java weak references.
Tim
Dmitry.Skiba
to the object, but doesn't prevent it from being GC'ed. You use 'get'
method to get the object, and that method returns null in case object
was collected. Weak reference don't change the rules of GC - objects
are still collected only when they are not referenced from root
objects (thread stacks and static variables).
In JNI weak references work the same though API is different
(NewWeakGlobalRef, etc.). However, non-weak JNI references (global and
local) prevent Java object from being collected even if that object is
not referenced from root objects (in contrast with non-weak references
in Java).
In JNIpp library weak references are used in situation where you need
to reference Java object from native object that is bound to that Java
object. Example: Java activity class has a pointer to the native
object (which is really an 'int' member). That native object has a
reference back to the Java object and that reference is weak when
native object is not referenced from native code (allowing Java object
to be collected) or strong (global) when native object is referenced
from native code (preventing Java object from being collected and
destroying native object).
Dmitry
Olivier Guilyardi
> With regards to statics at all: I do need a way to call to java from a
> different thread; so I need to have a reference somewhere. And global
> reference is the only thing that works (I've tried passing a local
> reference to a thread method, but all I got was invalid pointers).
I am not questioning the need for global references. You need that.
Also, I think I/we have introduced a bit of confusion, the problem isn't
strictly with /static/ variables but with global variables in native code.
JNI global references and global variables in C/C++ code are two distinct
things. You can keep a global reference in a C++ class member, in a C struct,
etc... I personally recommend that you follow OO principles, and avoid global
variables, especially for storing JNI global references.
> If I understood you correctly (Oliver), it is possible that when the
> use reenters the app, a NEW activity will be created, but the native
> library will be the same (because it is statically loaded)? And since
> library would be hanging on to a reference to the old activity, that
> would prevent it from being garbace collected, hence using memory. But
> if I was to release it in this case, it would be OK, right?
Yes you understand correctly what I meant. But the subtlety is that instance A
of an activity may be destroyed AFTER instance B of the same activity is
created. So, if your JNI global reference is stored in a native global variable,
and that you delete the JNI global reference from onDestroy(), you may both leak
the old activity, and delete the new activity reference right after creating it.
This is why Tim advises not to delete the reference in onDestroy(), but in
onCreate(), where you can test if the native global variable is already set, and
delete the reference if needed, just before you create a new reference.
As I said, I don't like this approach for several reasons. One reason is that it
uses memory while the app is idle. When the activity is destroyed it can not be
garbage collected until the user comes back to the app.
So you could use a weak reference for that, but I personally find this a bit
dangerous unless you really know what you are doing.
> I think I'll try to release it on onPause (I'll try to end all
> activities then) and reconnect it on onResume or onCreate. That's
> probably safest and would allow GC to collect the activity when the
> user is not running it (the app).
That could work.
> Oliver, you also write: "That said, I hope that you're not relying on
> static variables in native code,
> especially to hold your global references..."
>
> But there is a specific guidance on that in JNI tips that in order to
> do lookups, etc, one should create a global reference. How can I call
> from a native thread to the java world if I don't cache a reference?
Of course, you need to cache the global reference. After all, that's what global
references are usually meant for, being cached. I just warn you about global
variables. And avoiding these is rather easy, by following simple OO and
callback patterns.
For example, in onResume(), you can pass an object which implements a listener
interface to your native code and bind it to a callback. Then, in onPause(),
simply clear the callback and delete the global reference to the object. All
this can be achieved with native structs/classes, without the need for a single
global or static variable. This way, each activity instance operates in a well
encapsulated scope, and you don't need to care about conflicts at global scope.
--
Olivier
Tim Mensch
> This is why Tim advises not to delete the reference in onDestroy(),
> but in onCreate(), where you can test if the native global variable
> is already set, and delete the reference if needed, just before you
> create a new reference.
>
> As I said, I don't like this approach for several reasons. One reason
> is that it uses memory while the app is idle. When the activity is
> destroyed it can not be garbage collected until the user comes back
> to the app.
Yes it means that it still ends up sitting on a lot of memory, but users would hate for an app to take 8-10 seconds to resume after a notification came up, or after a full-screen ad was shown (at least one service I use lives in its own Activity).
I could consider releasing my JNI reference in onPause() and restoring it in onResume(), so that the JNI code doesn't need to hold on to global references; that wouldn't be a big deal. But honestly I don't think the amount of memory held by the activity is likely to be relevant; I have a couple of views, one of which being the GLSurfaceView that DOES release its memory, and the other being a container for ads to display in...if it took up even a couple hundred K, then I'd be surprised. And really releasing all the data gets me back to the long load time.
If something else really needs memory, my process will be killed for the multiple megabytes of graphics & sound effects that it's sitting on, not because of the (relatively) small Activity and its associated memory. Trying to be "friendly" to the OS can sometimes be at odds with being friendly to your users, and the latter are the ones paying the bills. :)
Tim