Native looper and custom messages

[Athos Bacchiocchi]

Hi all,

I need to know whether it is possible to use native ALooper to process custom messages and without using native activity. I saw how to use it to get sensor data and deal with input events, but there's no indication for any other possibility, except in a book about ndk, where I found this sentence:

"We have not talked about it yet but a third queue, the user queue, can be attached to the
looper. This queue is a custom one, unused by default and which can be used for your own
purpose. More generally, your application can use the same ALooper to listen to additional
file-descriptors." (android NDK beginner's guide - S. Ratabouil - p. 169)

But I found no other mention of this "user queue" anywhere else (neither in the book itself).

My messages are generated in native threads, and they need to be put in a queue so they can be processed in the "main" thread. Calling ALooper_forThread() I can get a reference to the looper for the main thread, but I don't know what to do with it.

The alternative, I guess, is to use a handler on the java side, sending and processing messages doing back and forth between native and java with jni calls. But I wanted to be sure there's no way to do the same thing staying on the native side.

thanks,
athos

[Dianne Hackborn]

You create a pipe, add the read side as another fd the looper will wait on, and read from it when you find out there is data.  You can decide whatever protocol you want for that data.  For example native_app_glue writes and reads single bytes that are command constants.  You could make a big data structure representing an event if you want something more complicated.  (Though of course things like variable-length strings can be complicated...  if you are careful about memory management, you could possibly pass them through as pointers.)

Almost all of native_app_glue is about implementing an event loop for sending commands from the main thread to the app thread it is creating.  Just turn that around and do the same thing but on the ALooper of the main thread.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To post to this group, send email to andro...@googlegroups.com.
To unsubscribe from this group, send email to android-ndk...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/android-ndk?hl=en.

--
Dianne Hackborn
Android framework engineer
hac...@android.com

Note: please don't send private questions to me, as I don't have time to provide private support, and so won't reply to such e-mails.  All such questions should be posted on public forums, where I and others can see and answer them.

[David Given]

On 21/06/12 17:35, Athos Bacchiocchi wrote:
[...]

> My messages are generated in native threads, and they need to be put in
> a queue so they can be processed in the "main" thread. Calling
> ALooper_forThread() I can get a reference to the looper for the main
> thread, but I don't know what to do with it.

Looper is a bit weird if you're used to traditional event queues, but
once you get your head around it the design's quite elegant.

What you do is create a fd pair with pipe(). Tell looper to listen to
one end. When you want to send a message from your engine thread, write
it to the pipe with write(). This will get buffered internally (pipes
get a 64kB buffer), and if the buffer is full your write will block.
Looper will then call a callback you specify in your main thread. You
can read a message with read().

It's important to note that you can treat the pipe as a message-based
system or a byte stream. For the former, make sure your messages are
shorter than PIPE_BUF and they'll get written atomically. It's wise to
make sure that you *read* them atomically as well, otherwise your
message boundaries can get out of sync and then you go mad. Fixed size
messages, or at least fixed size messages each followed by a variable
sized payload, are the way to go here.

For a byte stream, write any amount --- but you may get a short write,
and then it's your responsibility to send the rest. Plus, you may get
called back by Looper multiple times; I don't think Looper likes it if
you block during a read.

It's actually quite simple, very fast, and extremely flexible. I don't
have any code handy right now but if you like I can try and dig some out
tomorrow.

Essential reading:

http://linux.die.net/man/7/pipe

--
┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ───── http://www.cowlark.com ─────
│ "...thus there might be a great reduction in useless expenditure on
│ Nuclear rockets, reducing inflation and stabilising the price of cat
│ foods." --- UK pat. GB1426698

[Athos Bacchiocchi]

Il 22/06/2012 00:15, David Given ha scritto:
> Looper is a bit weird if you're used to traditional event queues, but
> once you get your head around it the design's quite elegant.

I'm actually quite new to event queues in general. I know the ms windows
event loop as a "user", and the code I'm porting is originally written
for windows, so i have to reproduce a part of this mechanism.

> It's important to note that you can treat the pipe as a message-based
> system or a byte stream. For the former, make sure your messages are
> shorter than PIPE_BUF and they'll get written atomically.

My messages are made of a message id, an handle to a window and two int
parameters (this will sound very familiar to windows programmers...), so
I think I can make them fixed-sized.

> I don't have any code handy right now but if you like I can try and
> dig some out tomorrow.

It would be really useful and much appreciated, in the meanwhile I will
study the references you and Dianne gave me and try to come up with
something working. Thank you both for the hints, they really helped me.

athos

[David Given]

Athos Bacchiocchi wrote:
[...]

> It would be really useful and much appreciated, in the meanwhile I will
> study the references you and Dianne gave me and try to come up with
> something working. Thank you both for the hints, they really helped me.

This is cut-and-pasted and bodged a bit, so there may be typos; and
there may also be subtle bugs in the logic. All I can say is that it
works for me...

struct Message { ... };

static int messagePipe[2];
static const int LOOPER_ID_MESSAGEPIPE = LOOPER_ID_USER;

/* Call from engine thread */
void post_message(Message* msg)
{
if (write(messagePipe[1], msg, sizeof(msg)) != sizeof(msg))
{
/* This may never happen if the writing end of the
* pipe is blocking. If it's non-blocking, then if
* the buffer is full the write will fail. */
LOGI("panic! callback buffer overrun!");
}
}

/* Invoked by ALooper to process a message */
static int messagepipe_cb(int fd, int events, void* user)
{
Message msg;

while (read(fd, &msg, sizeof(msg)) == sizeof(msg))
{
/* Do whatever needs doing when a message is
* received. Provided sizeof(msg) <= PIPE_BUF
* you should never get short reads. */
HandleMessage(msg);
}

return 1;
}

/* Call this from your main thread to set up the callback pipe. */
void setup_callback_pipe(void)
{
int i = pipe2(messagePipe, O_NONBLOCK | O_CLOEXEC);
assert(i != -1); /* I can't imagine why this would ever fail */

/* You may want to make messagePipe[1] blocking here,
* if you want the engine thread to block if the buffer
* fills up (rather than failing to send the message).
* I've never done this. YMMV. */

/* Register the file descriptor to listen on. */
ALooper_addFd(appState->looper,
messagePipe[0], LOOPER_ID_MESSAGEPIPE,
ALOOPER_EVENT_INPUT, messagepipe_cb, NULL);

}

--
┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ───── http://www.cowlark.com ─────

│ "Parents let children ride bicycles on the street. But parents do not
│ allow children to hear vulgar words. Therefore we can deduce that
│ cursing is more dangerous than being hit by a car." --- Scott Adams

[Dianne Hackborn]

Fwiw, the API wasn't really designed to provide an event queue itself, but to be a lower-level primitive that you could use to build whatever event model you wanted on top (or multiplex data streams or whatever).  Some standard helper code implementing a traditional event model would be good to have for people to build on.

[Athos Bacchiocchi]

Il 22/06/2012 11:50, David Given ha scritto:
> This is cut-and-pasted and bodged a bit, so there may be typos; and
> there may also be subtle bugs in the logic. All I can say is that it
> works for me...

Thanks, you all have been very helpful. I could setup my message queue
using David's suggested approach, and digging in
android_native_app_glue.c helped me understand better. I'm using a mutex
in the "message posting" function, to stay safe in case of concurrent
attempts to write from different threads, even though I read that writes
of less than PIPE_BUF bytes are atomic, so in my case it may be ok
without mutexes as well.

Also, I read in looper.h that the "ident" argument in
ALooper_addFd(...) call is ignored when a callback is specified (as I
do). So I guess it's not essential that I specify LOOPER_ID_USER (so I
don't even need to include android_native_app_glue.h). Is that correct?

athos