'SYS V semaphore like' functionality in Native Code.

[Raman Chalotra]

Hi !

I know that SYS V IPC is not supported on Android. 

I am porting some native code that uses semaphore for taking system-wide lock i.e. more than one process use this semaphore. I am not sure, but, probably POSIX shared mutex can be used to provide the similar functionality; however, even this is not supported on Android. 

Also, my existing code uses the SEM_UNDO functionality provided by the SYS V IPC. 

Could you please suggest me any alternate mechanism provided by Android that can help me here (in Native code itself), including the SEM_UNDO part.

Thanks,

Raman

[David Given]

Raman Chalotra wrote:
[...]

> Could you please suggest me any alternate mechanism provided by Android
> that can help me here (in Native code itself), including the SEM_UNDO part.

We do IPC between processes using Unix domain sockets, with abstract
socket addresses; we fork off a tiny server that listens on the socket
and forwards messages between the processes. With this approach you'd
have to either implement your own shared semaphore implementation in the
server process, but it does work. You may find it easier to change your
application logic to use message passing rather than semaphores.

Bear in mind that the Dalvik VM doesn't like fork() much, and goes into
conniptions if you try to do *any* work between the fork() and the
exec(). The code we use is as follows:

const char* argv[] = { exe, parameter, NULL };
pid_t pid = vfork();
switch (pid)
{
case -1: /* error */
break;

case 0: /* child */
{
execve(exe, (char* const*) argv, NULL);
_exit(0);
}

default: /* parent */
{
int status;
waitpid(pid, &status, 0);
break;
}
}

This seems to be robust, and appears to work on all the devices we have
access to.

The server process does some setup and then calls daemon(), putting
itself into the background. We can then wait for this with waitpid(),
which allows us to report an error if the server fails to start up.

Actually building the server executable is moderately exciting, using
the 'include $(BUILD_EXECUTABLE)' option in Android.mk (but also be
aware that the MIPS toolchain has bugs and requires special compilation
flags or it produces broken code). We then rename it to a filename like
server.so and put it in the application's lib directory, so that when
the application is installed the server is copied into the real
filesystem on the Android device and becomes runnable.

--
┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ───── http://www.cowlark.com ─────
│ "Parents let children ride bicycles on the street. But parents do not
│ allow children to hear vulgar words. Therefore we can deduce that
│ cursing is more dangerous than being hit by a car." --- Scott Adams

[Dianne Hackborn]

Let me please re-iterate that applications written with the NDK really, really should not be using fork.  This does not fit in the application model for third party applications.  These processes are not managed by the platform.  What will happen to such processes is completely not defined.  Don't be surprised if at some point the platform starts getting pretty aggressive about cleaning up such processes.

--
Dianne Hackborn
Android framework engineer
hac...@android.com

Note: please don't send private questions to me, as I don't have time to provide private support, and so won't reply to such e-mails.  All such questions should be posted on public forums, where I and others can see and answer them.

[Raman Chalotra]

Hi!

Exactly, calling fork is bad in a 3rd party application. As a matter
of fact, the code that I am working is a library and not an
application.

Doing some more analysis, I found that the libc.so on the emulator
does contain the posix semaphore ( 'sem_open' etc.l). However, I
could not find the documentation saying that this is part of the
standard ABI of Android. Any one having any idea about this ?

-Raman

> hack...@android.com

[David Turner]

On Thu, Apr 12, 2012 at 11:22 AM, Raman Chalotra <ramanc...@gmail.com> wrote:

Hi!

Exactly, calling fork is bad in a 3rd party application. As a matter
of fact, the code that I am working is a library and not an
application.

Doing some more analysis, I found that the libc.so on the emulator
does contain the posix semaphore ( 'sem_open'  etc.l). However, I
could not find the documentation saying that this is part of the
standard ABI of Android. Any one having any idea about this ?

sem_open() will return ENOSYS, don't count on it. Seriously, don't count on the platform to ever implement these features. They're broken by design.

Search for SYS-IPC.TXT for details.

 

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To post to this group, send email to andro...@googlegroups.com.
To unsubscribe from this group, send email to android-ndk...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/android-ndk?hl=en.

[David Given]

On 12/04/12 02:01, Dianne Hackborn wrote:
> Let me please re-iterate that applications written with the NDK really,
> really should not be using fork. This does not fit in the application
> model for third party applications. These processes are not managed by
> the platform. What will happen to such processes is completely not
> defined. Don't be surprised if at some point the platform starts
> getting pretty aggressive about cleaning up such processes.

Unfortunately we frequently don't have any choice --- when porting third
party code, rewriting it to be more Android-centric simply isn't an
option. (The library I'm currently working with, for example, took about
a hundred man-years to write.)

Given the commercial realities of actually having to get a product out,
then, we just have to use fork().

(I'd much rather use posix_spawn() if that were available...)

--
┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ───── http://www.cowlark.com ─────
│

│ "Never attribute to malice what can be adequately explained by
│ stupidity." --- Nick Diamos (Hanlon's Razor)

[Tor Lillqvist]

Given the commercial realities of actually having to get a product out,
then, we just have to use fork().

Then you have to live with the commercial reality of your product breaking when Google or OEMs update Android so that it starts actively killing off processes that shouldn't be there.

--tml
 

[Raman Chalotra]

Sorry, but I could not understand why to refer SYS-IPC.txt for
'sem_open'. SYS-IPC.txt is all about System V semaphores. However,
sem_open is POSIX version of the semaphore. Is there anything that I
am missing?

-Raman

On Apr 12, 2:36 pm, David Turner <di...@android.com> wrote:

[David Turner]

On Tue, Apr 17, 2012 at 7:02 PM, Raman Chalotra <ramanc...@gmail.com> wrote:

Sorry, but I could not understand why to refer SYS-IPC.txt for
'sem_open'. SYS-IPC.txt is all about System V semaphores. However,
sem_open is POSIX version of the semaphore.  Is there anything that I
am missing?

sem_open() has the same problem of requiring a system-global resource that can be really easily exhausted, requiring a reboot to cleanup.

If you look at the sem_open() implementation in GLibc you'll see that it depends on /dev/shm

[Chris Stratton]

On Thursday, April 12, 2012 7:57:52 PM UTC-4, David Given wrote:

Unfortunately we frequently don't have any choice --- when porting third
party code, rewriting it to be more Android-centric simply isn't an
option.

You might think a bit about why you absolutely need a separate process, and in particular why that has to be a clean-slate process that calls exec rather than a zygote-Dalvik one which dlopens an .so.

You actually can get Android to properly give you an independent, platform-tracked process by creating an Android service designated to run in a separate process; it will have the dead weight of it's own DVM (it's not quite all shared zygote inheritance), but you don't really have to do very much on the Java side, vs. the native code that you build as a library and immediately call into, create a thread, and within the thread call whatever you renamed main() to.  Really all you'd need the DVM for on an ongoing basis is to accept housekeeping calls from the platform.  Those notifications probably sort into "ignore" and "save state in preparation for shutdown".

It is kind of too bad though that there isn't a way to have a fully-android-proper bare native process.