mmap and synchronization
Olivier Guilyardi
I have started to use mmap() for high performance data sharing between an
activity and a service.
It works great, but I'm now looking for (simple lock) synchronization options,
and have found the following:
1 - named semaphores
2 - unnamed semaphore stored in the mmap'ed region
3 - pthread mutex stored in the mmap'ed region
I don't like option 1. because it apparently needs some kind of garbage
collection, and I'm worried about the state of such named semaphores in case of
crash.
Googl'ing around, I found that unnamed shared semaphores, as in 2., do not exist
in the posix spec. Are they supported on Linux/Android?
About 3., are shared mutexes supported? Must I set them up in a special way, any
attribute?
Any other lock/synchronization option?
Regards,
--
Olivier
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To post to this group, send email to andro...@googlegroups.com.
To unsubscribe from this group, send email to android-ndk...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/android-ndk?hl=en.
Dmitry Moskalchuk
Actually POSIX support unnamed semaphores - using sem_init function you
can create them. Look to the
<ndk>/build/platforms/android-3/arch-arm/usr/include/semaphore.h.
One note: if you plan to use semaphores in shared memory, you must
explicitly specify it in sem_init call:
sem_init(&sem, 1, 0);
Dmitry Moskalchuk
Olivier Guilyardi
okay, if I can put a semaphore in the mmap region that's perfect. I'll call
sem_init() on the server side only then I suppose.
But what if the server or client crash before unlocking the semaphore?
The semaphore will be destroyed whenever the server process dies, right?
Or will the semaphore stay locked because its value is kept in the underlying
mmap'ed file?
Olivier
Dmitry Moskalchuk
behavior.
If you use mmap for shared memory (mapping actual file to memory),
semaphore value will persist in this file after process shutdown
(graceful or abort - no matter). But unnamed semaphores are not kernel
objects so you will not leak kernel resources on crash. So after process
restart you could zero-initialize memory region related to semaphore and
call sem_init again - this will create new semaphore with new initial value.
Dmitry Moskalchuk
Dmitry Moskalchuk
supported in Android. Looking to the
http://android.git.kernel.org/?p=platform/bionic.git;a=blob;f=libc/bionic/semaphore.c;h=0c94600b731bc644321cab157687d00f6bd16934;hb=HEAD,
I've found sem_init implementation:
34 int sem_init(sem_t *sem, int pshared, unsigned int value)
35 {
36 if (sem == NULL) {
37 errno = EINVAL;
38 return -1;
39 }
40
41 if (pshared != 0) {
42 errno = ENOSYS;
43 return -1;
44 }
45
46 sem->count = value;
47 return 0;
48 }
That's it.
Dmitry Moskalchuk
Olivier Guilyardi
Olivier
Olivier Guilyardi
Then, what about the mutex thing?
Olivier
Dmitry Moskalchuk
memory. Just don't call them with pshared parameter set to non-zero.
Shared memory is not some very unusual - it is regular memory available
in your program so you can put your unnamed semaphore there and it
should work as I described before. Just call sem_init as below:
sem_init(&sem, 0, initial_count);
Dmitry Moskalchuk
Olivier Guilyardi
Olivier Guilyardi
Okay, I can confirm that using a shared semaphore located in a mmap'ed region works.
Thanks
Olivier
David Turner
Hey!
Okay, I can confirm that using a shared semaphore located in a mmap'ed region works.
Thanks
Olivier
Olivier Guilyardi
>
>
> On Tue, Jun 1, 2010 at 11:54 AM, Olivier Guilyardi <li...@samalyse.com
> <mailto:li...@samalyse.com>> wrote:
>
> Hey!
>
> Okay, I can confirm that using a shared semaphore located in a
> mmap'ed region works.
>
>
> Important note: this works in Eclair, it won't work in Froyo unless you
> explicitely use PTHREAD_PROCESS_SHARED.
I'm using a semaphore not a pthread mutex. And it does work as-is on Froyo in
the emulator. But is there something wrong with using a semaphore?
Also, if I set the pshared argument of sem_init() to 1, this function returns an
error (see Dmitry's remark about that in this thread).
> Also, no robust futex support, which means your code should deal with
> the fact that one process can die while holding the lock.
Not a problem in my case. The semaphore I'm currently working with isn't a lock,
but a counter for data packets sent through a lock-free ringbuffer.
Olivier
> <mailto:andro...@googlegroups.com>.
> To unsubscribe from this group, send email to
> android-ndk...@googlegroups.com
> <mailto:android-ndk%2Bunsu...@googlegroups.com>.
David Turner
On 06/01/2010 10:05 PM, David Turner wrote:
>
>
> On Tue, Jun 1, 2010 at 11:54 AM, Olivier Guilyardi <li...@samalyse.com
> <mailto:li...@samalyse.com>> wrote:I'm using a semaphore not a pthread mutex. And it does work as-is on Froyo in
>
> Hey!
>
> Okay, I can confirm that using a shared semaphore located in a
> mmap'ed region works.
>
>
> Important note: this works in Eclair, it won't work in Froyo unless you
> explicitely use PTHREAD_PROCESS_SHARED.
the emulator. But is there something wrong with using a semaphore?
Also, if I set the pshared argument of sem_init() to 1, this function returns an
error (see Dmitry's remark about that in this thread).
> Also, no robust futex support, which means your code should deal withNot a problem in my case. The semaphore I'm currently working with isn't a lock,
> the fact that one process can die while holding the lock.
but a counter for data packets sent through a lock-free ringbuffer.
Angus Lees
> The same change was done for condition variables
> (pthread_condattr_init/destroy/getpshared/setpshared were also added).
Aside: I noticed recently that these pthread_condattr_* functions were
missing from the r4 NDK libc. I ended up patching my headers with
inline stubs - I hope they weren't important ;)
- Gus
Dianne Hackborn
--
Dianne Hackborn
Android framework engineer
hac...@android.com
Note: please don't send private questions to me, as I don't have time to provide private support, and so won't reply to such e-mails. All such questions should be posted on public forums, where I and others can see and answer them.
Olivier Guilyardi
About (1), I need the highest IPC performance I can get, and the normal IBinder
things and such just won't work in my case. This is about TapeMachine
(http://tapemachine.samalyse.com/), which features a *live* waveform, which,
among other things, means that it is updated while recording. So the service
which is responsible for recording updates the signal peaks in shared memory,
and the GL thread on the activity side draws the waveform from that.
And the reason for running the audio engine in a service is simple: many users
have requested background recording.
I have already used the Binder things a lot in this app and another one. And
TapeMachine 1.3.2 which is currently on the market uses it a lot. But it's just
*too slow* and heavy in this case. I don't want to go into the details right
now, there's too many of them.
(2) When I realized this, I said to myself: okay, these devices are slow, let's
use the fastest thing I know, thus living aside sockets and pipes. And the
fastest thing that I know is lock-free ringbuffers, which are widely used in the
audio world, and safe for IPC as long as there is a single reader and a single
writer. But using this, I need a way to signal the other thread/process when new
messages arrives, hence the need for a semaphore.
And it apparently runs smoothly according to my first tests.
Do these reasons seem valid to you?
Olivier
On 06/02/2010 04:50 AM, Dianne Hackborn wrote:
> Hi,
>
> A couple comments:
>
> (1) I want to be sure you really need to do this. There is no need to
> use shared memory for interaction between an activity and a service.
> You only need to do this if you have for some reason force those two
> things to run in separate processes... and unless they are implemented
> in separate .apks, there are very few reasons to do that.
>
> (2) For the situation you are describing, why not use a pipe? It is
> extremely easy to send an fd across processes using the service's
> interace (which you are obviously already doing for your shared memory
> area), and there is nothing subtle about whether or not this will work
> across processes.
>
> On Tue, Jun 1, 2010 at 1:28 PM, Olivier Guilyardi <li...@samalyse.com
> <mailto:li...@samalyse.com>> wrote:
>
> On 06/01/2010 10:05 PM, David Turner wrote:
> >
> >
> > On Tue, Jun 1, 2010 at 11:54 AM, Olivier Guilyardi
> <li...@samalyse.com <mailto:li...@samalyse.com>
> > <mailto:andro...@googlegroups.com
> <mailto:andro...@googlegroups.com>>.
> > To unsubscribe from this group, send email to
> > android-ndk...@googlegroups.com
> <mailto:android-ndk%2Bunsu...@googlegroups.com>
> > <mailto:android-ndk%2Bunsu...@googlegroups.com
> <mailto:android-ndk%252Buns...@googlegroups.com>>.
> > For more options, visit this group at
> > http://groups.google.com/group/android-ndk?hl=en.
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "android-ndk" group.
> > To post to this group, send email to andro...@googlegroups.com
> <mailto:andro...@googlegroups.com>.
> > To unsubscribe from this group, send email to
> > android-ndk...@googlegroups.com
> <mailto:android-ndk%2Bunsu...@googlegroups.com>.
> > For more options, visit this group at
> > http://groups.google.com/group/android-ndk?hl=en.
>
> --
> You received this message because you are subscribed to the Google
> Groups "android-ndk" group.
> To post to this group, send email to andro...@googlegroups.com
> <mailto:andro...@googlegroups.com>.
> To unsubscribe from this group, send email to
> android-ndk...@googlegroups.com
> <mailto:android-ndk%2Bunsu...@googlegroups.com>.
> For more options, visit this group at
> http://groups.google.com/group/android-ndk?hl=en.
>
>
>
>
> --
> Dianne Hackborn
> Android framework engineer
> hac...@android.com <mailto:hac...@android.com>
>
> Note: please don't send private questions to me, as I don't have time to
> provide private support, and so won't reply to such e-mails. All such
> questions should be posted on public forums, where I and others can see
> and answer them.
>
Olivier Guilyardi
[...]
> For froyo, the pthread_mutex_t implementation was changed to use
> "private futexes", which are faster variants
> of futexes that cannot be shared between processes. Normal futexes are
> only used if you use PTHREAD_PROCESS_SHARED.
Okay, good to know.
> The same change was done for condition variables
> (pthread_condattr_init/destroy/getpshared/setpshared were also added).
>
> But semaphores were unchanged. They still use normal futexes and just
> happen to work in shared memory by accident
> (just like the previous mutex/cond implementation).
Is there /some kind/ of guarantee that it won't stop to work /by accident/?
Would it be safer for me to roll my own semaphore using pthread mutexes and
condition variables?
> When we say that we don't support PTHREAD_PROCESS_SHARED, it really
> means you should not expect the current
> behaviour to be cast in stone and not change in the future. Until we say
> we support it.
So, if I want to target android-3, I can use PTHREAD_PROCESS_SHARED, which will
be silently ignored on Froyo, but is needed on the later, right?
> > Also, no robust futex support, which means your code should deal with
> > the fact that one process can die while holding the lock.
>
> Not a problem in my case. The semaphore I'm currently working with
> isn't a lock,
> but a counter for data packets sent through a lock-free ringbuffer.
>
> Sounds good to me then :)
--
Olivier
Olivier Guilyardi
> On 06/01/2010 11:57 PM, David Turner wrote:
>
> [...]
>> For froyo, the pthread_mutex_t implementation was changed to use
>> "private futexes", which are faster variants
>> of futexes that cannot be shared between processes. Normal futexes are
>> only used if you use PTHREAD_PROCESS_SHARED.
>
> Okay, good to know.
One more question: I did notice that semaphores and pthread mutexes are slow. I
even ended up timing them and logging slow locks.
If I understand correctly, private futexes should bring some more performance.
So, would it be okay for me to use futex() directly, especially to speed up
those of my locks which are not shared across processes?
I see that futex() is defined on android-3, so that should work no?
Maybe that I could even reuse the new Froyo pthread mutex code...
David Turner
On 06/01/2010 11:57 PM, David Turner wrote:
[...]
> For froyo, the pthread_mutex_t implementation was changed to useOkay, good to know.
> "private futexes", which are faster variants
> of futexes that cannot be shared between processes. Normal futexes are
> only used if you use PTHREAD_PROCESS_SHARED.
Is there /some kind/ of guarantee that it won't stop to work /by accident/?
> The same change was done for condition variables
> (pthread_condattr_init/destroy/getpshared/setpshared were also added).
>
> But semaphores were unchanged. They still use normal futexes and just
> happen to work in shared memory by accident
> (just like the previous mutex/cond implementation).
Would it be safer for me to roll my own semaphore using pthread mutexes and
condition variables?
> When we say that we don't support PTHREAD_PROCESS_SHARED, it reallySo, if I want to target android-3, I can use PTHREAD_PROCESS_SHARED, which will
> means you should not expect the current
> behaviour to be cast in stone and not change in the future. Until we say
> we support it.
be silently ignored on Froyo, but is needed on the later, right?
> > Also, no robust futex support, which means your code should deal with
> > the fact that one process can die while holding the lock.
>
> Not a problem in my case. The semaphore I'm currently working with
> isn't a lock,
> but a counter for data packets sent through a lock-free ringbuffer.
>
> Sounds good to me then :)
--
Olivier
David Turner
Dianne Hackborn
About (1), I need the highest IPC performance I can get, and the normal IBinder
things and such just won't work in my case. This is about TapeMachine
(http://tapemachine.samalyse.com/), which features a *live* waveform, which,
among other things, means that it is updated while recording. So the service
which is responsible for recording updates the signal peaks in shared memory,
and the GL thread on the activity side draws the waveform from that.
And the reason for running the audio engine in a service is simple: many users
have requested background recording.
I have already used the Binder things a lot in this app and another one. And
TapeMachine 1.3.2 which is currently on the market uses it a lot. But it's just
*too slow* and heavy in this case. I don't want to go into the details right
now, there's too many of them.
(2) When I realized this, I said to myself: okay, these devices are slow, let's
use the fastest thing I know, thus living aside sockets and pipes. And the
fastest thing that I know is lock-free ringbuffers, which are widely used in the
audio world, and safe for IPC as long as there is a single reader and a single
writer. But using this, I need a way to signal the other thread/process when new
messages arrives, hence the need for a semaphore.
Dianne Hackborn
One more question: I did notice that semaphores and pthread mutexes are slow. Ieven ended up timing them and logging slow locks.
If I understand correctly, private futexes should bring some more performance.
So, would it be okay for me to use futex() directly, especially to speed up
those of my locks which are not shared across processes?
I see that futex() is defined on android-3, so that should work no?
Maybe that I could even reuse the new Froyo pthread mutex code...
Olivier Guilyardi
Hmm, well, I'm not saying that I understand it all, indeed. I must have missed
this very sentence:
"A Service is not a separate process. The Service object itself does not imply
it is running in its own process; unless otherwise specified, it runs in the
same process as the application it is part of."
But my Service *do* appear to run in a separate process.
Taking a close look, it seems like I have blindly copied the following when
looking for what to put in the manifest (taken from the page you link):
<service android:name=".app.MessengerService"
android:process=":remote" />
Hmm... But the Binder interface just looked like an IPC system. At some point in
time, I must have engraved in my mind that a Service is another process, because
that was so natural actually, thinking in terms of "normal" systems.
That said, moving the audio engine into a Service allowed me to workaround a
Donut issue which I banged my head against for weeks, see the "Concurrent file
access latency on Android 1.6" thread on this list. I need to check if using a
single process still works for this.
> I have already used the Binder things a lot in this app and another
> one. And
> TapeMachine 1.3.2 which is currently on the market uses it a lot.
> But it's just
> *too slow* and heavy in this case. I don't want to go into the
> details right
> now, there's too many of them.
>
>
> Hm, okay, I am a little suspicious of this, since a binder IPC takes 1ms
> on a slow device (and significantly less on a fast one), so if you can
> design your protocol to do one ipc per frame you should have no problem
> getting all of the points of the waveform to the app at 60 fps with
> plenty of cpu to spare.
Actually, this paragraph wasn't about the waveform anymore. I couldn't use
binder for that anyway with my current engine design. And I highly doubt you
could deal with the waveform through the binder anyway.
This paragraph was about sending commands to the audio engine. For example, the
waveform can be scrolled/scratched, and thus seeking must be performed very
fast, otherwise it feels sluggish. And it does feel better with ringbuffer approach.
The other problem with binder is that when receiving messages (such as position
update, etc...) through (binder) callbacks from the audio engine, the callbacks
are not called in the activity thread, thus I need to *allocate* and post
Runnables into the main Handler.
And I just can't allocate anything during recording or playback. When GC runs,
it may cause the media layer to freeze and thus hearable dropouts. Position,
input gain, cpu usage, etc.. updates are too frequent, that's too many allocs.
Plus I find the RemoteCallbackList usage terrible. Trust me or not, my new pure
C IPC design is much shorter and elegant, and it's faster.
> (2) When I realized this, I said to myself: okay, these devices are
> slow, let's
> use the fastest thing I know, thus living aside sockets and pipes.
> And the
> fastest thing that I know is lock-free ringbuffers, which are widely
> used in the
> audio world, and safe for IPC as long as there is a single reader
> and a single
> writer. But using this, I need a way to signal the other
> thread/process when new
> messages arrives, hence the need for a semaphore.
>
>
> The fastest thing is to run in the same process and not have to deal
> with a lot of this. :)
Possibly, however last but not least, I like C much better than Java, and that
could explain some of my choices ;)
Single process or not, I believe that much of my new code won't need to change,
since I'm dealing with multiple threads anyway. But if I can avoid the shared
semaphore and stuff, that may be better indeed.
Thank you for shredding some light on these matters :)
--
Olivier
Olivier Guilyardi
> On Wed, Jun 2, 2010 at 4:53 AM, Olivier Guilyardi <li...@samalyse.com
> <mailto:li...@samalyse.com>> wrote:
>
> One more question: I did notice that semaphores and pthread mutexes
> are slow. I
> even ended up timing them and logging slow locks.
>
>
> What is "slow"? If you are trying to achieve 60fps, all of these things
> are much much faster than you need at all to get updates at that rate.
Look, there's much more happening in TapeMachine than a 60fps opengl animation.
It's dealing with audio, file access, not to mention live encoding and stuff.
There are locks here and there because the audio engine is multi-threaded. What
I call a slow lock is when sem_wait() takes more than 1ms where the semaphore
only acts as a simple lock around a given variable.
> If I understand correctly, private futexes should bring some more
> performance.
> So, would it be okay for me to use futex() directly, especially to
> speed up
> those of my locks which are not shared across processes?
>
> I see that futex() is defined on android-3, so that should work no?
> Maybe that I could even reuse the new Froyo pthread mutex code...
>
>
> Honestly, I think it might be worth taking a step back and looking at
> what is driving these requests. You are forcing yourself down a path
> that is putting a huge onus on you to implement code that is going to
> work correctly on a wide (and ever expanding) variety of hardware and
> software that can have subtle changes in behavior that impact what you
> are doing. Any of the safe synchronization and communication primitives
> can be used to easily achieve a 60fps framerate... what is driving this
> requirement to have the ultimately fast mechanism? The only thing I can
> imagine is that you are trying to stream the entire sample data to the
> front-end UI; if that is actually the case, what about doing a little
> processing in the audio side to generate less data at 60fps that is
> shown by the UI?
Of course I'm not streaming the entire sample data to the UI, don't take me for
a fool. My waveform system is smart and efficient IMO, and the UI only accesses
a shared memory segment where reside precomputed peaks for fixed blocks of
frames. I need speed because a lot of things happen in this app, not just 60fps
opengl. Also, when I started coding this app, I directly took the highly
optimized way, design choice, that's it.
Now, for us to have a real discussion about my implementation, you would need to
have access to the source code, and that is not the case. This isn't FLOSS
sorry. Plus, this is my code and my choices after all :p
Where I strongly agree with you is about supporting various hardware (and
Android versions). To me the expression "public API" is a bit of a holy one. I
don't want code that "just works" if I get no guarantee about forward
compatibility.
And that's why I come here and ask about it. Anyway, don't worry I got the point
about futex, and I won't ever use them directly :)
--
Olivier
Olivier Guilyardi
>
> On Wed, Jun 2, 2010 at 8:11 AM, David Turner <di...@android.com
> <mailto:di...@android.com>> wrote:
>
> On Wed, Jun 2, 2010 at 3:48 AM, Olivier Guilyardi <li...@samalyse.com
> <mailto:li...@samalyse.com>> wrote:
>
> On 06/01/2010 11:57 PM, David Turner wrote:
>
[...]
> > But semaphores were unchanged. They still use normal futexes
Okay, got that. Anyway I've never used futexes, and it seems very tricky..
That said, is this Froyo code or am I looking in the wrong place?
http://android.git.kernel.org/?p=platform/bionic.git;a=blob;f=libc/bionic/pthread.c;h=7d4056d2556bcb0cb9591699f288999086b58b50;hb=HEAD
/* process-shared mutexes are not supported at the moment */
int pthread_mutexattr_setpshared(pthread_mutexattr_t *attr, int pshared)
{
if (!attr)
return EINVAL;
switch (pshared) {
case PTHREAD_PROCESS_PRIVATE:
case PTHREAD_PROCESS_SHARED:
/* our current implementation of pthread actually supports shared
* mutexes but won't cleanup if a process dies with the mutex held.
* Nevertheless, it's better than nothing. Shared mutexes are used
* by surfaceflinger and audioflinger.
*/
return 0;
}
return ENOTSUP;
}
> Implementing your own semaphore around mutex/cond variables should be ok
> as long as you understand the implications regarding PTHREAD_PROCESS_SHARED
> and the behaviour of the C library (or more exactly, various versions of
> the C library).
That looks like a good option. Otherwise my shared semaphore is going to break
when you refactor the implementation.
Basically, I could simply call the following and ignore its return value:
pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
Then the mutex would be shared, Froyo or not, right?
> > When we say that we don't support PTHREAD_PROCESS_SHARED, it
> really
> > means you should not expect the current
> > behaviour to be cast in stone and not change in the future.
> Until we say
> > we support it.
>
> So, if I want to target android-3, I can use
> PTHREAD_PROCESS_SHARED, which will
> be silently ignored on Froyo, but is needed on the later, right?
>
>
> Actually, on android-3, using PTHREAD_PROCESS_SHARED will return an
> error
> when initializing a mutex or semaphore, as far as I understand, the
> default implementation
> will be shareable (by accident).
>
> As you can see, when we say that some things are not supported, we
> really mean
> "do not use it, might break in the future".
I see, somehow got that.
--
Olivier
Dianne Hackborn
This paragraph was about sending commands to the audio engine. For example, thewaveform can be scrolled/scratched, and thus seeking must be performed very
fast, otherwise it feels sluggish. And it does feel better with ringbuffer approach.
The other problem with binder is that when receiving messages (such as position
update, etc...) through (binder) callbacks from the audio engine, the callbacks
are not called in the activity thread, thus I need to *allocate* and post
Runnables into the main Handler.
Plus I find the RemoteCallbackList usage terrible. Trust me or not, my new pure
C IPC design is much shorter and elegant, and it's faster.
Olivier Guilyardi
> On Wed, Jun 2, 2010 at 11:42 AM, Olivier Guilyardi <li...@samalyse.com
> <mailto:li...@samalyse.com>> wrote:
>
> This paragraph was about sending commands to the audio engine. For
> example, the
> waveform can be scrolled/scratched, and thus seeking must be
> performed very
> fast, otherwise it feels sluggish. And it does feel better with
> ringbuffer approach.
>
>
> For what it's worth, the events you are getting from the framework are
> all currently being delivered via Binder IPCs.
I'm not saying that it is so slow. But if I can provide slightly smoothing
scrolling, my users will be happy about it. I actually got some user feedback
about this. Binder is great and *I do use it*, but I prefer to go native as soon
as possible when something critical must be performed.
> The other problem with binder is that when receiving messages (such
> as position
> update, etc...) through (binder) callbacks from the audio engine,
> the callbacks
> are not called in the activity thread, thus I need to *allocate* and
> post
> Runnables into the main Handler.
>
>
> You definitely do want to avoid allocations, and that can be
> accomplished. Message objects are cached, so Message.obtain() will not
> allocate. If you need to use Runnable, you can make your own cache of
> Runnable objects to be able to re-use them.
Okay, I'm not very well initiated with such caching.
> Plus I find the RemoteCallbackList usage terrible. Trust me or not,
> my new pure
> C IPC design is much shorter and elegant, and it's faster.
>
>
> If your service is not forced into another process, you don't need to
> use RemoteCallbackList; it's a local object, so you can just do direct
> method calls.
This isn't the way the docs describe it. I suppose this is because when dealing
with a service, the docs can't assume that it is running in the same process.
Anyway, all of this is now very simple. In my other mail, I mentioned this:
"That said, moving the audio engine into a Service allowed me to workaround a
Donut issue which I banged my head against for weeks, see the "Concurrent file
access latency on Android 1.6" thread on this list. I need to check if using a
single process still works for this."
My tests show that the Donut workaround doesn't work anymore with a single
process. This is of first importance. I have spent a real lot of time on this
oddity. So I'm apparently going to stick with the separate process.
--
Olivier
David Turner
On 06/02/2010 05:13 PM, David Turner wrote:Okay, got that. Anyway I've never used futexes, and it seems very tricky..
>
>
That said, is this Froyo code or am I looking in the wrong place?
http://android.git.kernel.org/?p=platform/bionic.git;a=blob;f=libc/bionic/pthread.c;h=7d4056d2556bcb0cb9591699f288999086b58b50;hb=HEAD
/* process-shared mutexes are not supported at the moment */
int pthread_mutexattr_setpshared(pthread_mutexattr_t *attr, int pshared)
{
if (!attr)
return EINVAL;
switch (pshared) {
case PTHREAD_PROCESS_PRIVATE:
case PTHREAD_PROCESS_SHARED:
/* our current implementation of pthread actually supports shared
* mutexes but won't cleanup if a process dies with the mutex held.
* Nevertheless, it's better than nothing. Shared mutexes are used
* by surfaceflinger and audioflinger.
*/
return 0;
}
return ENOTSUP;
}
> Implementing your own semaphore around mutex/cond variables should be okThat looks like a good option. Otherwise my shared semaphore is going to break
> as long as you understand the implications regarding PTHREAD_PROCESS_SHARED
> and the behaviour of the C library (or more exactly, various versions of
> the C library).
when you refactor the implementation.
Basically, I could simply call the following and ignore its return value:
pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
Then the mutex would be shared, Froyo or not, right?
> > When we say that we don't support PTHREAD_PROCESS_SHARED, itI see, somehow got that.
> really
> > means you should not expect the current
> > behaviour to be cast in stone and not change in the future.
> Until we say
> > we support it.
>
> So, if I want to target android-3, I can use
> PTHREAD_PROCESS_SHARED, which will
> be silently ignored on Froyo, but is needed on the later, right?
>
>
> Actually, on android-3, using PTHREAD_PROCESS_SHARED will return an
> error
> when initializing a mutex or semaphore, as far as I understand, the
> default implementation
> will be shareable (by accident).
>
> As you can see, when we say that some things are not supported, we
> really mean
> "do not use it, might break in the future".
--
Olivier
Olivier Guilyardi
>
>
> On Wed, Jun 2, 2010 at 11:53 AM, Olivier Guilyardi <li...@samalyse.com
> <mailto:li...@samalyse.com>> wrote:
>
> On 06/02/2010 05:13 PM, David Turner wrote:
> >
> Okay, got that. Anyway I've never used futexes, and it seems very
> tricky..
>
> Indeed :)
>
> That said, is this Froyo code or am I looking in the wrong place?
>
> http://android.git.kernel.org/?p=platform/bionic.git;a=blob;f=libc/bionic/pthread.c;h=7d4056d2556bcb0cb9591699f288999086b58b50;hb=HEAD
[...]
> This is Eclair code (which itself was modified from Donut where
> PTHRAD_PROCESS_SHARED returned an error).
> The changes I have mentioned are in Froyo, iirc, which is not
> open-sourced yet.
Okay, but I quite don't get why you release binaries before sources.
[...]
> Basically, I could simply call the following and ignore its return
> value:
> pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
>
> Then the mutex would be shared, Froyo or not, right?
>
> Essentially yes, but it's really accidental so please put a huge warning
> comment
> around that code to explain what you are doing, in case you need to port
> this to
> another platform or something.
I'm already testing on x86, so I had some ifdef.. But well I found something else:
// This should be a safe way to create shared semaphore. The pshared must
// currently be zero on Android, but may become required in a later version
if (sem_init(&sem, 1, 0) && sem_init(&sem, 0, 0)) {
LOG("Failed to init shared semaphore");
}
Here I make the assumption that you will refactor the semaphore in a similar way
to what you did for the pthread mutex. That is: make the pshared parameter of
sem_init() work, and use private futexes if it's 0, or normal ones otherwise.
Currently, with this this code, the first call to sem_init() always fail, and
then the second succeeds, initializing an "accidentally" shared semaphore.
But whenever you add support for the pshared parameter, this code should
continue to work and create the shared semaphore that I need.
Isn't that correct? At least 99% ?
--
Olivier
Olivier Guilyardi
[...]
I suppose your silence means: no, /when we say that some things are not
supported, we really mean "do not use it, might break in the future"/
;)
--
Olivier
David Turner
I suppose your silence means: no, /when we say that some things are not
supported, we really mean "do not use it, might break in the future"/
;)
Olivier Guilyardi
>
>
> On Thu, Jun 3, 2010 at 2:57 PM, Olivier Guilyardi <li...@samalyse.com
> <mailto:li...@samalyse.com>> wrote:
>
>
> I suppose your silence means: no, /when we say that some things are not
> supported, we really mean "do not use it, might break in the future"/
>
> ;)
>
>
> My silence only means I'm too busy, usually.
>
> As for this specific case, it is likely to work. I guess it's very
> likely to work but is a gross hack that I cannot really recommend :-)
Okay, what about this one:
int sharedsem_init(sem_t *sem, int value) {
/* The following creates a process-shared semaphore, if supported */
int result = sem_init(sem, 1, value);
#ifdef ANDROID
/* WARNING: Android doesn't support explicitly shared semaphore (at
least until Android 2.2). But it happens that all semaphores can
actually be shared. At some point in time, support for the pshared
parameter may be added, and the first call to sem_init() above
will then succeed. Meanwhile, we fallback on trying to create an
/accidentally/ shared semaphore by setting the pshared parameter
to zero. This should be quite safe but remains dangerous in case
Android semaphores can't be shared by default anymore, and that
support for pshared is not added at the same time.
*/
if (result && errno == ENOSYS) {
result = sem_init(sem, 0, value);
}
#endif
return result;
}
Nice warning, isn't it? ;-)
This code is tested on Android 1.5, 1.6, 2.2, and on standard Linux x86.
--
Olivier
David Turner
--
Olivier
Olivier Guilyardi
Thanks
On 06/04/2010 07:24 PM, David Turner wrote:
> Yep, nice warning, I think you nailed it :)
>
> On Fri, Jun 4, 2010 at 3:07 AM, Olivier Guilyardi <li...@samalyse.com
> <mailto:li...@samalyse.com>> wrote:
>
> On 06/04/2010 12:21 AM, David Turner wrote:
> >
> >
> > On Thu, Jun 3, 2010 at 2:57 PM, Olivier Guilyardi
> <li...@samalyse.com <mailto:li...@samalyse.com>
> <mailto:andro...@googlegroups.com>.
> To unsubscribe from this group, send email to
> android-ndk...@googlegroups.com
> <mailto:android-ndk%2Bunsu...@googlegroups.com>.
Olivier Guilyardi
> On 06/02/2010 09:11 PM, Dianne Hackborn wrote:
>> On Wed, Jun 2, 2010 at 11:42 AM, Olivier Guilyardi <li...@samalyse.com
>> <mailto:li...@samalyse.com>> wrote:
>>
>> This paragraph was about sending commands to the audio engine. For
>> example, the
>> waveform can be scrolled/scratched, and thus seeking must be
>> performed very
>> fast, otherwise it feels sluggish. And it does feel better with
>> ringbuffer approach.
>>
>>
>> For what it's worth, the events you are getting from the framework are
>> all currently being delivered via Binder IPCs.
>
> I'm not saying that it is so slow. But if I can provide slightly smoothing
> scrolling, my users will be happy about it. I actually got some user feedback
> about this. Binder is great and *I do use it*, but I prefer to go native as soon
> as possible when something critical must be performed.
Just an additional note here: scrolling/seeking responsiveness seem to be much
affected by the way GestureDetector works. The detector isn't the culprit since
it tries to recognize various gestures and that is fine. But I expect to achieve
better responsiveness with a custom detector optimized for scroll gestures.
I just wanted to clarify this and correct what I said about Binder which doesn't
seem to be the bottleneck for this specific case.
--
Olivier
Dianne Hackborn
--
Olivier
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To post to this group, send email to andro...@googlegroups.com.
To unsubscribe from this group, send email to android-ndk...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/android-ndk?hl=en.
Olivier Guilyardi
satisfying solution, where I still use it but have my own code for scroll
detection. It was very easy to code by reading the GestureDetector source. I
actually need to detect taps, but it's absolutely harmless if I get scroll
events at the same time.
Nevertheless, I can still say that my shared memory solution is noticeably
faster for sending frequent engine-to-ui events. Maybe that's partly because
when I used callbacks through Binder I was allocating a lot of runnables in the
ui thread, I can't really say...
Olivier
On 06/08/2010 12:40 AM, Dianne Hackborn wrote:
> Ah yeah if the gesture detector doesn't give the behavior you want,
> don't use it. It is primarily intended for user interactions in the
> standard UI -- in particular when the user can do multiple things like
> tapping and scrolling. If they can only scroll, you would probably be
> better off just directly tracking the finger instead of going through
> the gesture detector's disambiguation for taps (which necessarily means
> scrolls can't start immediately).
>
> On Mon, Jun 7, 2010 at 8:09 AM, Olivier Guilyardi <li...@samalyse.com
> <mailto:li...@samalyse.com>> wrote:
>
> On 06/02/2010 09:45 PM, Olivier Guilyardi wrote:
> > On 06/02/2010 09:11 PM, Dianne Hackborn wrote:
> >> On Wed, Jun 2, 2010 at 11:42 AM, Olivier Guilyardi
> <li...@samalyse.com <mailto:li...@samalyse.com>
> <mailto:andro...@googlegroups.com>.
> To unsubscribe from this group, send email to
> android-ndk...@googlegroups.com
> <mailto:android-ndk%2Bunsu...@googlegroups.com>.
> For more options, visit this group at
> http://groups.google.com/group/android-ndk?hl=en.
>
>
>
>
> --
> Dianne Hackborn
> Android framework engineer
> hac...@android.com <mailto:hac...@android.com>
>
> Note: please don't send private questions to me, as I don't have time to
> provide private support, and so won't reply to such e-mails. All such
> questions should be posted on public forums, where I and others can see
> and answer them.
>