Re: first patch for process-shared semaphore
John Baldwin
> This is my first attempt to make process-shared mutex work, this means
> you can mmap(MAP_SHARED) a memory area, and put semaphore there,
> or you can sem_open a named semaphore, and just use it between
> processes, the named semaphore uses file system and mmap(), directory
> /tmp/.semaphore is used as IPC directory, any named semaphore
> locates in the directory. old semaphore implementation still exists
> to make it binary compatible, it uses symbol version.
>
> http://people.freebsd.org/~davidxu/patch/shared_semaphore_1.patch
I would suggest that you leave named semaphores as they currently exist and
follow this approach instead:
1) Named semaphores use ksem_*() still.
2) sem_init/sem_destroy operate on UTMX-backed semaphores identical to the
ones used in the current libthr code. The semid_t structure now becomes the
full structure that libthr currently allocates with a flag to indicate if it
is a "system" semaphore or otherwise. The pshared flag passed to sem_init()
can be used to set the sharing properties of the UMTX.
3) All of sem_init/sem_destroy is just in libc. Just move the libthr
implementation bits into libc.
--
John Baldwin
David Xu
ksem base shared semaphore is slow because whenever you call
sem_wait(), it always enters kernel even if count is non-zero,
sem_post() also always enters kernel even if there is no waiter.
but the new implementation is as simple as just an atomic operation
in these cases, I know another competitor OS is doing things in
this way.
John Baldwin
> On Thu, 24 Dec 2009 09:58:50 +0800
> David Xu <dav...@freebsd.org> wrote:
>
> > Alexander Kabaev wrote:
> > > On Thu, 24 Dec 2009 09:22:34 +0800
> > > David Xu <dav...@freebsd.org> wrote:
> > >> libthr does not require semaphore, it implements semaphore,
> > >> it is easier than other ways to implement the process-shared.
> > >>
> > > Let me rephrase: I do not think semaphores belong in libthr. They
> > > should be either in libc or in librt.
> > >
> > >
> > OK, does others really implement semaphore in librt ?
> > unfortunately, the librt already requires libpthread to implement
> > SIGEV_THREAD.
>
> I retract that. It appears that there is no consistency - Solaris put
> these into libc, Linux into libpthread ans SUSv2 hints that these
> belong with realtime functions. libthr is fine.
I vote for libc. Single-threaded processes can use sem_open() and PSHARED
sem_init() as well. Single-threaded processes can even use non-PSHARED
sem_init() by using fork() to create new "threads" that share the semaphore.
--
John Baldwin
John Baldwin
Yes, Solaris uses files in /tmp and Darwin uses special file descriptors
similar to what we do. However, you will have to restrict the namespace if
you go the /tmp route to be safe I think similar to what Solaris does (no path
separators, just simple names like 'foo'). You might also want to use the
same naming convention as Solaris if you go the /tmp route (I think they use a
path other than .semaphore under /tmp IIRC). Not sure if we want to do
anything special to ensure that those particular set of files in /tmp always
get purged on reboot to avoid weird bugs with semaphores unexpectedly
persisting across reboots.
--
John Baldwin
David Xu
semaphore related symbols from libthr ? In pratical, this is not a
problem, because libthr itself is not dlopen-safe, all missing semaphore
functions in libthr will be found in libc by rtld.
John Baldwin
> John Baldwin wrote:
>
> > Yes, Solaris uses files in /tmp and Darwin uses special file descriptors
> > similar to what we do. However, you will have to restrict the namespace if
> > you go the /tmp route to be safe I think similar to what Solaris does (no path
> > separators, just simple names like 'foo').
>
> current implementation.
Actually, the current implementation in 8+ allow for arbitrary pathnames.
However, that is a relatively recent change and it is probably ok to restrict
it again.
> > You might also want to use the
> > same naming convention as Solaris if you go the /tmp route (I think they use a
> > path other than .semaphore under /tmp IIRC).
>
> What is their name convention? The patch just create all semaphore files
> in directory /tmp/.semaphore and semaphore name is not changed.
They actually create two files in /tmp for each semaphore, a lock file and a
data file. _Solaris Internals_ doesn't really indicate why the lock file
exists. The files are named /tmp/SEML<name> and /tmp/SEMD<name> where <name>
does not include the leading '/', a sem_open() of '/foo' opens /tmp/SEMDfoo.
It is perhaps safer to not assume that a /tmp/.semaphore directory exists
and to create files in /tmp directly?
> > Not sure if we want to do
> > anything special to ensure that those particular set of files in /tmp always
> > get purged on reboot to avoid weird bugs with semaphores unexpectedly
> > persisting across reboots.
> >
>
> Yes, this is an issue, I would like to purge it on reboot.
Perhaps /etc/rc.d/cleartmp should always clean semaphores, or at least have a
separate rc.conf variable (similar to the one for X files) that is enabled by
default?
--
John Baldwin
John Baldwin
I would go with this approach. There is also some discussion about moving all
of libthr into libc as well and having a dummy libpthread now that we are back
to a single threading library and to avoid issues with dlopen() of libpthread,
etc.
--
John Baldwin
Alexander Kabaev
Removing symbols from library, versioned library especially, is a big
wrong. You should provide compat symbols for all functions that that
were there before.
--
Alexander Kabaev
John Baldwin
So can you have a 1.1 sem_init() in libthr that calls the 1.2 sem_init() in
libc? I'm ok with simple compat stubs. In this case the compat stubs might
actually need to do work though since they would need to malloc() a separate
cookie structure to map from the "old" semid_t that was just a pointer to the
new structure. Presumably even libc would need 1.1 compat shims in this case,
too. Not sure if you can have libthr's compat shims call the compat shims in
libc somehow to avoid code duplication.
--
John Baldwin
Daniel Eischen
I tend to agree. I also don't understand the desire to include libthr
in libc. Other than dlopen()ing libpthread (which can be worked
around other ways), I don't understand why we would need or want this.
--
DE
David Xu
>>> May I can move all semaphore functions into libc and remove all
>>> semaphore related symbols from libthr ? In pratical, this is not a
>>> problem, because libthr itself is not dlopen-safe, all missing
>>> semaphore functions in libthr will be found in libc by rtld.
>> I would go with this approach. There is also some discussion about
>> moving all of libthr into libc as well and having a dummy libpthread
>> now that we are back to a single threading library and to avoid
>> issues with dlopen() of libpthread, etc.
>>
>
> Removing symbols from library, versioned library especially, is a big
> wrong. You should provide compat symbols for all functions that that
> were there before.
>
I have updated the patch:
http://people.freebsd.org/~davidxu/patch/shared_semaphore_2.patch
this time, all semaphore code are moved into libc and libthr still
has semaphore stubs.
Also, now semaphore file is created in /tmp as name SEMD<name>,
this looks like Solaris does as jhb@ said, though the patch doesn't
use lock file, but use the semaphore file itself.