(Slightly OT): Communicating bwt processes using shared mem.
David
I know this is slightly off-topic for a group on threads but I feel this
applies much to threading apps as well....
Supposing if I have a multi-threaded process and a 'normal' process. I want
the normal process to communicate with any one of the threads in the
multi-threaded process by shared memory. Right now, I can only think of
using variables (with semaphores-locking) to communicate. But this means
polling in a tight loop for various variables, waiting for a command/request
to occur. And maybe to prevent the loop for sucking too much CPU cycles, I
can just add a call to usleep() for a short while. Is this how programmers
(even experienced ones) do things in situation like mine?
(BTW, is it possible to use cond variables and mutexes on shared memory
between processes?)
Thanks.
David
Rick Ross
> between processes?)
Depends on the operating system. Linux currently only supports Mutexes
and System V Semaphores across processes. Other operating systems should
have condition variables and read/write locks, but YMMV.
HTH
--
Rick Ross
Author: Kylix 2 Development.
http://www.amazon.com/exec/obidos/ASIN/1556227744/rickrosscom-20
Hack Bert
"David" <us...@127.0.0.1> wrote in message
news:191C91BDFE8ED411B844...@pfs21.ex.nus.edu.sg...
> Hi,
Ho.
>
> (BTW, is it possible to use cond variables and mutexes on shared memory
> between processes?)
>
it might be. there is an attribute that can be set in a mutex / cond.
variable called PTHREAD_MUTEX_PSHARED (or similiar, dont know the excat
wording). this is an optional feature for pthread libraries. if portability
is an issue for you, be aware that not too many system makers support it.
and the most important drawback: if you implement a client server
application, where a server must survive a client crashing while it has
locked a pthread_mutex_t, stay away from that feature and use system v ipc
(semop and friends, look at SEM_UNDO; message queues msgsend and friends).
and of course do not design datastructures in your shared memory that cause
the server to fail if the client aborts in the middle while it was changing
them.
if you want, i can elaborate much longer about the issue - i learned some
lessons while implementing a shared memory communications system for unix.
thomas
Alexander Terekhov
> > (BTW, is it possible to use cond variables and mutexes on shared memory
> > between processes?)
>
>
> Depends on the operating system. Linux currently only supports Mutexes
> and System V Semaphores across processes. Other operating systems should
> have condition variables and read/write locks, but YMMV.
Semaphore-based condition variable:
http://sources.redhat.com/pthreads-win32
Semaphore-based read/write lock:
http://groups.google.com/groups?as_umsgid=3B166244.F923B993%40web.de
Semaphore-based recursive lock (just in case you "port"
something, recursive locking is a "bad" idea to begin
with, IMHO):
http://groups.google.com/groups?as_umsgid=3BA5B950.6E21D198%40web.de
regards,
alexander.
David
> variable called PTHREAD_MUTEX_PSHARED (or similiar, dont know the excat
> wording). this is an optional feature for pthread libraries. if
portability
> is an issue for you, be aware that not too many system makers support it.
> and the most important drawback: if you implement a client server
> application, where a server must survive a client crashing while it has
> locked a pthread_mutex_t, stay away from that feature and use system v ipc
> (semop and friends, look at SEM_UNDO; message queues msgsend and friends).
> and of course do not design datastructures in your shared memory that
cause
> the server to fail if the client aborts in the middle while it was
changing
> them.
>
> if you want, i can elaborate much longer about the issue - i learned some
> lessons while implementing a shared memory communications system for unix.
>
> thomas
Hi Thomas,
I would really appreciate if you would tell me more abt your experiences
with a shared memory communications system.
Basically, what I am going to implement is a multithreaded server. Each
thread in the server app is actually handling a client request. I intend to
create a GUI interface for the server as a separate program. The GUI
interface is able to select which thread to communicate with (and hence
which client).
The GUI interface will show some information abt the particular client. It
is also possible to switch between clients on the fly.
Hence, there is a need for all threads in the server app to listen for
requests from the GUI interface. If the GUI interface wants to talk to this
particular thread, then that thread will response with its details. Hence, I
find that the best way to do this will be through shared memory (with each
thread polling for a particular variable). That variable will act as a
selector for a thread. In addition to waiting for requests/commands from the
GUI interface, the threads have to take care of the more important requests
from the clients too.
May I ask, is my use of polling justifiable in this case?
Thanks,
David
(If you prefer to email me personally, my email is lighthouse16 "at"
hotmail.com.)
David
Regards,
David
"Alexander Terekhov" <tere...@web.de> wrote in message
news:c29b5e33.02013...@posting.google.com...
Alexander Terekhov
Does this mean that I've managed to miss the
latest glibc/linuxthreads release with support
(implementation) of PTHREAD_PROCESS_SHARED
pthread_mutexattr_t, pthread_condattr_t and
pthread_rwlockattr_t ?!?!?!?!?!?!?!?!?!?!?!
>
> Regards,
> David
>
> "Alexander Terekhov" <tere...@web.de> wrote in message
> news:c29b5e33.02013...@posting.google.com...
> > Rick Ross <ri...@rick-ross.com> wrote in message
> news:<3C574A8F...@rick-ross.com>...
> > > > (BTW, is it possible to use cond variables and mutexes on shared
> memory
> > > > between processes?)
> > >
> > >
> > > Depends on the operating system. Linux currently only supports Mutexes
> > > and System V Semaphores across processes. Other operating systems should
> > > have condition variables and read/write locks, but YMMV.
> >
> > Semaphore-based condition variable:
> >
> > http://sources.redhat.com/pthreads-win32
^^^^^
Perhaps you just hate win32 and/or MS-Corp altogether ;-)
(which just prevented you to click on link and find
condvar.c with semaphore-based algorithm description)
OK, try this:
http://groups.google.com/groups?as_umsgid=3AEAC433.1595FF4%40web.de
regards,
alexander.
Rick Ross
> latest glibc/linuxthreads release with support
> (implementation) of PTHREAD_PROCESS_SHARED
> pthread_mutexattr_t, pthread_condattr_t and
> pthread_rwlockattr_t ?!?!?!?!?!?!?!?!?!?!?!
In the glibc version that comes with Red Hat 7.2 (glibc 2.2.4-13), the
pthread_rwlockattr_setpshared does not allow using
PTHREAD_PROCESS_SHARED, returning an error number ENOSYS
The comment above the check states the following
"For now it is not possible to shared (sic) a condition variable"
Alexander Terekhov
Rick Ross wrote:
>
> > Does this mean that I've managed to miss the
> > latest glibc/linuxthreads release with support
> > (implementation) of PTHREAD_PROCESS_SHARED
> > pthread_mutexattr_t, pthread_condattr_t and
> > pthread_rwlockattr_t ?!?!?!?!?!?!?!?!?!?!?!
>
> In the glibc version that comes with Red Hat 7.2 (glibc 2.2.4-13), the
> pthread_rwlockattr_setpshared does not allow using
> PTHREAD_PROCESS_SHARED, returning an error number ENOSYS
>
> The comment above the check states the following
>
> "For now it is not possible to shared (sic) a condition variable"
Well, I've briefly visited glib/linuxthreads cvs at:
http://sources.redhat.com/cgi-bin/cvsweb.cgi/libc/linuxthreads/?cvsroot=glibc
and unfortunately still found this:
int pthread_condattr_setpshared (pthread_condattr_t *attr, int pshared)
{
if (pshared != PTHREAD_PROCESS_PRIVATE && pshared !=
PTHREAD_PROCESS_SHARED)
return EINVAL;
/* For now it is not possible to shared a conditional variable. */
if (pshared != PTHREAD_PROCESS_PRIVATE)
return ENOSYS;
return 0;
}
int __pthread_mutexattr_setpshared (pthread_mutexattr_t *attr, int
pshared)
{
if (pshared != PTHREAD_PROCESS_PRIVATE && pshared !=
PTHREAD_PROCESS_SHARED)
return EINVAL;
/* For now it is not possible to shared a conditional variable. */
if (pshared != PTHREAD_PROCESS_PRIVATE)
return ENOSYS;
return 0;
}
int
pthread_rwlockattr_setpshared (pthread_rwlockattr_t *attr, int pshared)
{
if (pshared != PTHREAD_PROCESS_PRIVATE && pshared !=
PTHREAD_PROCESS_SHARED)
return EINVAL;
/* For now it is not possible to shared a conditional variable. */
if (pshared != PTHREAD_PROCESS_PRIVATE)
return ENOSYS;
attr->__pshared = pshared;
return 0;
}
Hmmm.. too bad, I hoped to finally get rid of all
this semaphore mess!! (I mean my homegrown sem-based
process shared cond.var, rwlock and rec.mutex)
regards,
alexander.
josh
> I know this is slightly off-topic for a group on threads but I feel this
> applies much to threading apps as well....
> Supposing if I have a multi-threaded process and a 'normal' process. I want
> the normal process to communicate with any one of the threads in the
> multi-threaded process by shared memory.
will be gone. And it is shared memory underneath.
David
problems
> will be gone. And it is shared memory underneath.
Thanks Josh. :-) I have thought abt your suggestion and it seems to be
workable. Just to clarify some points:
The domain sockets you are referring to is the UNIX domain sockets
(AF_UNIX)? I need to create a piece of shared memory at the start of the
program and when a connection comes from a client, the server main thread
will create a connected socket pair (by using socketpair() ) and then pass
one of these sockets fd to the new thread. It will also place the other
socket fd into the shared mem.
Now if the GUI want to talk to this thread, it will grab the socket fd from
the shared mem and send/recv from this socket fd.
I hope this is right. If not, pls do alert me to the pitfalls. BTW, is it
possible to create a multicast/broadcast UNIX domain socket? (or am I being
too greedy here?) :-)
Best,
David
josh
> > Could you use domain sockets instead? All interprocess synchronization
> problems
> > will be gone. And it is shared memory underneath.
>
> Thanks Josh. :-) I have thought abt your suggestion and it seems to be
> workable. Just to clarify some points:
>
> The domain sockets you are referring to is the UNIX domain sockets
> (AF_UNIX)?
Stevens' books (may good lord rest his soul)? If not, get them ! meantime look
into man pages, I'm sure it's there. I think you're right, but don't want to lie
to you coz I don't remember for sure.
> I need to create a piece of shared memory at the start of the
> program
Forget your shared memory fixed idea <g>. Just create sockets, exactly like you
would in any "normal" (networked) sockets situation, establish a connection, and
go ahead and read from/write to them (it) and that's it. No synchronization
needed, the OS will do all the stuff. One proc writes inot the socket, another,
on the receiving end, reads from it, that's it. If you need to service multiple
threads, well, you can open a socket per thread, or better yet, make one thread
read from one socket, and then dispatch somehow the received (whatever it is in
your case) to the target thread; you'll need to somehow identify threads then,
maybe thread IDs, or something like that. No biggie. This way you reader-thread
will block on the socket till something comes it, but the rest of threads can go
about doing something else. Well, there are many ways you could arrange that
thing, so either give more info, or just improvise something based on the above
yourself.
> and when a connection comes from a client, the server main thread
> will create a connected socket pair (by using socketpair() )
Why pair? Just socket/bind/listen/accept like you'd normally do (on the server,
that is, on the client, it's socket/connect; all as usual, nothing fancy, just
don't forget to use domain sockets when you create them. In fact, if you create
a regular socket, it'll work just the same, but more expensive as the data will
go down and up the protocol stack, whereas if you use domain sockets, this will
bu shunted much closer to the surface, and therefore be much more efficient.)
But you can start with normal sockets, coz functionally there'll be no
difference. If it's easier for you this way, but it should be pretty much the
same, coz all you need is to supply the domain socket id when you socket() your
socket.
> and then pass
> one of these sockets fd to the new thread. It will also place the other
> socket fd into the shared mem.
Like I've said, you don't need any god damn shared mem <g>. Domain sockets most
likely use shared memory themselves, but you don't need to bother with it, just
use the sockets and the OS will take care of the rest. It's exactly like using
pipes, you get something file-like and read/write via it. If you read and
there's nothing you'll block till there is something, unless you use timeouts,
etc. It's 100% regular socket programming.
> Now if the GUI want to talk to this thread, it will grab the socket fd from
> the shared mem and send/recv from this socket fd.
Forget the shared mem! Otherwise, you can lay your app as you wish.
> I hope this is right.
This is right, but you need to disembarrass yourself of the shared memory, you
don't need to deal with it, just forget it. Use (domain) socket connection,
that's it. No explicit synchronization will be necessary, the OS will
synchronize you, you'll only write/read and block (in the simplest case) if not
ready.
> If not, pls do alert me to the pitfalls. BTW, is it
> possible to create a multicast/broadcast UNIX domain socket?
No idea <g>. If you need to ship your stuff to several receivers, check
mailslots (or whatever the hell it's called, I forget. Message queues? Something
like that.) If you work with unix, then as a man of honour you must guy Stevens'
books, and he covers all this crap in minutest detail; those are great book.
Plus it'll be an appropriate tribute to that great guy. Richard Stevens, check
him out in a bookstore, outstanding stuff. Costly, but worhty. Do yourself a
favour.
> (or am I being too greedy here?) :-)
It's a good trait for a programmer. You must be lazy and greedy or you don't
qualify for the profession. Fat ass and lack of hygiene optional :-0
David
Thanks for explaining everything to me. :-) I appreciate all the help you
have given. (And those others who have replied as well.) I guess all that is
left is the hardwork of coding all these...
All the best,
David