IPC in pthreads
Richard Craig
I have a multi-threaded IP server, where each thread can serve a num
ber of clients. The 'main' thread performs an accept loop and each time a
client connects it farms it out to a thread (starting one if necessary). It
occurs according to the folowing:
1. The worker thread is in a poll(2) loop, and polls for data on a
ll the client sockets and a pipe(2) input.
2. The server finds a thread that has the least connections and then
writes to the threads pipe, giving it a magic number and a pointer to
the a structure containing client information.
3. The worker thread reads this (eventually) from the pipe and adds
it to its own list, and then sends a confirmation back to the main thread.
4. The main thread, after sending the data to the client thread, pol
ls its own pipe for some period of time, to wait for the confirmation,
if it hasn't received it sends a cancel message (in [3] the threads search
es for a cancel message).
Basically, I don't like this method. The aim was to ensure that each thread
modifies its own data. The main thread could technically just add the
client to the worker threads list, but it doesn't. I also don't like all
the pipes. Also their may be a race condition between [3] and [4], where
the thread may exit (which it does when all its connections close).
My question, is there a better way to handle this sort of thing?
any thoughts welcome,
richard.
--
Richard Craig | `The man who trades freedom for security
-----------------------| does not deserve, nor will he ever
<ricc...@uk2.net> | receive, either.' (Benjamin Franklin)
David Schwartz
> Basically, I don't like this method. The aim was to ensure that each thread
> modifies its own data. The main thread could technically just add the
> client to the worker threads list, but it doesn't. I also don't like all
> the pipes. Also their may be a race condition between [3] and [4], where
> the thread may exit (which it does when all its connections close).
>
> My question, is there a better way to handle this sort of thing?
>
> any thoughts welcome,
I would just have the main thread add the work to the worker thread's
task list. You can dispense with the pipes entirely by polling with a
sensible time out. Ideally, you would 'poll' with a one second timeout,
but this could mean a wait of up to one second for the worker thread to
'notice' the new task. If that delay is acceptible, then definitely do
it that way.
DS
Vince Mounts
wrote:
>Hi,
> I have a multi-threaded IP server, where each thread can serve a num
>ber of clients. The 'main' thread performs an accept loop and each time a
>client connects it farms it out to a thread (starting one if necessary). It
>occurs according to the folowing:
>
<SNIP>
If you are using C++ you may want to have a look at
http://www.cs.wustl.edu/~schmidt/ACE.html ACE is a open source library
that handles many complex threading/IPC patterns. There are also a
number of academic papers on the patterns it uses. Its liscense is
such that you can use it for anything you, commercial included,
without needed to make the source of your app available.
Alexander Terekhov
Vince Mounts wrote:
[...]
> If you are using C++ you may want to have a look at
> http://www.cs.wustl.edu/~schmidt/ACE.html ACE is a open source library
Stay away from ACE... ~schmidt et al papers and books are really good,
but ACE lib is nothing but buggy bloatware. Rather, the entire "DOC"
group should have fully concentrated on C++ version of "PACE"[1]
(C++ bindings/wrappers for POSIX.1; threads including)...
regards,
alexander.
[1] http://groups.google.com/groups?selm=3C8FCC2F.9B4B1AAB%40web.de
(Subject: Re: C++ concurrency library feature-set)
Attila Feher
>
> Vince Mounts wrote:
> [...]
> > If you are using C++ you may want to have a look at
> > http://www.cs.wustl.edu/~schmidt/ACE.html ACE is a open source library
>
> Stay away from ACE... ~schmidt et al papers and books are really good,
> but ACE lib is nothing but buggy bloatware. Rather, the entire "DOC"
> group should have fully concentrated on C++ version of "PACE"[1]
> (C++ bindings/wrappers for POSIX.1; threads including)...
You are the first telling this. I have talked to many successfully
employing ACE...
A
Alexander Terekhov
"Many" [all around this planet; I'm too ;-)] "successfully" {de/em}ploy
MS-Windows... and mass deployment of MS-CLI/.NET brain-dead-beyond-the-
limits stuff is just-around-the-corner as well.
Have you, personally {and your team}, done ACE-lib code review?
regards,
alexander.
Attila Feher
Not me personally. I was talking to respected names as well as guys
from the US, Greece etc. What I have pointed out is that you are the
first one in my life whom I see bash ACE. Which is not true with all
the MS products you referred to. BTW, I start to think that you are a
Linux fanatic... I mean you have not given any concrete fact to base
your dislike upon, but AFAIS you hate everything else but pthreads...
including MS. ;-)
A
Alexander Terekhov
Attila Feher wrote:
[...]
> BTW, I start to think that you are a Linux fanatic...
Yeah, I know.
> I mean you have not given any concrete fact to base
> your dislike upon, but AFAIS you hate everything else but pthreads...
> including MS. ;-)
Well, believe me, I can bombard you to death by various links
[and even more in&out e-mails from my archive] on quite a few
ACE/MS{CLI|.NET}/pthreads/linuxthreads-too-BTW "concrete" facts
that I hate. For now, given that you seem to love so much ACE,
take this:
http://groups.google.com/groups?selm=C12569D7.0074491A.00%40d12mta05.de.ibm.com
http://groups.google.com/groups?selm=C12569E5.005BE639.00%40d12mta05.de.ibm.com
http://groups.google.com/groups?selm=C1256A72.005F1860.00%40d12mta05.de.ibm.com
http://groups.google.com/groups?selm=200105100838.f4A8bwu12332%40mailgate3.cinetic.de
regards,
alexander.
Daniel Miller
ACE provides much more functionality than merely threads. If someone
wants only a threads library, then all of the extra
network-communications portions of ACE is unnecessary overhead/bloat as
Alexander Terekhov (tersely) points out. In answer to this valid
criticism of ACE, the PACE subset is being factored out of ACE as a
lower layer within ACE to provide a wrapper layer for operating systems
calls such as threads. PACE intentionally focuses merely on wrapping
operating system calls without imposing the bulk of ACE's larger
network-communications goals. Alexander Terekhov's claim of ACE being
too big for certain jobs has merit and has been recognized by multiple
people since no later than the mid-1990s. As Alexander Terekhov's link
mentioned, PACE can be advocated as a smaller-footprint substitute for
the larger ACE when all that is desired is a portable operating system
wrapper layer.
It is up to the original poster to decide whether his design is
better suited:
1) to using the smaller PACE to model only the threads (appropriately
ignoring ACE's network-communications capabilities) or
2) to using the larger ACE to model both the threads as well as the
IPC currently implemented via pipes (appropriately taking advantage of
ACE's network-communications capabilities).
Nicolás Lichtmaier
> case (I think) the poll will be interupted? I don't know much about how
> signals work in pthreads, and from what I've heard the signal implemention
> is iffy on linux (the development platform).
Why don't you just do this:
Add the fd direcly to the thread's list.
Send a dummy byte through a control socket to wake up the thread. You
discard that byte.