tpool
John Kelly
how many threads I could create. At 125 threads, top reported about
1G of virtual memory.
>Tasks: 103 total, 1 running, 102 sleeping, 0 stopped, 0 zombie
>Cpu(s): 0.7% us, 0.4% sy, 0.0% ni, 98.3% id, 0.4% wa, 0.0% hi, 0.2% si
>Mem: 385252k total, 374396k used, 10856k free, 60924k buffers
>Swap: 1028152k total, 0k used, 1028152k free, 155972k cached
>
> PID PPID USER S NI PR TIME %CPU %MEM RES VIRT CODE SHR COMMAND
> 5206 1 clamav S 0 16 0:11 0.0 13.2 49m 62100 44 1100 clamd
>17214 13547 root S 0 16 0:01 0.9 7.4 27m 1032m 4 1236 tclsh
Next I tried 150 threads, but it crashed. Are there any linux kernel
limits I can raise to start more threads?
>can't create a new thread
> while executing
>"tpool::post -detached $clientpool [list clientthread $sock $host $port]"
> (procedure "clientdetach" line 6)
> invoked from within
>"clientdetach sock138 127.0.0.1 4006"
> ("after" script)
I don't think threads-max is the problem:
>kernel.threads-max = 6126
>fs.file-max = 37907
--
Internet service
www.isp2dial.com
Michael Schlenker
> I was curious about tpool threads, so I wrote an echo server to see
> how many threads I could create. At 125 threads, top reported about
> 1G of virtual memory.
>
>> Tasks: 103 total, 1 running, 102 sleeping, 0 stopped, 0 zombie
>> Cpu(s): 0.7% us, 0.4% sy, 0.0% ni, 98.3% id, 0.4% wa, 0.0% hi, 0.2% si
>> Mem: 385252k total, 374396k used, 10856k free, 60924k buffers
>> Swap: 1028152k total, 0k used, 1028152k free, 155972k cached
>>
>> PID PPID USER S NI PR TIME %CPU %MEM RES VIRT CODE SHR COMMAND
>> 5206 1 clamav S 0 16 0:11 0.0 13.2 49m 62100 44 1100 clamd
>> 17214 13547 root S 0 16 0:01 0.9 7.4 27m 1032m 4 1236 tclsh
>
>
> Next I tried 150 threads, but it crashed. Are there any linux kernel
> limits I can raise to start more threads?
Its not usually helpful to have more threads then cpu cores in your
machine, you don't win any performance any more.
But agreed, it should not crash. Try filing a bug report for the
package, best with a script that reliably reproduces the crash and
details about your Linux machine (kernel version SMP or non-SMP, glibc
version, CPU etc.)
Michael
John Kelly
<sch...@uni-oldenburg.de> wrote:
>> Next I tried 150 threads, but it crashed. Are there any linux kernel
>> limits I can raise to start more threads?
>Its not usually helpful to have more threads then cpu cores in your
>machine, you don't win any performance any more.
Right. This is a protoype. If it works, I can buy a 64 processor box
later. ;-)
>But agreed, it should not crash.
Well it has to run out of resources at some point, but I hoped that
would be later, at 5,000 threads or more. I did get Windows 2003 to
start about 1,000 threads, before it flopped.
If Windows 2003 can start 1,000 threads, seems like linux should go
higher than 125. But I don't know where else to look, for increasing
linux limits. My script is running as root, so user limits are not a
factor.
Are there any hard Tcl limits compiled in? I suppose I could look at
the source, but I'm not that far along yet.
>Try filing a bug report for the package, best with a script that
>reliably reproduces the crash and details about your Linux machine
>(kernel version SMP or non-SMP, glibc version, CPU etc.)
OK.
--
Internet service
www.isp2dial.com
John Kelly
> http://wiki.tcl.tk/_ref/3829
which leads to:
> Threads vs. events
> http://wiki.tcl.tk/1904
which leads to:
> Why Events Are A Bad Idea
> http://www.usenix.org/events/hotos03/tech/full_papers/vonbehren/vonbehren_html/index.html
which says:
> Threaded systems typically face a tradeoff between risking stack
> overflow and wasting virtual address space on large stacks
... and reminds me of something I read about linux stack size, 4k vs.
8k. Maybe stack size is a factor ... searching ...
> https://lists.dulug.duke.edu/pipermail/dulug/2003-March/006719.html
ulimit -s unlimited
Got me up to about 300 threads, but still crashed. Then,
ulimit -v unlimited
Got me up to 900 threads. At that point I was using so much virtual
memory (2 gig, on a box with only 384 meg ram), I didn't try to go any
higher.
>top - 15:57:03 up 8:59, 2 users, load average: 1.57, 0.98, 0.48
>Tasks: 105 total, 1 running, 104 sleeping, 0 stopped, 0 zombie
>Cpu(s): 90.3% us, 9.3% sy, 0.0% ni, 0.0% id, 0.0% wa, 0.0% hi, 0.4% si
>Mem: 385252k total, 372636k used, 12616k free, 484k buffers
>Swap: 1028152k total, 4912k used, 1023240k free, 14148k cached
>
> PID PPID USER S NI PR TIME %CPU %MEM RES VIRT CODE SHR COMMAND
> 7840 6863 root S 0 16 2:30 98.2 64.5 242m 2068m 4 1276 tclsh
> 5163 1 clamav S 0 16 0:07 0.0 8.2 30m 43132 44 868 clamd
I understand AOLserver uses Tcl, how do they scale up? Do they pass
work off to a relatively small pool of Tcl threads?
Each of my Tcl threads will provide a proxy between a remote user and
a back end server, where each user has their own process. 5,000 would
be a good start, I suppose if I need to scale higher I can reimplement
with a compiled language.
--
Internet service
http://www.isp2dial.com/
Michael Schlenker
> I understand AOLserver uses Tcl, how do they scale up? Do they pass
> work off to a relatively small pool of Tcl threads?
Take a look at:
http://www.aolserver.com/docs/intro/tcl2k/html/
>
> Each of my Tcl threads will provide a proxy between a remote user and
> a back end server, where each user has their own process. 5,000 would
> be a good start, I suppose if I need to scale higher I can reimplement
> with a compiled language.
If its just a proxy, using the rather heavyweight Tcl threads might be
not the best solution.
Maybe this has some interesting information for you:
http://www.maplefish.com/todd/tcl_net_idioms.html
Michael
John Kelly
<sch...@uni-oldenburg.de> wrote:
>> Each of my Tcl threads will provide a proxy between a remote user and
>> a back end server, where each user has their own process. 5,000 would
>> be a good start, I suppose if I need to scale higher I can reimplement
>> with a compiled language.
>If its just a proxy, using the rather heavyweight Tcl threads might be
>not the best solution.
>Maybe this has some interesting information for you:
>http://www.maplefish.com/todd/tcl_net_idioms.html
I say "proxy" loosely. I have in mind a webmail server that talks
http to web browsers and imap to a backend server, keeping state for
each user session, in a way similar to the University of Cambridge
prayer webmail server, but using Tcl, instead of C.
I first thought of threads, instead of events, because some imap ->
http conversions may be slower than others, and like Todd said in his
paper, you don't want to have fast freight waiting on the slow.
But his paper got me thinking ...
I could use the event model for the socket cross connections between
the user and the imap server, doing only the essential network I/O.
The slower work of converting imap server responses to formatted web
pages could be handed off to a relatively small pool of Tcl threads.
The more I think about it, the more sense it makes to isolate the two
types of work ...
Thanks for the link, Michael.
Gerald W. Lester
You may want to use TML to do the conversion to web pages, I think you will
be surprised at how fast converting the IMAP server responses to web pages
can really be.
My guess is you can easily do several hundred a second (assuming you can get
the data that fast from the IMAP servers).
For what you are describing I see no needs for threads. I'd also start with
the IMAP package out of TclLib plus either TclHttpd or WUB as the base of
the server -- either will do 90%+ of the work for you.
--
+--------------------------------+---------------------------------------+
| Gerald W. Lester |
|"The man who fights for his ideals is the man who is alive." - Cervantes|
+------------------------------------------------------------------------+
tom.rmadilo
> I understand AOLserver uses Tcl, how do they scale up? Do they pass
> work off to a relatively small pool of Tcl threads?
It might be worth pointing out that threads in Tcl are due to
AOLserver. In other words, the developers of AOLserver (actually
NaviServer) made a few changes to Tcl to make it thread safe.
AOLserver is an application server designed for high performance
network application programming.
Another point is that Tcl has a C level API. AOLserver uses this API
for memory management and a bunch of other stuff, like adding thread
support, shared memory variables/arrays, most of this is available to
Tcl scripting as well. What this means is that you can write your
application using Tcl scripts inside AOLserver using the same concepts
as any other C or C++ threaded program.
So how does AOLserver scale up? Currently each instance (process) of
AOLserver listens on any number of ip/ports. Incoming requests are
handled by a driver which chooses a threadpool. A thread is selected
from the pool and a Tcl interp is also selected and associated with
the thread. The request is run and both the interp and thread are
returned. This is just for http requests, but there is an imap module:
http://aolserver.cvs.sourceforge.net/aolserver/nsimap/
This is a C module with a Tcl scripting API.
> Each of my Tcl threads will provide a proxy between a remote user and
> a back end server, where each user has their own process. 5,000 would
> be a good start, I suppose if I need to scale higher I can reimplement
> with a compiled language.
Without details of why each user needs a separate process, it is hard
to say much. But I can't think if a reason to think threads and per-
user-process at the same time. In this case, I would just use
something like DJB's tcpserver to startup the process.
Why use threads?
* Good model for multiple users with different use patterns.
* Both long and short running requests easily managed at the same
time.
* Easy to setup, maintain and reuse expensive-to-start
resources...like backend connections to another server (database,
imap, ldap).
But when you use an application server which has already handled the
threading code, then all that is left is the less difficult (and more
interesting) parts of what each thread does.
John Kelly
<tom.r...@gmail.com> wrote:
>So how does AOLserver scale up? Currently each instance (process) of
>AOLserver listens on any number of ip/ports. Incoming requests are
>handled by a driver which chooses a threadpool. A thread is selected
>from the pool and a Tcl interp is also selected and associated with
>the thread. The request is run and both the interp and thread are
>returned. This is just for http requests, but there is an imap module:
>
>http://aolserver.cvs.sourceforge.net/aolserver/nsimap/
Thanks for the info and that link, I'll have a look when I find more
time ...
>> Each of my Tcl threads will provide a proxy between a remote user and
>> a back end server, where each user has their own process.
>Without details of why each user needs a separate process, it is hard
>to say much. But I can't think if a reason to think threads and per-
>user-process at the same time.
I understand, let me clarify.
The back end server is UW imap. It's managed by xinetd, which starts
a new pid for each connection. The imap deamon process begins life as
root, sends a welcome message, and waits a short time for the user to
login. Once the user authenticates, it drops root and sets uid to the
authenticated user, so the user can access their files with safe and
correct permissions.
That means my http <-> imap gateway will be talking to different pids
on the backend imap server, but the gateway server itself will handle
all http client connections with one pid.
>Why use threads?
There are strengths and weaknesses of both models, threads vs. events.
In my server, there is a natural separation of work, where events are
the best fit for some work, threads for other work. And since Tcl has
both, I can use both.
John Kelly
<Gerald...@cox.net> wrote:
>You may want to use TML to do the conversion to web pages, I think you will
>be surprised at how fast converting the IMAP server responses to web pages
>can really be.
>My guess is you can easily do several hundred a second (assuming you can get
>the data that fast from the IMAP servers).
>For what you are describing I see no needs for threads. I'd also start with
>the IMAP package out of TclLib plus either TclHttpd or WUB as the base of
>the server -- either will do 90%+ of the work for you.
Thanks for the info, I will look into it.
In terms of performance, you may be right that threads are not needed.
However, the work my server will do, divides naturally into three main
components.
1. Talk to clients
2. Talk to a backend server
3. Translate between the two
The http keepalive on the client side will be fairly short, no more
than 5 minutes or so. The imap keepalive on the server side defaults
to 30 minutes, per RFC.
It seems natural to divide the connection handing work, into a front
end talking http to clients, and a back end talking to an imap server.
And there must be a translator between the two, a third component.
Users need to read and think, so there will be quiet time on each
connection, with no translation work in progress. I speculate the
ratio could be as high as 100 to 1, active connections vs. active
translation tasks.
If that holds true, then for each 5,000 active connections, I only
need about 50 translation workers. In that case, thread pools are a
natural fit for handling the translation work.
Uwe Klein
> 1. Talk to clients
> 2. Talk to a backend server
> 3. Translate between the two
Tcl enables you to do this eventbased.
Look into
[ fileevent readable|writable]
[ trace variable ]
and friends.
See:
http://wiki.tcl.tk/1904
Tcl excelles in event based programming.
uwe
Donal K. Fellows
> There are strengths and weaknesses of both models, threads vs. events.
> In my server, there is a natural separation of work, where events are
> the best fit for some work, threads for other work. And since Tcl has
> both, I can use both.
Best advice so far on this whole thread.
Donal.
Neil Madden
...
>> Why use threads?
>
> There are strengths and weaknesses of both models, threads vs. events.
> In my server, there is a natural separation of work, where events are
> the best fit for some work, threads for other work. And since Tcl has
> both, I can use both.
This seems an entirely sensible approach. It should be noted that much
of the advice against threads is really against shared-state
concurrency. Tcl's threads don't share any state by default, so are
really more like separate processes that run individual event loops
(where message between threads are just one more source of events). This
sounds like a pretty good fit for your project and shouldn't really
cause any more problems than a single-threaded event loop would. Each
event callback should still run in (almost) complete isolation to any
other, as they operate on entirely different sets of variables (in
different interpreters).
Of course, if you later decide that you really need to share some state
between threads than Tcl can support this too, incrementally sharing
just those parts that really need it.
-- Neil