ANNC: Got hardware speed? IOCPSOCK v2.0
David Gravereaux
(6:50 PM 3/23/2004)
NEW for 2.0: tunable fconfigures.
http://sf.net/project/showfiles.php?group_id=73356
http://sf.net/projects/iocpsock
This is a new sockets channel driver for windows NT/2K/XP. The aim is to
get it ready for inclusion in the core. It's main difference from the
existing core tcp driver is the I/O model in use. This uses
overlapped I/O with completion ports over the existing Win3.1 model
of WSAAsyncSelect.
It works great with tclhttpd. As a server, the front-end just DOES NOT
get over-run. In the performance tests I've run, I just can't get the
listening socket to ever fail (see below for the -backlog fconfigure).
With the stock channel driver, connect errors begin around 30 connections
per second. Right now, I can do 80/sec with zero errors and push the test
tool well over that knee without incurring any errors.
Using a simple server and client script that exchange a simple "hello" and
"hello back" with the client doing the graceful shutdown, I'm able to do
1315 exchanges (hits) per second on a 2.4Ghz box running XP. It does full
hardware speed (100mb/s) at around 35% CPU on the receiver.
Per socket resources are now tunable. The new fconfigures are:
-backlog: Sets the size of the AcceptEx pool. Directly relates to
the "bullet-proof" aspect of the front-end. Under
testing, I have found a setting of 700 to be impossible
to ever cause a failure to a client connection (YMMV).
Server socket only. For query, it returns the cap and
actual in-use as a list.
-sendcap: Sets the limit of concurrent WSASend calls allowed on
the socket or for new sockets if this is a server
socket. For query, it returns the cap and actual
in-use in a list.
-recvburst: Sets the cap allowed to the "(A)utomatic (B)urst
(D)etection". If set to one, ABD is turned off in favor
of normal flow control. For server sockets, this setting
is copied to new connections. For query, it returns the
cap and actual in-use in a list.
To see ABD in action: http://iocpsock.sourceforge.net/netio.jpg
As the sender (on the left) increases it's -sendcap, thus allowing
a higher concurrent WSASend count, thus sending rate, the receiver
(on the right) enters a burst condition and increases its WSARecv
count to match the incoming flow. Though not shown, turning off ABD
for normal flow-controlled behavior on the receiver will increase CPU
usage, but works well just the same.
There are essentially two modes one would operate a server under.
Either, one wants high throughput on few connections (FTPD) or a high
amount of short lived connections transferring very little (HTTPD).
The new fconfigures allow for both modes.
My suggestions are:
FTPD HTTPD
--------------------------------------
-backlog 10 500
-sendcap 20 1
-recvburst 20 1
It's amazingly stable. It provides one command called [socket2] and
behaves just like [socket], but with the added fconfigures.
It does IPv6, but is mostly untested in that respect. The -sockname
and -peername fconfigures don't do IPv6 yet, though, but you can
create IPv6 sockets just by using an IPv6 address.
IMPORTANT things to know:
1) Don't set the -buffersize fconfigure to less than 4096. This
limitation will be fixed in a future release. Call me lazy..
2) The readable file event handler is only notified once per event
to read. Because of my design for efficiency, no polling behavior
is used. IOW, when the readable event handler is called (because
some bytes came in to the socket) and the script does not, for some
reason, call [read] or [gets] and no more bytes follow, the readable
event handler will not be called again (unless UpdateInterest() is
called by the generic layer, which is not a guarantee that it will do
so). The same is true for the EOF condition. If the readable event
handler does not close the socket for the [read] that returns an
empty string followed by an [eof] that returns true, the readable
event handler WILL NOT BE CALLED over and over until it does close
the socket unlike the core socket driver which will repeat unhandled
events. FYI, the core socket repeats events because it is a polling
design, just look at SocketSetupProc() in win/tclWinSock.c and you'll
see that each iteration of the event loop must poll every socket
instance open to see if it is in a ready state.
My thoughts on this are simple.. Use a good script for the readable
handler that behaves properly and please.. always check for [eof]
after you [read] or [gets], not before.
Enjoy.. Slam it hard, iocpsock can take it. Special thanks to Lyris
Technologies for allowing this to be open source.
--
David Gravereaux <davy...@pobox.com>
[species: human; planet: earth,milkyway(western spiral arm),alpha sector]
Bruce Hartweg
David Gravereaux wrote:
> IOCPSOCK -- ver 2.0 a/k/a "The 'Got hardware speed?' release".
> (6:50 PM 3/23/2004)
>
> NEW for 2.0: tunable fconfigures.
>
> http://sf.net/project/showfiles.php?group_id=73356
> http://sf.net/projects/iocpsock
>
<snip good things ...>
> IMPORTANT things to know:
>
> 1) Don't set the -buffersize fconfigure to less than 4096. This
> limitation will be fixed in a future release. Call me lazy..
>
I would never call you lazy
> 2) The readable file event handler is only notified once per event
> to read. Because of my design for efficiency, no polling behavior
> is used. IOW, when the readable event handler is called (because
> some bytes came in to the socket) and the script does not, for some
> reason, call [read] or [gets] and no more bytes follow, the readable
> event handler will not be called again (unless UpdateInterest() is
> called by the generic layer, which is not a guarantee that it will do
> so). The same is true for the EOF condition. If the readable event
> handler does not close the socket for the [read] that returns an
> empty string followed by an [eof] that returns true, the readable
> event handler WILL NOT BE CALLED over and over until it does close
> the socket unlike the core socket driver which will repeat unhandled
> events. FYI, the core socket repeats events because it is a polling
> design, just look at SocketSetupProc() in win/tclWinSock.c and you'll
> see that each iteration of the event loop must poll every socket
> instance open to see if it is in a ready state.
>
> My thoughts on this are simple.. Use a good script for the readable
> handler that behaves properly and please.. always check for [eof]
> after you [read] or [gets], not before.
>
what if you read some, but not all of the data? will the event fire again?
I can't think of any good reason for a readable fileevent not read anything,
but I can think of plenty of reasons to only read a portion of available data
and trust the event to keep firing as long as there is more data to read.
Bruce
David Gravereaux
Yes, the generic layer sends a verify through the watchProc after reading
from the inputProc. That part works normally. Just if the script doesn't
call [read] or [gets] at all is when a verify won't come again until new
data arrives. I can understand why the core behaves this way, so it
emulates select() properly.