[ANNOUNCE] async sockets support library
Samual M Rushing
ftp://ftp.nightmare.com/pub/python/python-ext/sockets/async.tar.gz
--------------------------------------------------
I've been sitting on this code for a while, planning to release it
once I got the time to revamp the documentation. I last planned to do
it about a month ago. 8^)
Realizing that I hate writing documentation, and that I'll probably
manage to procrastinate into next year at this rate, I'm going to go
ahead and turn it loose.
--------------------------------------------------
This is a library to assist in writing asynchronous (event-driven)
sockets programs. On Windows, this means using the WSAAsyncSelect
extension, and on Unix, it means select-based multiplexing. The
included modules work in both environments.
Event-driven network programming is ideal for graphical user
interfaces like Windows, X11, etc... where you can't use blocking
operations without making the entire program appear to freeze.
It's also a very elegant way to solve lots of problems that arise in
real-world programs that must manage many connections at once - like
web servers and clients - _without_ the need for threads.
I hope to eventually integrate this with the SocketServer module.
For those of you that have already been using it, the latest additions
(undocumented, of course 8^) include a 'consumer fifo' module that
greatly simplifies the task of building state machines, using a
'continuation' model. I'm very excited about it - it's a powerful
tool for building sophisticated GUI interfaces. [not to be tooting my
own horn. 8^) ]
Also included are lots of small sample modules, servers and clients
for protocols like http, finger, and pop3. 'pop3_2.py' is demo of the
new consumer stuff, it is an rfc1725-compliant pop3 client, complete
with support for the optional extensions like APOP and UIDL.
--------------------------------------------------
You can read the draft documentation at:
http://www.nightmare.com/pythonwin/async_sockets.html
[can't avoid the plug, though...]
For a glimpse of a program that I'm building with this stuff (and
PythonWin, of course), check out:
http://www.nightmare.com/~rushing/nntp_browser.gif
[you'll probably have to pull it up into a separate image viewer,
both netscape and mosaic seem to like to muck it up]
-Sam
--
Sam Rushing The curious task of economics is
rus...@nightmare.com to demonstrate to men how little
http://www.nightmare.com/~rushing they really know about what they
Currently looking for imagine they can design. -F.A.Hayek
contract employment!
Scott Hassan
I tried this module and it seemed to need a few changes to support
asynchronous connect under Linux. Here is a simple patch to
async/unix/asyncore.py.
I am going to make Sylu <http://www-db.stanford.edu/~hassan/SYLU/>
use this package. :) Hopefully, Sylu will run under Windows now!
Scott
--
------------------------------------------------------------------------------
Scott W. Hassan "The trees know something
has...@cs.stanford.edu in their boughs and the
<http://www-db.stanford.edu/~hassan> cats in their prance."
Samual M Rushing
sh> I tried this module and it seemed to need a few changes to
sh> support asynchronous connect under Linux. Here is a simple
sh> patch to async/unix/asyncore.py.
We may be suffering from linux version skew (I'm running 1.2.13 ELF).
After applying the patch, all my demos broke! 8^) [specifically, by
removing non-connected sockets from 'read_fds', I never get a
connection event.]. What linux are you running?
I'm also having trouble with the variant behavior of other Unix socket
implementations [some of them are throwing EWOULDBLOCK when calling
connect()] - which I'm looking into.
There were a few little problems that broke the demos in some cases,
which I've fixed. I've also gone through all the demos on both
Windows and Linux, making sure each one works. And I put together a
README file [in the demo directory] that explains how to run each
of them.
An updated version is now available at
ftp://ftp.nightmare.com/pub/python/python-ext/sockets/async.tar.gz
Note that this still won't work on some (many?) unixen - so far Digital
Unix (and probably all of OSF/1) and SGI Irix - but the demos have
been verified on Windows.
sh> I am going to make Sylu
sh> <http://www-db.stanford.edu/~hassan/SYLU/> use this package. :)
sh> Hopefully, Sylu will run under Windows now!
Great! This will work only with the latest PythonWin... I don't know
about WPY. What socket library does it use? [_socket.pyd + socket.py
should work]
-Sam
James C. Ahlstrom
>about WPY. What socket library does it use? [_socket.pyd + socket.py
>should work]
WPY is moving in the direction of using _your_ socket library! I
am starting to use a DLL access method which enables WPY to use
any available core Python DLL. I haven't finished testing it
but soon you should be able to use WPY with your Py130-1.dll
and access whatever (great!) socket support you have in that DLL.
There is an issue about naming the DLL which I think is important.
Mark was under pressure to name his DLL with its version number,
and to change this name for each new version. This makes it
easy to keep track of versions. But to use that DLL, WPY must
know its name. If this name keeps changing, I must (a) compile in
a lengthy list of possible names, (b) search for a name pattern on
the (complicated) path for DLLs, (c) include some hack for informing
WPY of the correct DLL to import (such as a command line option).
This is usually handled by using a constant DLL name. For example,
there are many versions of "winsock.dll" all with exactly this
name. It makes it harder to keep track of which version you have,
but any app can get socket support by loading exactly "winsock.dll",
and you can upgrade your winsock without changing any apps.
I would prefer that the Python core DLLs each have a constant name,
but I have no solution to the version problem which Mark tried to
solve.
Comments and solutions please.
Jim
Skip Montanaro
[ lament about DLL names deleted ... ]
I would prefer that the Python core DLLs each have a constant name,
but I have no solution to the version problem which Mark tried to
solve.
Comments and solutions please.
Jim, it sounds like you need symbolic links... :-)
Skip Montanaro | Looking for a place to promote your music venue, new CD,
sk...@calendar.com | festival or next concert tour? Place a focused banner
(518)372-5583 | ad in Musi-Cal! http://concerts.calendar.com/
----- Support free speech online: http://www.eff.org/blueribbon.html -----
James C. Ahlstrom
Well, the quick answer is that Windows doesn't have symbolic
links. But the real answer is more complicated. AFAIK Windows
uses the file name to decide whether the DLL has already been
loaded. Suppose you link Py130-1.dll to Py130-2.dll so that
WPY can load the old name Py130-1.dll. And suppose socket.pyd
was linked with Py130-2.dll (as it must be since currently
you must update all *.pyd files for each new core dll). Then
when WPY runs, Windows will load Py130-1.dll, but when
socket.dll is loaded, it will load Py130-2.dll. So you
wind up with two Pythons!
Jim
Dan Connolly
This is great stuff! Yet another step towards nice clean programming
platforms for the net. [I think... I actually haven't looked at
it closely yet.]
I maintain that concurrency support is _essentail_ for distributed
applications development (see: http://www.w3.org/pub/WWW/OOP/HyperMediaDev)
and that threads are a convenient and powerful programming model.
The M3 threads interface is my favorite:
http://www.research.digital.com/SRC/m3sources/html/thread/src/Common/Thread.i3.html
(it's also documented extensively in tech reports and the SPWM3 book).
I/O, especially network I/O, is a critical source of concurrency
to exploit.
I think python should support the M3 threads interface. But the
existing base of C-code modules aren't thread-safe. So adding
preemptive multitasking to python would be a complete overhaul.
But preemptive threading isn't the only way. Mac system 7 and Win3.1
show that cooperative multitasking can work.
I think coroutines can be added to python in a straightforward way.
The only tricky parts are the I/O interactions. And Mr. Rushing
seems to have done the bulk of the work.
This would allow folks to code using the convenient concurrency idioms
(threads, mutexes, condition variables) on top of most of the existing
python modules.
I've been trying to get Guido to put this high on the priority
list for some time.
See, for example:
http://www.cweb.com/contrib/psamail/95/11/msg0288.mail
I think the lack of this sort of concurrency support complicates
the Grail source immensely, and I really want Grail to be easily
hackable.
In article <x6lokv9...@squirl.nightmare.com> rus...@squirl.nightmare.com (Samual M Rushing) writes:
> From: rus...@squirl.nightmare.com (Samual M Rushing)
> Newsgroups: comp.lang.python
> Date: 21 Mar 1996 01:50:17 -0500
> Organization: Nightmare Software, UnInc.
>
> Available from
> ftp://ftp.nightmare.com/pub/python/python-ext/sockets/async.tar.gz
> --------------------------------------------------
[...]
> This is a library to assist in writing asynchronous (event-driven)
> sockets programs. On Windows, this means using the WSAAsyncSelect
> extension, and on Unix, it means select-based multiplexing. The
> included modules work in both environments.
>
> Event-driven network programming is ideal for graphical user
> interfaces like Windows, X11, etc... where you can't use blocking
> operations without making the entire program appear to freeze.
--
Daniel W. Connolly "We believe in the interconnectedness of all things"
Research Scientist, MIT/W3C PGP: EDF8 A8E4 F3BB 0F3C FD1B 7BE0 716C FF21
<conn...@w3.org> http://www.w3.org/pub/WWW/People/Connolly/
Guido van Rossum
> the Grail source immensely, and I really want Grail to be easily
> hackable.
Yes, Dan, you're right. Now that POSIX threads have been standardized
I'll be less reluctant to require threads. Just wait... (Grail 0.3
won't have threads yet sorry. It's almost done anyway.)
--Guido van Rossum <gu...@CNRI.Reston.VA.US>
URL: <http://www.python.org/~guido/>
Hadar Pedhazur
[stuff deleted]
> This is usually handled by using a constant DLL name. For example,
> there are many versions of "winsock.dll" all with exactly this
> name. It makes it harder to keep track of which version you have,
> but any app can get socket support by loading exactly "winsock.dll",
> and you can upgrade your winsock without changing any apps.
> I would prefer that the Python core DLLs each have a constant name,
> but I have no solution to the version problem which Mark tried to
> solve.
> Comments and solutions please.
I am sure I am missing something simple, but can you not just use the
registry in Win95 and NT? You look up a well-known key, and if it is
not there, use your default name. Users can update the registry with
the name of their preferred DLL if/when a newer version is available.
--
Hadar Pedhazur
Global Equity Derivatives
Union Bank of Switzerland (UBS)
Mark Hammond
> registry in Win95 and NT? You look up a well-known key, and if it is
> not there, use your default name. Users can update the registry with
> the name of their preferred DLL if/when a newer version is available.
in the .EXE. Usually, the DLL name _is_, as the .lib file is used by
the link. Your solution would work, but would require every entry
point in python.dll to be manually extracted via LoadProcAddress...
Mark.
----------------------------------------
Mark Hammond - MHam...@skippinet.com.au
James C. Ahlstrom
Mark Hammond <MHam...@skippinet.com.au> wrote:
>> I am sure I am missing something simple, but can you not just use the
>> registry in Win95 and NT? You look up a well-known key, and if it is
>> not there, use your default name. Users can update the registry with
>> the name of their preferred DLL if/when a newer version is available.
>Thats a good idea. However, it implies the DLL name is not embedded
>in the .EXE. Usually, the DLL name _is_, as the .lib file is used by
>the link. Your solution would work, but would require every entry
>point in python.dll to be manually extracted via LoadProcAddress...
I think that actually works. I am not using a .lib file, I use
LoadLibrary() which just needs a string name. And Mark is right,
I must call GetProcAddress() for every entry point. Good idea.
Jim
Mark Hammond
> LoadLibrary() which just needs a string name. And Mark is right,
> I must call GetProcAddress() for every entry point. Good idea.
Problem is Python.dll exports around 300 entry points. Although the
majority of extension modules only need a select few, there is no way
of anticipating future .pyd requirements...
Also, I'd be interested to know how you handle data items?