Launching Multiple Servers
Salvatore
Given a class named ServerCreator,
is it possible to create multiple servers on different port and
managing each of them separately, using SocketServer module ?
Thanks your responses
and even if you don't respond ;-)
Regards
Salvatore
Norbert Klamann
> Hello,
>
> Given a class named ServerCreator,
> is it possible to create multiple servers on different port and
> managing each of them separately, using SocketServer module ?
Why SocketServer ? Wouldn't 'Twisted' do the trick ? It delivers a lot
of the necessary infrastructure to do these things the right way(tm).
>
> Thanks your responses
HTH
> and even if you don't respond ;-)
>
> Regards
>
> Salvatore
All the best
Norbert
us...@domain.invalid
> Why SocketServer ? Wouldn't 'Twisted' do the trick ? It delivers a lot
> of the necessary infrastructure to do these things the right way(tm).
Twisted does certainly the trick, but i must admit that is seems complex
and the documentation is poor.
If you know some good doc. i'm your client
Regards
Salvatore
Andrew Bennetts
Twisted isn't so much complex as *large*. The trick is knowing which bits
matter to you, and which parts to ignore -- unfortunately, or docs are
fairly mediocre at helping you with that :(
Regardless, I think Twisted is worth investing a little time into. The
essentials for writing servers are here:
http://twistedmatrix.com/documents/howto/servers
That doc doesn't mention this, but listening on multiple ports is easy; you
can simply do:
reactor.listenTCP(10001, factory1)
reactor.listenTCP(10002, factory2)
# etc...
You can even use the same factory on multiple ports.
Feel free to ask for help on twisted...@twistedmatrix.com, or on
#twisted on irc.freenode.net -- there's usually someone around who's happy
to help.
-Andrew.
us...@domain.invalid
> Regardless, I think Twisted is worth investing a little time into. The
> essentials for writing servers are here:
> http://twistedmatrix.com/documents/howto/servers
>
> That doc doesn't mention this, but listening on multiple ports is easy; you
> can simply do:
>
> reactor.listenTCP(10001, factory1)
> reactor.listenTCP(10002, factory2)
> # etc...
>
> You can even use the same factory on multiple ports.
>
> Feel free to ask for help on twisted...@twistedmatrix.com, or on
> #twisted on irc.freenode.net -- there's usually someone around who's happy
> to help.
>
> -Andrew.
Thank you very much Andrew, now i'm sure i'll have some help
i will investigate more
regards
Salvatore
Peter Hansen
>
> On Thu, May 22, 2003 at 09:43:31AM +0200, us...@domain.invalid wrote:
> > >Norbert Klamann wrote
> >
> >
> > >Why SocketServer ? Wouldn't 'Twisted' do the trick ? It delivers a lot
> > >of the necessary infrastructure to do these things the right way(tm).
> >
> > Twisted does certainly the trick, but i must admit that is seems complex
> > and the documentation is poor.
> > If you know some good doc. i'm your client
>
> Twisted isn't so much complex as *large*. The trick is knowing which bits
> matter to you, and which parts to ignore -- unfortunately, or docs are
> fairly mediocre at helping you with that :(
I don't entirely agree. Yes, Twisted is large, but there isn't really
any trick needed. Just read the necessary How-Tos (such as on how to
make a simple TCP server) and (with a basic background in sockets) you
should have a working basic multiple-server app within a half hour or
less. Heck, just use the example code and run it: it works.
There's no reason to worry about the size of Twisted, or figure out
"what to ignore". There's an incredible amount to Twisted (I assume)
beyond what I've used so far, but that fact has not in any way
inhibited my work as Twisted is so well structured.
Maybe I'm weird: I even found the docs to be pretty good.
> Regardless, I think Twisted is worth investing a little time into.
Definitely! :)
-Peter
Steve Holden
> > Regardless, I think Twisted is worth investing a little time into.
>
> Definitely! :)
>
Me too!
[cough] Might I just take the opportunity to advertise two relevant
tutorials those attending OSCON might want to consider? My own is "Network
Programming in Python", described at
http://conferences.oreillynet.com/cs/os2003/view/e_sess/4165
and the same afternoon Itamar Shtull-Trauring is giving "Network Programming
with the Twisted Framework", see
http://conferences.oreillynet.com/cs/os2003/view/e_sess/3905
I'm planning to attend Itamar's session, it looks like it will be what I
need to get me started on Twisted.
regards
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/pwp/
Andrew Bennetts
> Andrew Bennetts wrote:
> >
> > Twisted isn't so much complex as *large*. The trick is knowing which bits
> > matter to you, and which parts to ignore -- unfortunately, or docs are
> > fairly mediocre at helping you with that :(
>
> I don't entirely agree. Yes, Twisted is large, but there isn't really
> any trick needed. Just read the necessary How-Tos (such as on how to
> make a simple TCP server) and (with a basic background in sockets) you
> should have a working basic multiple-server app within a half hour or
> less. Heck, just use the example code and run it: it works.
>
> There's no reason to worry about the size of Twisted, or figure out
> "what to ignore". There's an incredible amount to Twisted (I assume)
> beyond what I've used so far, but that fact has not in any way
> inhibited my work as Twisted is so well structured.
>
> Maybe I'm weird: I even found the docs to be pretty good.
We get mixed feedback. Some people do seem to get overwhelmed by the
breadth of Twisted, and have trouble finding what they need to know. Other
people, like yourself, seem to manage just fine.
This is mainly conjecture on my part, but as far as I can tell, the main
reason why some people have trouble is they can't narrow down the docs and
examples to what matters most to them. We have a fairly large amount of
terminology, which can leave people wondering "What the hell is 'reactor',
'factory', 'woven', 'spread', 'pb', 'TAP', 'Deferred'..." (which is why I
wrote the glossary).
Our docs aren't structured in such a way that you can say "I want to find
out how to implement a new protocol" or "I want to deploy a Twisted Web
server", and immediately know which How-Tos are relevant to you. And the
example code is very poorly categorised, and in places could do with more
comments describing what they demonstrate.
Despite all that, the docs at least fairly comprehenisive now[1] (over 250
pages as a PDF!), and they do seem to be enough for at least some people.
I'd think I'd rate them as "ok" rather than "pretty good"; but I think it's
healthy for developers to be a little harsh about their own work <wink>.
-Andrew.
[1] When I started using Twisted, I had to learn pretty much everything by
asking on IRC; even the "Writing Servers" document didn't exist. Of
course, that might be because I'm unusually thick rather than because
the docs were unusually sparse... :)
Peter Hansen
>
> On Thu, May 22, 2003 at 08:45:29AM -0400, Peter Hansen wrote:
> > Yes, Twisted is large, but there isn't really
>
> We get mixed feedback. Some people do seem to get overwhelmed by the
> breadth of Twisted, and have trouble finding what they need to know. Other
> people, like yourself, seem to manage just fine.
>
> This is mainly conjecture on my part, but as far as I can tell, the main
> reason why some people have trouble is they can't narrow down the docs and
> examples to what matters most to them.
That's probably true. And *I* would conjecture that it's because many
people don't really know *what* they want when they start with something
like Twisted. Actually, the more general rule is that most people don't
really know what they want, period, and it's a problem at the root of the
sorry state of software development (most projects failing, poor quality, etc.).
I would suggest that one reason why I have no difficulty diving in and
almost immediately getting useful results from something as broad as
Twisted (or, say, wxPython, or Zope) is that my goals are -- as a direct
result of adopting the XP mentality -- very focused and simple.
When I started with Twisted, I didn't want to build a massively distributed
multiplayer game with interfaces to MSN and Kerberos or anything like that.
I just wanted to be able to accept a socket connection and disconnect if a
particular pattern was not received within a timeout.
Only after I got that much working did I start thinking about the next
step, which in this case was to add a check for "too much data received
before authentication", and then "after authentication, connect to this
other thread for further communication". Short, simple stories, easily
implemented (whether with raw sockets, Medusa, or Twisted).
As a result of taking baby steps like this, I knew exactly which Howtos
interested me initially (specifically, "Writing a server" under the
Low-Level Twisted section). The example code in the Howto looked like
it should work, so of course the first step would be to verify my
Twisted installation and the Howto itself by getting the examples to
execute properly, which it did with no muss or fuss.
At that point, the "Scheduling" Howto seemed particular relevant for
figuring out how to disconnect after a timeout, and Lo, there was the
reactor.callLater() method.
At no point did I worry about the presence of "Cred", or "IM", or
"Spread", since they didn't seem likely to be of help at that point.
Heck, I still don't really know what they're supposed to do, except
in a vague, hand-waving kind of way.
To anyone struggling with something like Twisted, feeling that it's
"too complex" to get a handle on, I strongly recommend learning more
about agile methods of development, specifically Extreme Programming (XP).
I'm confident that if you do, you will no longer look on something
like Twisted with trepidation, but instead with great relief, knowing
that most of the capability you might need will, at the time you need
it, reveal itself with fairly little effort on your part.
Given the agile manner in which Twisted is developed, maybe it's not
surprising that it can be easily adopted by someone else following an
agile approach...
-Peter
Andrew Dalke
> This is mainly conjecture on my part, but as far as I can tell, the main
> reason why some people have trouble is they can't narrow down the docs and
> examples to what matters most to them. We have a fairly large amount of
> terminology, which can leave people wondering "What the hell is 'reactor',
> 'factory', 'woven', 'spread', 'pb', 'TAP', 'Deferred'..." (which is why I
> wrote the glossary).
I, for one, agree with that. I understand names like 'reactor' (because of
Sam Rushing ;) and 'factory' - it's the Twisted-specific names which
throw me for the loop. "pb" means nothing to me. I remember the
reference to "peanut butter" which induced the names "jelly" and "banana",
but then I forget which is which. Even its expansion to "Perspective
Broker"
doesn't help me much. I would rather prefer the name "Proxy Broker"...
assuming that's what it does.
> And the
> example code is very poorly categorised, and in places could do with more
> comments describing what they demonstrate.
Or that the code does unexpected things. I was experimenting with it
and found that *importing* Twisted overrode how logging is done. My
import warnings (a 2.2 extension used under 2.3) were going into
never-never land. I tracked that down and I believe it's fixed in the
current code.
In general, I got the feeling that Twisted does its own thing. That is,
it uses its own names, it override at least some ways Python does
things, and I noticed it tried to fake a "sychronized" keyword for
some of the logging, rather than use a more Pythonic Queue. It
even has its own VMS-style command-arg parsing (including
the automatic prefix expansion I don't like) rather than use Python's.
Admittedly, the logging code I looked at is considered old, so perhaps
feelings have changed, but there was talk of writing a new logging facility
because the one to be included in Python 2.3 wasn't good enough,
which again suggests a tendency towards idiosyncratic code.
That doesn't mean I won't use it, I just suggest that this tendency to
write code with non-standard behaviour (wrt "Pythonic" code) makes
it harder for some people to use Twisted.
> [1] When I started using Twisted, I had to learn pretty much everything by
> asking on IRC; even the "Writing Servers" document didn't exist.
I've done IRC now 3 times, all for Twisted. IRC just doesn't really fit
my mind, which has been warped by Usenet-style interactions.
Andrew
da...@dalkescientific.com
Andrew Bennetts
> Andrew Bennetts:
> > This is mainly conjecture on my part, but as far as I can tell, the main
> > reason why some people have trouble is they can't narrow down the docs and
> > examples to what matters most to them. We have a fairly large amount of
> > terminology, which can leave people wondering "What the hell is 'reactor',
> > 'factory', 'woven', 'spread', 'pb', 'TAP', 'Deferred'..." (which is why I
> > wrote the glossary).
>
> I, for one, agree with that. I understand names like 'reactor' (because of
> Sam Rushing ;) and 'factory' - it's the Twisted-specific names which
> throw me for the loop. "pb" means nothing to me. I remember the
> reference to "peanut butter" which induced the names "jelly" and "banana",
> but then I forget which is which. Even its expansion to "Perspective
> Broker"
> doesn't help me much. I would rather prefer the name "Proxy Broker"...
> assuming that's what it does.
No, "Perspective Broker" is a sane name for it, as a PB server is a broker
for "perspectives". There's now a doc regarding Perspectives that's not
tied to PB (they're actually a Cred concept), which might help you a little:
http://twistedmatrix.com/documents/howto/pclients
Most people seem to find the PB docs adequate to comprehending at least
basic PB -- although I don't think the sandwich puns help many people. Not
to mention the North American bias -- Australians like myself don't expect
to find jelly on a sandwich!
If you don't need a remote object protocol, you can safely ignore
twisted.spread, and it's associated sandwich modules. (But for reference,
pb sits upon jelly sits upon banana...)
> > And the
> > example code is very poorly categorised, and in places could do with more
> > comments describing what they demonstrate.
>
> Or that the code does unexpected things. I was experimenting with it
> and found that *importing* Twisted overrode how logging is done. My
> import warnings (a 2.2 extension used under 2.3) were going into
> never-never land. I tracked that down and I believe it's fixed in the
> current code.
I'm not an expert in this code, but twisted.python.log has been
reimplemented in 1.0.5, and presumably it's better ;)
(Also, there's code in the sandbox that implements the old log API using the
logging module of Python 2.3; perhaps that will be updated for our new log
code at some point).
> In general, I got the feeling that Twisted does its own thing. That is,
> it uses its own names, it override at least some ways Python does
> things, and I noticed it tried to fake a "sychronized" keyword for
> some of the logging, rather than use a more Pythonic Queue. It
> even has its own VMS-style command-arg parsing (including
> the automatic prefix expansion I don't like) rather than use Python's.
Python 2.3 isn't released yet, and I find writing a
twisted.python.options.Usage subclass with some attributes far nicer and
more readable than writing the boilerplate code that getopt requires. I
think in this case having our own command-line parser was reasonable --
Twisted originally supported 1.5.2, after all, and its users certainly
didn't have an optparse module :)
I'm not sure what you're referring to with "automatic prefix expansion I
don't like"?
(There's also code in the sandbox to implement twisted.python.options using
optparse.)
> Admittedly, the logging code I looked at is considered old, so perhaps
> feelings have changed, but there was talk of writing a new logging facility
> because the one to be included in Python 2.3 wasn't good enough,
> which again suggests a tendency towards idiosyncratic code.
Agreed. I've heard both sides of this argument, but I don't know enough to
be able to comment usefully on it. One big problem is that Twisted supports
2.2.0, and so requiring the 2.3 logging package would burden users with an
extra dependency (ditto optparse/Optik).
> That doesn't mean I won't use it, I just suggest that this tendency to
> write code with non-standard behaviour (wrt "Pythonic" code) makes
> it harder for some people to use Twisted.
We've certainly reimplemented one or two (or ten...) wheels here and there.
Another big one you haven't mentioned is we have our own XML parser and DOM
implementation. I've even seen the phrase "Twisted: The Batteries Python
Forgot" used in jest.
On the other hand, asyncore already did asynchronous networking for Python,
and we've reimplemented that too -- but I think the result is something much
better than asyncore. If we've reinvented something, there will be a reason
for it. But as you point out, it can make life harder for our users.
> > [1] When I started using Twisted, I had to learn pretty much everything by
> > asking on IRC; even the "Writing Servers" document didn't exist.
>
> I've done IRC now 3 times, all for Twisted. IRC just doesn't really fit
> my mind, which has been warped by Usenet-style interactions.
I know how you feel -- some people want me to use some form of instant
messaging (in addition to IRC!), and it just doesn't quite feel right to me
:)
-Andrew.
Andrew Dalke
> No, "Perspective Broker" is a sane name for it, as a PB server is a broker
> for "perspectives". There's now a doc regarding Perspectives that's not
> tied to PB (they're actually a Cred concept), which might help you a
little:
>
> http://twistedmatrix.com/documents/howto/pclients
Okay, I'll take a look at that. Thanks!
> Not
> to mention the North American bias -- Australians like myself don't expect
> to find jelly on a sandwich!
I like neither peanut butter nor bananas on a sandwich. Straight jelly
for me. Peanut butter goes on a graham cracker, and bananas are
eaten alone (just when the skin starts getting speckled), embedded in
gelatin (preferably cherry flavored), or in banana bread. (The latter
is also very good with jelly.)
> I'm not an expert in this code, but twisted.python.log has been
> reimplemented in 1.0.5, and presumably it's better ;)
After my prodding ;)
> I'm not sure what you're referring to with "automatic prefix expansion I
> don't like"?
Twisted expands an option given it's prefix. Suppose you have
an option like "--logfile", with no other options starting with "l" or
"lo" or "...". Then as I read the Twisted docs, it will automatically
expand "--log" to "--logfile". This is what VMS did as well.
It seems good, but the problem is that people write scripts which
assume things will never change, so assume "--log" is the 'correct'
option.
Now add a new feature enabled by '--login'. This also has the
prefix "--log", so will break scripts which expected '--log' to
match '--logfile' -- unless you explicitly add all the prefixes or
put in a "when multiple prefixes exist, prefer this one" option.
This makes things more confusing, so I would rather not allow
prefix support.
So my complaint about writing a new command-arg parser
in Twisted is not that a new one exists -- optparse shows that
getopt has problems -- it's that the behaviour implemented is
quite different from how Python or other packages work, and
is a behaviour that I would rather not have.
> One big problem is that Twisted supports
> 2.2.0, and so requiring the 2.3 logging package would burden users with an
> extra dependency (ditto optparse/Optik).
Both the logging and optparse modules support older versions.
Include the code in Twisted, and in a few releases issues a
deprecated warning with a "from optparse import *" as the
implementation.
> On the other hand, asyncore already did asynchronous networking for
Python,
> and we've reimplemented that too -- but I think the result is something
much
> better than asyncore. If we've reinvented something, there will be a
reason
> for it. But as you point out, it can make life harder for our users.
I don't doubt there's a reason. I worry that the reason doesn't always
consider the impact it has on people who aren't the implementors.
Andrew
Andrew Bennetts
> Andrew Bennetts:
>
> > reimplemented in 1.0.5, and presumably it's better ;)
>
> After my prodding ;)
Yes -- thank you! :)
> > I'm not sure what you're referring to with "automatic prefix expansion I
> > don't like"?
>
> Twisted expands an option given it's prefix. Suppose you have
> an option like "--logfile", with no other options starting with "l" or
> "lo" or "...". Then as I read the Twisted docs, it will automatically
> expand "--log" to "--logfile". This is what VMS did as well.
> It seems good, but the problem is that people write scripts which
> assume things will never change, so assume "--log" is the 'correct'
> option.
Ah, I see. You're right, it does do that. I never even realised that it
did that. On further investigation, this actually happens inside the getopt
module!
> Now add a new feature enabled by '--login'. This also has the
> prefix "--log", so will break scripts which expected '--log' to
> match '--logfile' -- unless you explicitly add all the prefixes or
> put in a "when multiple prefixes exist, prefer this one" option.
>
> This makes things more confusing, so I would rather not allow
> prefix support.
I'm inclined to agree. Having two ways to do something is already arguably
at least one way too many ;)
> So my complaint about writing a new command-arg parser
> in Twisted is not that a new one exists -- optparse shows that
> getopt has problems -- it's that the behaviour implemented is
> quite different from how Python or other packages work, and
> is a behaviour that I would rather not have.
Well, twisted.python.usage uses getopt internally, presumably to help ensure
it's behaviour is reasonably consistent with existing python programs... so
blame getopt, not Twisted! ;)
> Both the logging and optparse modules support older versions.
> Include the code in Twisted, and in a few releases issues a
> deprecated warning with a "from optparse import *" as the
> implementation.
I'm inclined to agree, but obviously I can't speak for the rest of the
Twisted team.
> > On the other hand, asyncore already did asynchronous networking for
> > Python, and we've reimplemented that too -- but I think the result is
> > something much better than asyncore. If we've reinvented something,
> > there will be a reason for it. But as you point out, it can make life
> > harder for our users.
>
> I don't doubt there's a reason. I worry that the reason doesn't always
> consider the impact it has on people who aren't the implementors.
I think we're very conscious of the impact these sorts of decisions cause
our users. That doesn't guarantee we make the right decisions, but we do at
least try :)
-Andrew.
Dave Benjamin
> I'm planning to attend Itamar's session, it looks like it will be what I
> need to get me started on Twisted.
Once you're up to speed, maybe you could squeeze in a bit about Twisted into
a second edition of PWP. The framework seems to be getting quite a bit of
attention lately, and it's a viable alternative to the asyncore library.
Just a thought. =)
Peace,
Dave