Suggestions for quick communications?
Nathan
where the server and client talk to each other over the network.
My initial implementation uses the built-in socket module, but I can't
find a way to flush the socket! Any suggestions for a simple way for
fast communications, or a way to make sockets work fast? (right now,
the sockets seem to just wait about 2 seconds, and then transmit,
which is rather funny to watch, but not very playable).
~ Nathan
Drew Smathers
>
> I'm attempting to write a simple little real-time server-client game
> where the server and client talk to each other over the network.
>
There is no such thing as a simple real-time server-client game. In
less you mean a not-really-real-time (TM) server-client game.
> My initial implementation uses the built-in socket module, but I can't
> find a way to flush the socket! Any suggestions for a simple way for
> fast communications, or a way to make sockets work fast? (right now,
> the sockets seem to just wait about 2 seconds, and then transmit,
> which is rather funny to watch, but not very playable).
>
>
Just use Twisted. Then you might actually come close to the "simple"
objective. Nonetheless, you might want to google TCP_NODELAY for
your particular problem.
Cheers,
--
\\\\\/\"/\\\\\\\\\\\
\\\\/ // //\/\\\\\\\
\\\/ \\// /\ \/\\\\
\\/ /\/ / /\/ /\ \\\
\/ / /\/ /\ /\\\ \\
/ /\\\ /\\\ \\\\\/\
\/\\\\\/\\\\\/\\\\\\
d.p.s
Nathan
>
> On Thu, Feb 21, 2008 at 10:50 AM, Nathan <nathan...@gmail.com> wrote:
> >
> > I'm attempting to write a simple little real-time server-client game
> > where the server and client talk to each other over the network.
> >
>
> There is no such thing as a simple real-time server-client game. In
> less you mean a not-really-real-time (TM) server-client game.
I'm aware of the realities. I just meant it's not a turn-based game
(for example) where 2-second lag would be acceptable.
> > My initial implementation uses the built-in socket module, but I can't
> > find a way to flush the socket! Any suggestions for a simple way for
> > fast communications, or a way to make sockets work fast? (right now,
> > the sockets seem to just wait about 2 seconds, and then transmit,
> > which is rather funny to watch, but not very playable).
> >
>
> Just use Twisted. Then you might actually come close to the "simple"
> objective. Nonetheless, you might want to google TCP_NODELAY for
> your particular problem.
Twisted and "simple" have nothing to do with each other in my
experience. I even bought the Twisted book, but I have a really hard
time getting into it (not to mention that the developers seem to live
on the opposite side of the world and have odd attitudes when you
start asking them questions). I really don't want to deal with a
completely separate event loop. I'd also like to avoid external
dependencies where possible. (Right now I only depend on python
itself and pyglet)
I'm researching the TCP_NODELAY option, which looks promising, but
there's a curious lack of explanation of those types of flags and how
to use the on http://docs.python.org/lib/module-socket.html --- I
suppose they assume you'll come with socket knowledge from C.
Referencing these sources...
http://www.scribd.com/doc/134861/Sockets-Programming-with-python
http://mail.python.org/pipermail/python-list/2006-April/378814.html
...I tried doing the following in my code, with no observable effect
(still ~1-2 sec lag):
(-- server-side snippets --)
def player_thread(conn):
conn.setsockopt(socket.SOL_SOCKET, socket.TCP_NODELAY, 1)
[snip]
[snip]
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((HOST, PORT))
sock.listen(0)
[snip]
# Create player_thread's as players connect
try:
while True:
conn, addr = sock.accept()
thread_id = thread.start_new_thread(player_thread, (conn,))
except Exception, e:
print e
finally:
sock.close()
(-- client-side snippets ---)
def server_connection(keystate_queue, updates_queue):
# We send keystate to the server
# The server sends updates to us
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.connect((HOST, PORT))
conn.setsockopt(socket.SOL_SOCKET, socket.TCP_NODELAY, 1)
[snip]
~ Nathan
Drew Smathers
>
> On Thu, Feb 21, 2008 at 9:00 AM, Drew Smathers <drew.s...@gmail.com> wrote:
> >
> > On Thu, Feb 21, 2008 at 10:50 AM, Nathan <nathan...@gmail.com> wrote:
> > >
> > > I'm attempting to write a simple little real-time server-client game
> > > where the server and client talk to each other over the network.
> > >
> >
> > There is no such thing as a simple real-time server-client game. In
> > less you mean a not-really-real-time (TM) server-client game.
>
> I'm aware of the realities. I just meant it's not a turn-based game
> (for example) where 2-second lag would be acceptable.
>
>
> > > My initial implementation uses the built-in socket module, but I can't
> > > find a way to flush the socket! Any suggestions for a simple way for
> > > fast communications, or a way to make sockets work fast? (right now,
> > > the sockets seem to just wait about 2 seconds, and then transmit,
> > > which is rather funny to watch, but not very playable).
> > >
> >
> > Just use Twisted. Then you might actually come close to the "simple"
> > objective. Nonetheless, you might want to google TCP_NODELAY for
> > your particular problem.
>
> Twisted and "simple" have nothing to do with each other in my
> experience.
I'm sorry to hear that :( Do you think low-level socket programming is simple?
> I even bought the Twisted book, but I have a really hard
> time getting into it (not to mention that the developers seem to live
> on the opposite side of the world and have odd attitudes when you
> start asking them questions).
Really? I've found the people on the Twisted mailing list to be very
helpful and take a lot of time to explain features of the library or
provide links to the appropriate documentation. The same can be said
of the IRC channel. Questions generally don't go ignored. So please
don't go hating on people who are working hard to answer questions day
in and day out (for a nominal fee of $0 per question), and continue to
write and maintain awesome open source software.
In terms of what side of the world they're on? I think it's the side
where people understand concurrency and network programming very, very
well.
Austin Wood
>
> On Thu, Feb 21, 2008 at 9:00 AM, Drew Smathers <drew.s...@gmail.com> wrote:
> >
> > On Thu, Feb 21, 2008 at 10:50 AM, Nathan <nathan...@gmail.com> wrote:
> > > My initial implementation uses the built-in socket module, but I can't
> > > find a way to flush the socket! Any suggestions for a simple way for
> > > fast communications, or a way to make sockets work fast? (right now,
> > > the sockets seem to just wait about 2 seconds, and then transmit,
> > > which is rather funny to watch, but not very playable).
> > >
> >
> > Just use Twisted. Then you might actually come close to the "simple"
> > objective. Nonetheless, you might want to google TCP_NODELAY for
> > your particular problem.
>
> Twisted and "simple" have nothing to do with each other in my
> experience. I even bought the Twisted book, but I have a really hard
> time getting into it (not to mention that the developers seem to live
> on the opposite side of the world and have odd attitudes when you
> start asking them questions). I really don't want to deal with a
> completely separate event loop. I'd also like to avoid external
> dependencies where possible. (Right now I only depend on python
> itself and pyglet)
>
> I'm researching the TCP_NODELAY option, which looks promising, but
> there's a curious lack of explanation of those types of flags and how
> to use the on http://docs.python.org/lib/module-socket.html --- I
> suppose they assume you'll come with socket knowledge from C.
The socket module in Python is a direct copy of the C (POSIX) socket
API (modulo struct sizes). So they do assume you have socket knowledge
from C.
If you want low-latency you'll need to use UDP instead of TCP (that
is, SOCK_DGRAM instead of SOCK_STREAM).
TCP is a reliable two-way communication protocol so packets are
guaranteed to reach the other host *in order*.
UDP is connectionless and one-way (iirc)
TCP is slow because, basically, 2 reasons;
1- Every packet received needs an ACKnowledgement sent back.
2- Nagle's algorithm which delays tiny packets to increase throughput
(but increases latency)
To use UDP properly you'll need to write a networking layer that can
deal with dropped and out-of-order packets.
HTH
Drew Smathers
Which is what TCP_NODELAY is for (2).
> To use UDP properly you'll need to write a networking layer that can
> deal with dropped and out-of-order packets.
And to make a long story short, you will fail miserably at this like
thousands of other programmers who heard UDP is super-duper fast and
thought it would be *fun* to make a real-time networked game based on
UDP. Or you will succeed only partially with an implementation that
completely sucks and works slower than TCP. Either way, you will
spend a lot of time debugging the UDP side of your application and
have little time leftover to have fun writing your game.
Assuming you want to maintain your sanity, just stick to TCP.
Alec Thomas
Try Pyraknet [1]. There are several sample projects on the site, and
raknet (a C network abstraction library) itself is a commonly used
library in gaming circles.
As for the standard library, you probably want to look at
asyncore/asynchat. But TBH, I'd use an existing library. Networking
code is notorious for being easy to start and difficult to perfect.
Simon Wittber
for me.
http://exactlysimilar.org/downloads/socketome-0.05.tar.gz
Some (poor) docs here:
http://exactlysimilar.org/docs/socketome/
I won't make any claims to ease of use or suitability, but I will say
I wrote it because I don't like wrestling with Twisted, and I like to
control my own run loop.
I also agree with other posters comments re: UDP. It's hard work, and
you'll most likely end up reimplementing TCP.
Nathan
> > Twisted and "simple" have nothing to do with each other in my
> > experience.
>
> I'm sorry to hear that :( Do you think low-level socket programming is simple?
>
> > I even bought the Twisted book, but I have a really hard
> > time getting into it (not to mention that the developers seem to live
> > on the opposite side of the world and have odd attitudes when you
> > start asking them questions).
>
> Really? I've found the people on the Twisted mailing list to be very
> helpful and take a lot of time to explain features of the library or
> provide links to the appropriate documentation. The same can be said
> of the IRC channel. Questions generally don't go ignored. So please
> don't go hating on people who are working hard to answer questions day
> in and day out (for a nominal fee of $0 per question), and continue to
> write and maintain awesome open source software.
Well, I suppose it's time for me to give Twisted a try again. I had
my bad experiences with it back in 2005, so hopefully much has changed
since then. I'll join the ML this time instead of poking around IRC.
At least the twisted web site has graphics this time around, so maybe
they have some decent docs too...
So any advice on the competing event loops?
~ Nathan
David
Simon's comment, but is not addressed specifically to Simon.
If your reason for not using Twisted is in wanting to own your
main-loop, and your main-loop requirements include that it be
restartable, you might take a look at my 'PausingReactor'.
It is a Twisted reactor, derived from the default selectReactor, that
is written around a generator to allow exiting and restarting.
It has .release() and .resume() methods to get you out of, and back
into, the reactor. So, even though, the Twisted reactor is strictly
*run-once*, you can treat it as though it were exitable and
restartable. The reactor state is held in the generator while
execution continues outside the reactor.
There is a test script for it, but I admit I have not made extensive
use of it just yet.
The source code (it's brief) is cut-and-pasteable from the post:
http://pitchersduel.wordpress.com/2007/12/09/pausingreactorpy/
A description and usage example are here:
http://pitchersduel.wordpress.com/2007/12/07/pausing-reactor/
This does not address every objection to Twisted, for example, it is
still not a *run as fast as possible* mainloop, but it might be useful
to some of you.
David
--
dke...@travelbyroad.net
Pitcher's Duel -> pitchersduel.python-hosting.com
Nathan
Thanks for the suggestion. If twisted doesn't pan out, I'll give
pyraknet a shot.
Nathan
> On Feb 22, 12:50 am, Nathan <nathan.sto...@gmail.com> wrote:
> I'm not sure if you will find this easier than Twisted, but it works
> for me.
Heh. Well with that glowing endorsement, I'll put socketome down as
alternate #2.
> I also agree with other posters comments re: UDP. It's hard work, and
> you'll most likely end up reimplementing TCP.
Noted. I think I also agree. I just want to be able to send short
messages very fast and often with a minimum of fuss, which is why I
came asking for suggestions...
~ Nathan
Drew Smathers
That's great to hear. :)
David's PausingRector seems interesting - and might be especially
relevant in turn-based games. There are also other easy ways to
integrate with the default reactor on your platform. One such way is
to use a Coiterator. For example:
from twisted.internet import reactor, task
from twisted.python import log
import sys
class Runtime:
...
def run(self):
while not win.has_exit:
...
yield 1
def shutdown(self, result):
print 'bye'
reactor.stop()
def bailout(self, reason):
reason.printTraceback()
reactor.stop()
log.startLogging(sys.stdout)
runspot = Runtime()
task.coiterate(runspot.run()).addCallback(runspot.shutdown).addErrback(runspot.bailout)
reactor.run()
Another technique is to use a LoopingCall. Some people want to keep
the reactor and the game in separate threads - so _threadedselect
reactor will help you do this. This is some (very ugly) code I hacked
out to test a threadedselect with pyglet:
http://hg.enterthefoo.com/Yue/file/fdab6dcbd1c4/docs/examples/game/game.py
(see LoopingRuntime class)
Twisted of course provides non-blocking I/O so you can sometimes get
slightly better performance if you don't use an additional thread.
These days I just use coiterators as described above.
For you semi-real-time protocol layer, I would strongly suggest
looking into AMP - it's a wire-level, bidirectional, extensible remote
messaging protocol:
http://twistedmatrix.com/trac/browser/trunk/twisted/protocols/amp.py
Hope this helps.
Cheers,
TheKaptain
Checkout Pyro!!
Pyro is short for PYthon Remote Objects. It is an advanced and
powerful Distributed Object Technology system written entirely in
Python, that is designed to be very easy to use. Never worry about
writing network communication code again, when using Pyro you just
write your Python objects like you would normally. With only a few
lines of extra code, Pyro takes care of the network communication
between your objects once you split them over different machines on
the network. All the gory socket programming details are taken care
of, you just call a method on a remote object as if it were a local
object!
http://pyro.sourceforge.net/
David
> >
>
> That's great to hear. :)
>
> David's PausingRector seems interesting - and might be especially
> relevant in turn-based games. There are also other easy ways to
> integrate with the default reactor on your platform. One such way is
> to use a Coiterator. For example:
Thanks for the positive words. The implementation is not slow,
though, as your 'turn-based' comment might suggest. In any reactor,
the reactor gives up control for the duration of a callback. The same
code you can put into a reactor callback using selectReactor, you can
put between the reactor runs in PausingReactor. Same code, same time
consumption, same reactor delay.
PausingReactor just allows you to write code in a different style than
with selectReactor; for some of us, that is an easier style.
> from twisted.internet import reactor, task
> from twisted.python import log
> import sys
>
> class Runtime:
> ...
> def run(self):
> while not win.has_exit:
> ...
> yield 1
> def shutdown(self, result):
> print 'bye'
> reactor.stop()
> def bailout(self, reason):
> reason.printTraceback()
> reactor.stop()
>
> log.startLogging(sys.stdout)
> runspot = Runtime()
> task.coiterate(runspot.run()).addCallback(runspot.shutdown).addErrback(runspot.bailout)
> reactor.run()
I looked into this approach, and while it is better than chained
non-generator callbacks, it is still limited. If your run() method
calls subroutines that need significant runtime, and therefore have
their own internal yields, you have to use a syntax like "while ... :
yield 1":
def subcall():
# do stuff
yield 1
# do other stuff
yield 1
def run(self):
while not_done:
# do stuff
while subcall(): yield 1
With PausingReactor, you can do:
def subcall():
# do stuff
reactor.resume()
# do more stuff
reactor.resume()
def run():
while not_done:
# do stuff
reactor.resume()
subcall()
reactor.callLater(0,reactor.release())
reactor.run()
run()
The code can nest subroutines calls arbitrarily deep, and the reactor
can be resumed from anywhere, whenever the game is ready for user
interaction. Presumably, the intervals between resumes would be
brief.
This is a bit off-topic for a pyglet list, so I will stop here.
Thanks for the opportunity to make these points.
David
---
Richard Jones
> If your reason for not using Twisted is in wanting to own your
> main-loop, and your main-loop requirements include that it be
> restartable, you might take a look at my 'PausingReactor'.
>
> It is a Twisted reactor, derived from the default selectReactor, that
> is written around a generator to allow exiting and restarting.
A better approach that might be taken by someone such as yourself who has some
experience with reactors is to adapt the new pyglet.app eventloop (which is
very reactor-like) so it is able to be a Twisted reactor. I believe that's
the best approach for integrating Twisted and pyglet.
pyglet.app has many benefits of its own - being usable as a Twisted reactor
would be icing on the cake :)
Richard
Drew Smathers
>
> > > So any advice on the competing event loops?
> > >
> >
> > That's great to hear. :)
> >
> > David's PausingRector seems interesting - and might be especially
> > relevant in turn-based games. There are also other easy ways to
> > integrate with the default reactor on your platform. One such way is
> > to use a Coiterator. For example:
>
> Thanks for the positive words. The implementation is not slow,
> though, as your 'turn-based' comment might suggest.
By the way, I wasn't meaning to imply that. I was just trying to
conjure up use cases for pausing the reactor.
Ben Sizer
> I'm attempting to write a simple little real-time server-client game
> where the server and client talk to each other over the network.
>
> My initial implementation uses the built-in socket module, but I can't
> find a way to flush the socket! Any suggestions for a simple way for
> fast communications, or a way to make sockets work fast?
the alternatives just seem to change the complexity rather than remove
it. If you set the nodelay flag as specified elsewhere in this thread,
you should see a significant improvement. Otherwise, if you are
sending small periodic updates and don't mind if occasionally one
doesn't get through or comes out of order, use UDP. (But it's not
worth attempting to rewrite half of TCP in UDP, or you'll probably
come out with something exhibiting the worst of both.)
--
Ben Sizer
Nathan
I would be happy sticking with the sockets if the nodelay flag
actually changed something, but it didn't have any noticeable effect.
I posted my code in an earlier email if you're interested in looking
at it. As it is, I'm trying to figure out how to get Twisted and
pyglet to play together, starting with the code Drew linked to in his
earler email.
~ Nathan
Drew Smathers
Twisted won't assume nodelay, btw. However it does provide an
interface for querying/setting. For example, assuming amp is your
protocol:
class YourAmpProtocol(amp.AMP):
def makeConnection(self, transport):
if not transport.getTcpNoDelay():
transport.setTcpNoDelay(True)
super(NoDelayAMP, self).makeConnection(transport)
...
Cheers,
David
> > Thanks for the positive words. The implementation is not slow,
> > though, as your 'turn-based' comment might suggest.
>
> By the way, I wasn't meaning to imply that. I was just trying to
> conjure up use cases for pausing the reactor.
PausingReactor may not have been the best name for it.
ResumableReactor may be more descriptive, or maybe SuspendableReactor,
to indicate that the reactor can return from a run() call, and then
resume later. Both of those names have their problems as well.
David
--
Ben Sizer
>
> > To be honest, I would probably just stick with plain sockets, because
> > the alternatives just seem to change the complexity rather than remove
> > it. If you set the nodelay flag as specified elsewhere in this thread,
> > you should see a significant improvement. Otherwise, if you are
> > sending small periodic updates and don't mind if occasionally one
> > doesn't get through or comes out of order, use UDP. (But it's not
> > worth attempting to rewrite half of TCP in UDP, or you'll probably
> > come out with something exhibiting the worst of both.)
>
> I would be happy sticking with the sockets if the nodelay flag
> actually changed something, but it didn't have any noticeable effect.
it - but I think there's probably nothing wrong with the actual
sockets, and instead some sort of issue in the rest of the code, eg.
blocking on an accept() call, or waiting for thread switching, or
attempting to read/write too much, etc. A 2 second wait for the data
is far too much; more typical is 200ms of buffering. You may find that
shifting to an asyncore-based approach, or just writing your own
select()-based handling (perhaps single-threaded), works better than
trying to multithread each connection and perhaps getting it wrong.
--
Ben Sizer
David
In case anybody cares, here it is.
---------- Forwarded message ----------
From: David <dvke...@gmail.com>
Date: Feb 24, 2008 10:14 AM
Subject: Re: Suggestions for quick communications?
To: Richard Jones <r1char...@gmail.com>
On Fri, Feb 22, 2008 at 4:14 PM, Richard Jones <r1char...@gmail.com> wrote:
> On Sat, 23 Feb 2008, David wrote:
> > If your reason for not using Twisted is in wanting to own your
> > main-loop, and your main-loop requirements include that it be
> > restartable, you might take a look at my 'PausingReactor'.
> >
> > It is a Twisted reactor, derived from the default selectReactor, that
> > is written around a generator to allow exiting and restarting.
>
> A better approach that might be taken by someone such as yourself who has some
> experience with reactors is to adapt the new pyglet.app eventloop (which is
> very reactor-like) so it is able to be a Twisted reactor. I believe that's
> the best approach for integrating Twisted and pyglet.
The charm of using Twisted's reactor in lieu of various alternatives,
is in its use of select().
My understanding of select() is that it is an operating system
function (available on a generous range of platforms) that can monitor
sockets continuously while using minimal cpu resources. It can be
extended to monitor non-socket inputs by using the select() timeout,
and polling those alternative inputs periodically.
Monitoring sockets continously, and others periodically is inferior to
monitoring everything in a low-cpu os call, but that option does not
seem to be available. The select approach seems preferable to an
alternative that has to poll everything, and a select call with a ~0
timeout (as some event-loop integrations do) is basically a poll.
I am not familiar with pyglet's mainloop, and am not motivated to
learn it right now. I subscribed to the list with the thought of
porting my game engine 'Spyre' to pyglet from pygame; Pyglet is a
lower overhead library for an OpenGL/Python game engine than
Pygame+PyOpenGL. I have backburnered that, though, in favor of
getting my game done and out to gamers (development is four years old
and counting ;().
David
Is there still time to put a new Pygame-based release out to qualify
for the next Pyweek?
> pyglet.app has many benefits of its own - being usable as a Twisted reactor
> would be icing on the cake :)
>
>
> Richard
>
--
Drew Smathers
Hmmm ... a single-threaded select()-based handler. I wonder if there
are any libraries that already provide this?
Nathan
> David's PausingRector seems interesting - and might be especially
> relevant in turn-based games. There are also other easy ways to
> integrate with the default reactor on your platform. One such way is
> to use a Coiterator. For example:
>
> from twisted.internet import reactor, task
> from twisted.python import log
> import sys
>
> class Runtime:
> ...
> def run(self):
> while not win.has_exit:
> ...
> yield 1
> def shutdown(self, result):
> print 'bye'
> reactor.stop()
> def bailout(self, reason):
> reason.printTraceback()
> reactor.stop()
>
> log.startLogging(sys.stdout)
> runspot = Runtime()
> task.coiterate(runspot.run()).addCallback(runspot.shutdown).addErrback(runspot.bailout)
> reactor.run()
> slightly better performance if you don't use an additional thread.
> These days I just use coiterators as described above.
>
> For you semi-real-time protocol layer, I would strongly suggest
> looking into AMP - it's a wire-level, bidirectional, extensible remote
> messaging protocol:
>
> http://twistedmatrix.com/trac/browser/trunk/twisted/protocols/amp.py
It works! Thanks, Drew, for the post above. It was instrumental in
guiding me into how to use Twisted with pyglet. Using your code as a
starting point, the docstrings in amp.py that you linked to, the
ampserver.py and ampclient.py examples in the Twisted docs, and some
pointers from my friend who has used Twisted a lot more than me, I was
able to get it working. And it works well!
In my brain-dead first implementation (clients send keypresses and
releases to server, server controls movement and tells all clients
where players are, client draws the screen around the 'owned'
character), I can run the server and 8 copies of the client
simultaneously (all on my MacBook Pro) and still achieve ~25fps on
each of the 8 512x512-window clients with minimal noticeable lag!
Okay, I'm convinced. Twisted/AMP + pyglet is much better than
threads/sockets + pyglet.
~ Nathan
Ben Sizer
>
> Hmmm ... a single-threaded select()-based handler. I wonder if there
> are any libraries that already provide this?
hour's work at most anyway. http://www.amk.ca/python/howto/sockets/
has the basics. Really, basic networking is much simpler than people
think it is, and most of the libraries you get offered (Twisted not
included) require you to make just as many different calls as you
would using bare sockets.
--
Ben Sizer
Nathan
> Twisted won't assume nodelay, btw. However it does provide an
> interface for querying/setting. For example, assuming amp is your
> protocol:
>
> class YourAmpProtocol(amp.AMP):
> def makeConnection(self, transport):
> if not transport.getTcpNoDelay():
> transport.setTcpNoDelay(True)
> super(NoDelayAMP, self).makeConnection(transport)
Thanks for that! I added that to my code. I don't know if it's
making any performance difference, but it does go through that code
block, so at least it's setting it where it wasn't set before...
~ Nathan
Richard Jones
> Okay, I'm convinced. Twisted/AMP + pyglet is much better than
> threads/sockets + pyglet.
I'd be interested to see a sample. Did you use LoopingCall or the Coiterator
or ??
Richard
Simon Wittber
> Hmmm ... a single-threaded select()-based handler. I wonder if there
> are any libraries that already provide this?
python thread.
The select function is really a piece of cake to use, and is available
in the stdlib select module. I'd recommend anyone interested in using
sockets check it out.
Drew Smathers
>
> On Feb 27, 1:04 am, "Drew Smathers" <drew.smath...@gmail.com> wrote:
>
> > Hmmm ... a single-threaded select()-based handler. I wonder if there
> > are any libraries that already provide this?
>
> The socketome package I mentioned above uses select, in the main
> python thread.
>
Ok ... I was holding my silence on the first response to my
*question*, but to avoid any further confusion: my question was
entirely rhetorical. :) I already know of several libraries that
provide this.
Thanks for taking the time to answer, though.
Adam Bark
Nathan
I used the Coiterator, almost verbatim from your example. Here's the
loop part of my server, which is pretty much all of my server-specific
code except the AMP stuff. You'll notice I have some extra stuff
stubbed out (a little window, that doesn't ever get drawn, window
events dispatched without ever being acted upon, etc.). The code's
pretty rough and changing at a quick pace, but I'm open to
constructive criticism if anyone feels so inclined.
...loop part of the server...
class RMServer(window.Window):
def __init__(self, x, y):
super(RMServer, self).__init__(x, y)
def run(self):
global player_dict
global next_player
while True:
dt = server_game_clock.tick()
self.dispatch_events()
for i in range(1,next_player):
player_dict[i].update(dt, lines)
yield 1
def shutdown(self, result):
print "Gracefully shutting down"
reactor.stop()
def bailout(self, reason):
print "Ouch! Bailing out..."
reason.printTraceback()
reactor.stop()
if __name__ == '__main__':
# Server loop part
rm_server = RMServer(64, 64)
task.coiterate(rm_server.run()).addCallback(rm_server.shutdown).addErrback(rm_server.bailout)
# AMP part
amp_factory = Factory()
amp_factory.protocol = RMCommunication
reactor.listenTCP(PORT, amp_factory)
print "Server up and listening on port", PORT
reactor.run()
...loop part of my client...
class RMWindow(window.Window):
def __init__(self, x, y):
super(RMWindow, self).__init__(x, y)
[snip, snip, snip]
def run(self):
glEnable(GL_LINE_SMOOTH);
glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
glHint(GL_LINE_SMOOTH_HINT, GL_DONT_CARE);
glLineWidth(2);
#glShadeModel(GL_FLAT);
while not self.has_exit:
dt = client_game_clock.tick()
if self.player_man:
self.player_man.update(dt, self.lines)
self.amp_client.callRemote(UpdatePlayer,
player=self.player, player_state=self.player_man.get_state())
self.dispatch_events()
glClearColor(0.0, 0.0, 0.0, 0.0);
glClear(GL_COLOR_BUFFER_BIT);
self.draw()
self.flip()
for i in sorted(self.other_players):
self.other_players[i].update(dt, self.lines)
# Tell twisted to go get everyone else's states, which will
hopefully happen before the next loop here
if self.player_man:
self.amp_client.callRemote(GetStates).addCallback(self.update_other_players)
yield 1
def shutdown(self, result):
print "Gracefully shutting down"
reactor.stop()
def bailout(self, reason):
print "Ouch! Bailing out..."
reason.printTraceback()
reactor.stop()
if __name__ == '__main__':
if len(sys.argv) != 2:
print "USAGE: ./running-man.py IP_ADDRESS_OF_SERVER -- For
example: ./running-man.py 10.1.2.3"
sys.exit(-1)
rm_window = RMWindow(VIEWPORT_WIDTH, VIEWPORT_HEIGHT)
client_factory = ClientCreator(reactor,
amp.AMP).connectTCP(sys.argv[1],
PORT).addCallback(rm_window.install_amp_client)
task.coiterate(rm_window.run()).addCallback(rm_window.shutdown).addErrback(rm_window.bailout)
reactor.run()
Drew Smathers
> So what's so bad about UDP anyway?
>
I don't know. So, what's so great about pizza?
Seriously, what even is the context of your question? (Hint: quote the
related post above your question - this provides context.)
Cheers,
Adam Bark
On Thu, Feb 21, 2008 at 6:25 PM, Austin Wood <bon...@gmail.com> wrote:
>
> On Fri, Feb 22, 2008 at 6:40 AM, Nathan <nathan...@gmail.com> wrote:
> >
> > On Thu, Feb 21, 2008 at 9:00 AM, Drew Smathers <drew.s...@gmail.com> wrote:
> > >
> > > On Thu, Feb 21, 2008 at 10:50 AM, Nathan <nathan...@gmail.com> wrote:
>
> > > > My initial implementation uses the built-in socket module, but I can't
> > > > find a way to flush the socket! Any suggestions for a simple way for
> > > > fast communications, or a way to make sockets work fast? (right now,
> > > > the sockets seem to just wait about 2 seconds, and then transmit,
> > > > which is rather funny to watch, but not very playable).
> > > >
> > >
> > > Just use Twisted. Then you might actually come close to the "simple"
> > > objective. Nonetheless, you might want to google TCP_NODELAY for
> > > your particular problem.
> >
> > Twisted and "simple" have nothing to do with each other in my
> > experience. I even bought the Twisted book, but I have a really hard
> > time getting into it (not to mention that the developers seem to live
> > on the opposite side of the world and have odd attitudes when you
> > start asking them questions). I really don't want to deal with a
> > completely separate event loop. I'd also like to avoid external
> > dependencies where possible. (Right now I only depend on python
> > itself and pyglet)
> >
> > I'm researching the TCP_NODELAY option, which looks promising, but
> > there's a curious lack of explanation of those types of flags and how
> > to use the on http://docs.python.org/lib/module-socket.html --- I
> > suppose they assume you'll come with socket knowledge from C.
>
> The socket module in Python is a direct copy of the C (POSIX) socket
> API (modulo struct sizes). So they do assume you have socket knowledge
> from C.
>
> If you want low-latency you'll need to use UDP instead of TCP (that
> is, SOCK_DGRAM instead of SOCK_STREAM).
> TCP is a reliable two-way communication protocol so packets are
> guaranteed to reach the other host *in order*.
> UDP is connectionless and one-way (iirc)
> TCP is slow because, basically, 2 reasons;
> 1- Every packet received needs an ACKnowledgement sent back.
> 2- Nagle's algorithm which delays tiny packets to increase throughput
> (but increases latency)
Which is what TCP_NODELAY is for (2).
> To use UDP properly you'll need to write a networking layer that can
> deal with dropped and out-of-order packets.
And to make a long story short, you will fail miserably at this like
thousands of other programmers who heard UDP is super-duper fast and
thought it would be *fun* to make a real-time networked game based on
UDP. Or you will succeed only partially with an implementation that
completely sucks and works slower than TCP. Either way, you will
spend a lot of time debugging the UDP side of your application and
have little time leftover to have fun writing your game.
Assuming you want to maintain your sanity, just stick to TCP.
Cheers,
On Wed, Feb 27, 2008 at 12:01 PM, Adam Bark <adam....@gmail.com> wrote:
> So what's so bad about UDP anyway?
>
I don't know. So, what's so great about pizza?
Seriously, what even is the context of your question? (Hint: quote the
related post above your question - this provides context.)
Cheers,
Drew Smathers
Ok. I didn't state UDP is bad, though. The internet wouldn't work
so well without it. I'm not going to give you an essay on why I think
using UDP is hard. But to list just two reasons briefly:
1. Flow control
2. Routers
Implementing a robust flow control algorithm is difficult to ordinary
people like myself. Regarding 2, UDP is a connectionless protocol and
in an online game, your peers are likely behind a NAT.
gherron
On Feb 27, 11:05 am, "Adam Bark" <adam.jt...@gmail.com> wrote:
> On 21/02/2008, Drew Smathers <drew.smath...@gmail.com> wrote:
>
>
>
>
>
> > On Thu, Feb 21, 2008 at 6:25 PM, Austin Wood <bonf...@gmail.com> wrote:
>
> > > On Fri, Feb 22, 2008 at 6:40 AM, Nathan <nathan.sto...@gmail.com>
>
> > > > On Thu, Feb 21, 2008 at 9:00 AM, Drew Smathers <
>
> > > > > On Thu, Feb 21, 2008 at 10:50 AM, Nathan <nathan.sto...@gmail.com>
> > > So what's so bad about UDP anyway?
>
> > I don't know. So, what's so great about pizza?
>
> > Seriously, what even is the context of your question? (Hint: quote the
> > related post above your question - this provides context.)
>
> > Cheers,
>
> Ok I didn't think it was particularly necessary as you flat out said that he
> would fail if he tried to use UDP but I was just wondering why everybody
> apparently finds it so hard to use UDP.
many ways for
communication to fail, and to be robust, your program has to be
recognize/handle/recover-from any of them. UDP is extra hard over TCP
because it provides less guarantees /information about the
communication:
* Packets may be dropped with you being notified.
* Packets may arrive in an order different in which they were sent
--
and again you won't be notified.
* There is no sense of a connection. If you don't receive packets
from the other end you have to notice this fact yourself, and guess
why. Did the other end close down nicely and forget to tell you (or
did
it tell you but that packet got dropped), or did some part of the
network die, or are they still active but delayed (or active and
sending
but the packets are being dropped), or ... or ... or ... there's no
end.
* Communication in the other direction is fraught with the same
problems. You can send anything you want over UDP, but have no idea
if
they are being delivered, received, or responded to.
Nathan
out most of the extra stuff to just show how I made Twisted and pyglet
work together (which is basically a direct copy of what Drew did). I
thought I already posted it to this thread, but I suppose once more
won't hurt. I'm making a 2D networked game. For fun.
class RMWindow(window.Window):
def __init__(self, x, y):
super(RMWindow, self).__init__(x, y)
def install_amp_client(self, amp_client):
self.amp_client = amp_client
self.amp_client.callRemote(Register).addCallback(create_player)
def run(self):
while not self.has_exit:
dt = client_game_clock.tick()
self.dispatch_events()
self.clear()
self.draw()
self.flip()
yield 1
def shutdown(self, result):
reactor.stop()
def bailout(self, reason):
reason.printTraceback()
reactor.stop()
if __name__ == '__main__':
rm_window = RMWindow(VIEWPORT_WIDTH, VIEWPORT_HEIGHT)
client_factory = ClientCreator(reactor,
RMClientProtocol).connectTCP('IPADDR',
PORT).addCallback(rm_window.install_amp_client)
task.coiterate(rm_window.run()).addCallback(rm_window.shutdown).addErrback(rm_window.bailout)
reactor.run()
On Tue, Mar 4, 2008 at 12:50 PM, Snor <l...@snorland.com> wrote:
> Hi,
>
> I'm also wanting to implement twisted while using pyglet.app... after
> reading the whole thread I'm still a little confused as to what final
> solution you eventually used. If I might be so cheeky as to ask you to
> provide me with the simplest example of this setup?
>
> Any help would be much appreciated!
>
> By the way I am curious as to what you plan on making :)
>
> Leo
>