long polling/comet-style http handler

[dan90]

A question for the HTTP gurus: How do i implement an HTTP server which
does not close the connection after serving up each response?

I'm writing a project that needs a long-polling comet-style HTTP
server myself, and Denis' long-polling example (
http://blog.gevent.org/2009/10/10/simpler-long-polling-with-django-and-gevent
) looks like it might be a nice start. However, I'm having trouble
making the example work for my use case. in particular, I don't want
my comet server to close the HTTP response, but rather to stream data
to it gradually without closing said connection. (this is required by
the specification I'm working to -
http://wiki.gephi.org/index.php/Specification_-_GSoC_Graph_Streaming_API#Connecting_to_the_API_server
)

However, when i inspect Denis' chat example using the Chrome Web
Developer extension, it seems that it implements long polling by
having the server return a complete response after each XMLHttpRequest
object, after which the client JS immediately opens a new one. indeed,
gevent.pywsgi.WSGIServer seems to return complete HttpResponse objects
per default.

So, could i trouble you folks for tips as to the best way to implement
an HTTP server which keeps the request alive? Is there some
functionality buried in gevent.pywsgi.WSGIHandler that i should using?
Or will i need to start by subclassing the handler? Or subclassing
something else entirely?

(First time delving into detail of HTTP this deply, so forgiveany
clumsiness in my vocabulary)

Thanks kindly folks
Dan

[Ralf Schmitt]

dan90 <d...@possumpalace.org> writes:

> So, could i trouble you folks for tips as to the best way to implement
> an HTTP server which keeps the request alive? Is there some
> functionality buried in gevent.pywsgi.WSGIHandler that i should using?
> Or will i need to start by subclassing the handler? Or subclassing
> something else entirely?

pywsgi will not close the connection if the client is using HTTP/1.1
(and not sending the "Connection: close" header). So, it should already
do what you want.

Cheers,
- Ralf

[bruce bushby]

Have you tried: http://pypi.python.org/pypi/gevent-websocket

[dan90]

Thanks folks.

@bruce - fantastic library! how i wish i could use it, since
Websockets are way more appropriate for the task in hand. however,
that's not how the websockets folks rolled in this case. I'm going to
suggest they do so. In the interim, I will be using your that library
for another project of mine, as the API is way nicer than the google
reference implementation. Thanks.

@ralf - your tip was the clue I needed.

For future travellers down this path, i think my confusion was twofold
- firstly the jquery library seems not to recycle ajax objects per
default. (Is that even possible? My browser-fu is weak) Secondly, the
webchat example, and indeed the stream-web example (
http://blog.gevent.org/2009/12/05/more-comet-with-gevent/ ) use django
HttpResponse objects which AFAICT don't care like streaming. So I had
at last to dive into the WSGI spec and work out how that stuff works.
and the answer is returning an iterator.

http://www.python.org/dev/peps/pep-0333/#buffering-and-streaming

That was far, far simpler than I expected. all i have to do is yield
from my data source in the application function passed to the
WSGIServer instance. This is now working, and I'll post the code up
here the moment it does anything useful

Cheers
Dan

On Sep 3, 6:39 pm, bruce bushby <bruce.bus...@googlemail.com> wrote:
> Have you tried:http://pypi.python.org/pypi/gevent-websocket
> <http://pypi.python.org/pypi/gevent-websocket>
>
>
>
>
>
>
>
> On Fri, Sep 3, 2010 at 8:24 AM, dan90 <d...@possumpalace.org> wrote:
> > A question for the HTTP gurus: How do i implement an HTTP server which
> > does not close the connection after serving up each response?
>
> > I'm writing a project that needs a long-polling comet-style HTTP
> > server myself, and Denis' long-polling example (
>

> >http://blog.gevent.org/2009/10/10/simpler-long-polling-with-django-an...

> > ) looks like it might be a nice start. However, I'm having trouble
> > making the example work for my use case. in particular, I don't want
> > my comet server to close the HTTP response, but rather to stream data
> > to it gradually without closing said connection. (this is required by
> > the specification I'm working to -
>

> >http://wiki.gephi.org/index.php/Specification_-_GSoC_Graph_Streaming_...