Asynchronous HTTP client
Ping
I'm trying to find a way to create an asynchronous HTTP client so I
can get responses from web servers in a way like
async_http_open('http://example.com/', callback_func)
# immediately continues, and callback_func is called with response
as arg when it is ready
It seems twisted can do it, but I hesitate to bring in such a big
package as a dependency because my client should be light. Asyncore
and asynchat are lighter but they don't speak HTTP. The asynchttp
project on sourceforge is a fusion between asynchat and httplib, but
it hasn't been updated since 2001 and is seriously out of sync with
httplib.
I'd appreciate it if anyone can shed some lights on this.
Thanks,
Ping
Lie Ryan
If you want something quite lightweight, you can spawn a thread for the
call:
import threading, urllib
class AsyncOpen(threading.Thread):
def __init__(self, url, callback):
super(AsyncOpen, self).__init__()
self.url = url
self.callback = callback
def run(self):
# of course change urllib to httplib-something-something
content = urllib.urlopen(self.url).read()
self.callback(content)
def asyncopen(url, callback):
AsyncOpen(url, callback).start()
def cb(content):
print content
asyncopen('http://www.google.com', cb)
exa...@twistedmatrix.com
>Hi,
>
>I'm trying to find a way to create an asynchronous HTTP client so I
>can get responses from web servers in a way like
>
> async_http_open('http://example.com/', callback_func)
> # immediately continues, and callback_func is called with response
>as arg when it is ready
>
>It seems twisted can do it, but I hesitate to bring in such a big
>package as a dependency because my client should be light. Asyncore
>and asynchat are lighter but they don't speak HTTP. The asynchttp
>project on sourceforge is a fusion between asynchat and httplib, but
>it hasn't been updated since 2001 and is seriously out of sync with
>httplib.
Why should it be "light"? In what way would using Twisted cause
problems for you?
Jean-Paul
exa...@twistedmatrix.com
>2010/3/7 <exa...@twistedmatrix.com>
>>>
>>>I'm trying to find a way to create an asynchronous HTTP client so I
>>>can get responses from web servers in a way like
>>>
>>> async_http_open('http://example.com/', callback_func)
>>> # immediately continues, and callback_func is called with response
>>>as arg when it is ready
>>>
>>>It seems twisted can do it, but I hesitate to bring in such a big
>>>package as a dependency because my client should be light. Asyncore
>>>and asynchat are lighter but they don't speak HTTP. The asynchttp
>>>project on sourceforge is a fusion between asynchat and httplib, but
>>>it hasn't been updated since 2001 and is seriously out of sync with
>>>httplib.
>>
>>problems
>>for you?
>>
>>Jean-Paul
>
>may
>be used in all kinds of environments - Linux, Mac OS X, Windows, web
>hosting, etc by others. It is not impossible to have twisted as a
>dependency, but that makes deployment a larger job than simply
>uploading a
>Python file.
Twisted is packaged for many Linux distributions (perhaps most of them).
Many web hosts provide it. It's also shipped with OS X.
Windows may be an issue, but note that there's a binary Windows
installer (as of 10.0, an MSI, so installation can be easily automated).
>I'm willing to use twisted, but I'd like to explore lighter
>alternatives
>first.
Jean-Paul
Antoine Pitrou
> I'm writing an open source python client for a web service. The client
> may be used in all kinds of environments - Linux, Mac OS X, Windows, web
> hosting, etc by others. It is not impossible to have twisted as a
> dependency, but that makes deployment a larger job than simply uploading
> a Python file.
If it can be used by non-Python users, you will have to package it using
py2exe or py2app anyway, in which case Twisted will be bundled
automatically and the size overhead won't be very large.