Twisted python HTTP client

[Lucas Taylor]

I started a modest implementation of a Twisted python wrapper around
the Collecta HTTP API if anybody is interested in hacking on it. I
worked on it awhile ago for a local user group presentation, and then
never got around to finishing it (the TODO details what is missing).
It has basic functionality for executing http queries with xml results
and some abstractions for querying Stories, Comments, Photos, etc.

Hopefully someone will find it useful:

http://bitbucket.org/ltvolks/txcollecta

There is a demo app:
$ twistd -noy examples/simple_web.tac

Will launch a server that can be accessed and queried via:
http://localhost:8080/

The docs are decent and there is an api reference:
file://path/to/your/clone/txcollecta/docs/html/index.html

And there are some basic tests (json support is still absent):
$ trial txcollecta.tests


Sample usage:

from txcollecta.client import CollectaClient

API_KEY = '12345678901234567890'
c = CollectaClient(API_KEY)

# Unparsed results
result = c.search('"Sports Team"')
def parseAtom(result):
# diy Atom parsing goes here
print result # punt!
result.addCallback(parseAtom).addErrback(log.err)

# Parsed dict-like object returned from feedparser
parsed = c.getParsed('"meaningless gossip"')
def processParsed(result):
print result
parsed.addCallback(processParsed)

# Shortcuts exist for specifying desired categories (Stories,
Comments, Updates, Photos, Videos)
stories = c.getStories(query='Individual Search Terms OR "Search
Term"')
def processStories(result):
for story in result.stories:
print story.title, story.links
stories.addCallback(processStories)

photos = c.getPhotos(query='"attractive individuals"')
def processPhotos(result):
# naive demo; may destroy everything!
for idx, photo in enumerate(result.photos.enclosures):
fname = path.basename(urlparse(photo.href).path)
d = twisted.web.client.downloadPage(photo.href, fname)
# Add callback/errback for when file is downloaded and saved
to disk
d.addCallback(lambda _: pass)
d.addErrback(log.err)
photos.addCallback(processPhotos).addErrback(log.err)

[Christopher Zorn]

Very cool. This should be useful to a lot of people.