Re: [tornado] HTTP proxy with Tornado

[Andrew Grigorev]

HTTP proxy in tornado developed with an ability to use custom filters
for some urls in mind:
https://github.com/ei-grad/charon/blob/master/charon.py

May be its code could be useful for you.

27.10.2012 05:13, Samat Jain пишет:
> I want to write an HTTP proxy w/ Tornado. Cobbling together something
> together based on the docs and various mailing post lists, I came up
> with (be aware: almost no exception handling or error checking):
>
> (also at http://bpaste.net/show/jz9SWHr4bddE2mQrAxD8/)
>
> #!/usr/bin/env python
> # -*- coding: utf-8 -*-
> from __future__ import print_function
> from __future__ import unicode_literals
>
> import tornado.gen
> import tornado.ioloop
> import tornado.web
> import tornado.curl_httpclient
> import tornado.httpclient
>
> class ProxyHandler(tornado.web.RequestHandler):
> @tornado.web.asynchronous
> @tornado.gen.engine
> def get(self):
> url = self.request.uri
> print('request url: ', url)
>
> request = tornado.httpclient.HTTPRequest(
> url=url,
> method=self.request.method,
> body=self.request.body,
> headers=self.request.headers,
> allow_ipv6=True,
> follow_redirects=True),
>
> client = tornado.httpclient.AsyncHTTPClient()
>
> response = yield tornado.gen.Task(client.fetch, request)
>
> if response.error and not isinstance(response.error,
> tornado.httpclient.HTTPError):
> self.write("Internal server error:\n" + str(response.error))
> raise tornado.web.HTTPError(500)
> else:
> self.set_status(response.code)
> for header in ("Date", "Cache-Control", "Server",
> "Content-Type", "Location"):
> v = response.headers.get(header)
> if v:
> self.set_header(header, v)
> if response.body:
> self.write(response.body)
> self.finish()
>
>
> if __name__ == '__main__':
>
> app = tornado.web.Application([
> (r'.*', ProxyHandler),
> ], debug=True)
> app.listen(8888)
>
>
> tornado.httpclient.AsyncHTTPClient.configure("tornado.curl_httpclient.CurlAsyncHTTPClient")
>
> tornado.ioloop.IOLoop.instance().start()
>
> Except, it doesn't work! When I point my Web browser at it, I get
> exceptions:
>
> request url: http://winswitch.org/dists/quantal/InRelease
> ERROR:root:Uncaught exception GET
> http://winswitch.org/dists/quantal/InRelease (127.0.0.1)
> HTTPRequest(protocol='http', host='winswitch.org', method='GET',
> uri='http://winswitch.org/dists/quantal/InRelease',
> version='HTTP/1.1', remote_ip='127.0.0.1', body='', headers={'Host':
> 'winswitch.org', 'Cache-Control': 'max-age=0', 'Accept': 'text/*',
> 'User-Agent': 'Debian APT-HTTP/1.3 (0.9.7.5ubuntu5)'})
> Traceback (most recent call last):
> File "/usr/lib/python2.7/dist-packages/tornado/web.py", line 1000,
> in _stack_context_handle_exception
> raise_exc_info((type, value, traceback))
> File "/usr/lib/python2.7/dist-packages/tornado/web.py", line 1118,
> in wrapper
> return method(self, *args, **kwargs)
> File "/usr/lib/python2.7/dist-packages/tornado/gen.py", line 120, in
> wrapper
> runner.run()
> File "/usr/lib/python2.7/dist-packages/tornado/gen.py", line 343, in
> run
> yielded = self.gen.throw(*exc_info)
> File "tornado-http-proxy.py", line 29, in get
> response = yield tornado.gen.Task(client.fetch, request)
> File "/usr/lib/python2.7/dist-packages/tornado/gen.py", line 366, in
> run
> self.yield_point.start(self)
> File "/usr/lib/python2.7/dist-packages/tornado/gen.py", line 241, in
> start
> self.func(*self.args, **self.kwargs)
> File "/usr/lib/python2.7/dist-packages/tornado/curl_httpclient.py",
> line 82, in fetch
> self._process_queue()
> File "/usr/lib/python2.7/dist-packages/tornado/curl_httpclient.py",
> line 213, in _process_queue
> curl.info["headers"])
> File "/usr/lib/python2.7/dist-packages/tornado/curl_httpclient.py",
> line 276, in _curl_setup_request
> curl.setopt(pycurl.URL, utf8(request.url))
> File "/usr/lib/python2.7/dist-packages/tornado/escape.py", line 168,
> in utf8
> assert isinstance(value, unicode)
> AssertionError
> ERROR:root:500 GET http://winswitch.org/dists/quantal/InRelease
> (127.0.0.1) 1.31ms
>
> After debugging this a bit on IRC, there's now a bug:
> https://github.com/facebook/tornado/issues/618 (that unfortunately no
> one's taken a look at yet).
>
> Am I doing something wrong? Using Tornado 2.3 and Python 2.7.3 on
> Ubuntu 12.10 (but the same happens on Tornado 2.4).


--
Andrew

[Ben Darnell]

The tuple is coming from this trailing comma.

-Ben