cgiproxy - tool to counteract internet censorship
Michael Foord
restrictions. It is a python cgiproxy called approxy.py and includes a
family of related scritps to allow unrestricted browsing in a censored
environment. The script needs isntalling on a server you can reach....
http://www.voidspace.org.uk/atlantibots/pythonutils.html#cgiproxy
If anyone wants any help with it feel free to contact me.
CGI proxy - approx.py
Version 0.5.0 31st August 2004
proxycleaner.py
Version 1.0.1 3rd September 2004
approxClientproxy.py
Version 0.1.0 18th August 2004
Where I work, all internet access is via a censoring proxy, meaning
that most internet locations are unavailable. This is often
inconvenient. approx.py is a CGI proxy that will fetch webpages for
me, and can be used by anyone in a censored internet environment -
e.g. Work, libraries, internet cafes, colleges and other oppressive
regimes. It uses an HTMLParser to modify links, images etc so that
they also go through the proxy. It is inspired by the James Marshall
perl CGIproxy. approx is not as sophisticated as the James Marshall
one (it doesn't help with anonymity) but :
gives you the option of retrieving a page unmodified
doesn't mangle mailto/nntp links
will fetch https resources (with the accompanying security risk)
doesn't mangle local url references (within the page)
works in conjuction with approxClientproxy to allow fully transparent
browsing through the proxy !
useful DEBUG mode (for http and CGI environment disgnostics..
including cookies)
doesn't produce invalid XHTML which the perl parser can do (which
confuses IE)
As it's written in Python it's very easy to adapt. If you need an HTML
parser that will copy a page only modifying certain tags, you could
use it's Scraper class. It does *most* of what HTMLParser.HTMLParser
does - except without choking on bad HTML. Version 1.3.0 of Scraper
includes a patch by Paul Perkins that fixes a bug when dealing with
tags without attributes.
approx is still fairly beta, but functional. Cookie handling with
ClientCookie (or cookielib) now works - including support for multiple
users. User cookie management still to be added. Most POST requests
work except for list values and file uploads. Authentication is the
next major issue to work on. Get ready to start using CGIproxy. The
approx.zip comes with the ClientCookie module.
Try out a version of approx.py running in restricted mode. It will
only fetch pages from the www.voidspace.org.uk domain - but you can
see it functioning. Try Out approx.py Hosted at Xennos.
approx.py and the James Marshall perl cgiproxy modify pages fetched
through them. This can be a real pain if you want to save these pages
and possibly distribute them. Particularly if you want to keep your
proxy location private. proxycleaner.py will undo the URL
modifications for you.
approx works by mangling URLs so that all image fetches and links etc
go through the proxy. This doesn't work with javascript and few other
things (like some CSS etc - although this will be fixed). This isn't
ideal - where you are free to run programs on the machine
approxClientproxy (aPc) offers a better alternative.
approxClientproxy.py is a client script that works in conjunction with
approx.py - it runs on your machine. Instead of having approx modify
pages you can point your browser at approxClientproxy which handles
the communication between itself and approx. All the traffic goes
through aPc which does the URL mangling for you... At the moment it's
a crude hack of TCPwatch by the Zope corp.. but it works fine and will
only improve. It allows transparent unrestricted http access in a
restricted environment. Having a client-server model also allows for
interesting possibilities like url and content encryption for those
who might need it. It would be easy to build this into a windoze
executable if anyone wanted to use it on a machine without python
installed.
If you need help installing approx on a server, using aPc, or any
other questions - feel free to mail me.