Automatic proxy detection support (issue 5451071)

[jar...@jaraco.com]

Reviewers: jcgregorio,

Message:
This patch addresses issue 159.

Please review this at http://codereview.appspot.com/5451071/

Affected files:
M CHANGELOG
M python2/httplib2/__init__.py
M python2/httplib2test.py

[dur...@gmail.com]

Jumping in here uninvited, but I had one thing I thought might be worth
mentioning.

http://codereview.appspot.com/5451071/diff/1/python2/httplib2/__init__.py
File python2/httplib2/__init__.py (right):

http://codereview.appspot.com/5451071/diff/1/python2/httplib2/__init__.py#newcode767
python2/httplib2/__init__.py:767: env_var = method+'_proxy'
I believe that https should respect http_proxy if https_proxy isn't set,
but I could also be making that up. Also, I've never actually heard
of/seen anyone use https_proxy.

http://codereview.appspot.com/5451071/

[jar...@jaraco.com]

On 2011/12/02 20:39:05, Augie fackler wrote:
> I believe that https should respect http_proxy if https_proxy isn't
set, but I
> could also be making that up. Also, I've never actually heard of/seen
anyone use
> https_proxy.

That seems like a good suggestion. My intention was to implement the
behavior consistent with the community conventions and with the
implementation in urllib2. I'll double-check the spec and urllib2
behavior and make it consistent or comment any justifiable deviance.

We use https_proxy where I work, though it's always set to the same
thing as the http_proxy.

Thanks for the suggestions.

http://codereview.appspot.com/5451071/

[jar...@jaraco.com]

On 2011/12/02 21:08:14, jaraco wrote:
> On 2011/12/02 20:39:05, Augie fackler wrote:
> > I believe that https should respect http_proxy if https_proxy isn't
set, but I
> > could also be making that up. Also, I've never actually heard
of/seen anyone
> use
> > https_proxy.

> That seems like a good suggestion. My intention was to implement the
behavior
> consistent with the community conventions and with the implementation
in
> urllib2. I'll double-check the spec and urllib2 behavior and make it
consistent
> or comment any justifiable deviance.

I couldn't find any examples online where HTTPS requests fall back to
the http_proxy. The best I can tell, https_proxy must be set for HTTPS
requests to go through the proxy. This behavior is consistent with my
findings in urllib2:

>>> import os
>>> del os.environ['https_proxy']
>>> import urllib2
>>> res = urllib2.urlopen('https://www.jaraco.com')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.6/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib/python2.6/urllib2.py", line 391, in open
response = self._open(req, data)
File "/usr/lib/python2.6/urllib2.py", line 409, in _open
'_open', req)
File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
result = func(*args)
File "/usr/lib/python2.6/urllib2.py", line 1169, in https_open
return self.do_open(httplib.HTTPSConnection, req)
File "/usr/lib/python2.6/urllib2.py", line 1136, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [Errno 110] Connection timed out>

In the example above, http_proxy is still set, and if urllib2 were to
fall back, it would be able to request that URL just fine, but it does
not. It tries to connect directly and times out (due to firewall
restrictions for direct connections).

Unless someone can find an authoritative specification or widely-used
representative implementation that suggests falling back to http_proxy,
I believe the current implementation is correct.

http://codereview.appspot.com/5451071/