using boto for tunneling HTTPS through a proxy

[Michael Schwartz]

Hi,

Some gsutil (which users boto) users have encountered a problem when tunneling HTTPS through a proxy, and we came across pull 227, in which hybris42 proposed commenting out the \r\n sent after the proxy auth header lines. In the pull request discussion mibanescu noted that the additional \r\n is correct according to an (outdated) Internet Draft, for which I've been unable to locate a normative current spec requiring that behavior. mibanescu also suggested a change to try sending with the additional \r\n and then if that fails, try without it. After a bit more discussion the issue was left unresolved for the past 10 months.

One of our users tested three different proxy implementations, and in each case found boto with the additional \r\n didn't succeed, and commenting out the additional \r\n caused it to work. I'd like to come up with a resolution that works for the proxy users, but I'm not confident that removing the extra \r\n will not break other proxies -- and I don't like the overhead/latency of trying it both ways.

To address this problem I'd like to add a config option that defaults to the current (additional \r\n) behavior, but that users can set to not send the additional \r\n. Does anyone have thoughts/concerns about that approach?

Thanks,

Mike

[Mitchell Garnaat]

Hi Mike -

I have no concerns about the approach you describe.  My only thought is whether or not this should be the default behavior since the testing done seems to suggest the current approach simply doesn't work.

Mitch

[Michael Schwartz]

Hi Mitch,

Changing the default is probably the right thing to do. How about if I make the change on the develop branch and send mail on the boto-users list telling people about it, noting that if they're tunneling HTTPS through a proxy and encounter the error they can revert to the previous behavior by setting the config variable?

Thanks,

Mike

[Mitch Garnaat]

Sounds right to me.  Thanks, Mike!