proxy authentication problem

[Nasrullah]

Hi all,
My spider is working fine behind the proxy with out authentication.
But when i try to use proxy authentication... it does not work..
i set meta this way:
meta['proxy'] = 'http://username:passwd@proxy_server:port'

Any suggestion ????
Thanks

[Pablo Hoffman]

It doesn't work that way. Request meta can only contain the proxy
host/port, not the credentials. You have to pass the authentication in
the "Proxy-Authorization" field. See the code of ProxyMiddleware for
more details:
https://github.com/scrapy/scrapy/blob/master/scrapy/contrib/downloadermiddleware/httpproxy.py#L47

[Nasrullah]

@Pablo
I have handled the proxy authentication in the downloader middleware.
This extracts the creds and proxy from the
request.meta['proxy'] and then set the proxy in request.meta with
proxy url
and Proxy Authorization in request.headers with the creds.

I have sent u a pull request on git hub @ https://github.com/scrapy/scrapy/pull/55

On Nov 14, 4:44 pm, Pablo Hoffman <pablohoff...@gmail.com> wrote:
> It doesn't work that way. Request meta can only contain the proxy
> host/port, not the credentials. You have to pass the authentication in
> the "Proxy-Authorization" field. See the code of ProxyMiddleware for

> more details:https://github.com/scrapy/scrapy/blob/master/scrapy/contrib/downloade...
>
> On 11/14/2011 09:31 AM,Nasrullahwrote: