how to capture different kind of error?

[lin di]

    i use free proxys at http://www.freeproxylists.net/. when one ip gets error, i need to switch to next low latency ip. 

    there exists several kinds of error like 

           "failed 1 times): An error occurred while connecting: 104: Connection reset by peer",

           "Connection to the other side was lost in a non-clean fashion.",

           Connection was refused by other side: 111: Connection refused,

           User timeout caused connection failure

    how can i capture them?

    thanks in advance.

[lin di]

sorry to disturb you guys.

finally, i found , process_exception function in download middleware can capture it.  if it doesn't work, i will try hook signal spider_error.

在 2013年4月4日星期四UTC+8下午9时40分27秒，lin di写道：

[ym zhang]

Did you make it ? I have ran to similiary problem here. I was using scrapy with unstable proxies too, spider always closed early than expected for all kinds of network exception. I am curioused about how you managed it .Can you give me some help here. Thanks:)

[Rolando Espinoza]

You can check the retry middleware: https://github.com/scrapy/scrapy/blob/master/scrapy/downloadermiddlewares/retry.py#L29

It uses the process_exception method to intercept network/timeout issues.

Rolando

--
You received this message because you are subscribed to the Google Groups "scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users...@googlegroups.com.
To post to this group, send email to scrapy...@googlegroups.com.
Visit this group at https://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.

[ym zhang]

Thanks, that 's very kind of you. I have handled those issues both in process_exception and errback function, thanks again.  Followed you on google plus : ) 

--
You received this message because you are subscribed to a topic in the Google Groups "scrapy-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/scrapy-users/mtGP6mWhdWA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to scrapy-users...@googlegroups.com.