User timeout caused connection failure

Havy

unread,

Sep 21, 2011, 11:26:07 PM9/21/11

to scrapy-users

Hi, All,

I'm using scrapy to crawl some pages, but encountering the following
error.
* I've set the "DOWNLOAD_TIMEOUT" to 1200
* the first log is printed by a downloadermiddleware, which mens the
request processed.
* from the log timestamp, the time difference is only 38s, why?

2011-09-22 09:03:41+0800 [42-001] DEBUG: request processed, http://xxxxx.html

2011-09-22 09:04:19+0800 [42-001] ERROR: Error downloading <http://
xxxxx.html>: [Failure instance: T
raceback (failure with no frames): <class
'twisted.internet.error.TimeoutError'>: User timeout caused connection
failure.]

Thank you in advance!

- Havy

Havy

unread,

Sep 29, 2011, 4:36:17 AM9/29/11

to scrapy-users

Anybody can help me out?
Thanks!

On Sep 22, 11:26 am, Havy <wanghewei1...@gmail.com> wrote:
> Hi, All,
>
> I'm using scrapy to crawl some pages, but encountering the following
> error.
> * I've set the "DOWNLOAD_TIMEOUT" to 1200
> * the first log is printed by a downloadermiddleware, which mens the
> request processed.
> * from the log timestamp, the time difference is only 38s, why?
>

> 2011-09-22 09:03:41+0800 [42-001] DEBUG: request processed,http://xxxxx.html

Pablo Hoffman

unread,

Oct 25, 2011, 3:01:00 AM10/25/11

to scrapy...@googlegroups.com

Makes no sense to me.

Maybe that error is from a different request?. Can you reproduce this
with the start url?

Reply all

Reply to author

Forward