stop spider using CloseSipder exception

887 views
Skip to first unread message

Radu Marginean

unread,
Nov 27, 2011, 5:46:18 AM11/27/11
to scrapy...@googlegroups.com
Hi,

In Scrapy 0.14, raising CloseSpider exception should shutdown the
spider.I am trying to raise CloseSpider from my items pipeline. It
works fine when the exception is thrown from the init method but does
not work when raised from the spider_opened method. Why is that?

Many thanks.
Radu

Rolando Espinoza La Fuente

unread,
Nov 27, 2011, 9:57:46 PM11/27/11
to scrapy...@googlegroups.com
CloseSpider exception is only supported in the spider callbacks.

Raising it from a pipeline's __init__ method does not make
sense as the pipelines are instantiated before any spider is opened.

To close an spider for any part of your code you should use
`engine.close_spider`
method. See this extension for an usage example:
https://github.com/scrapy/scrapy/blob/master/scrapy/contrib/closespider.py#L61

~Rolando

> --
> You received this message because you are subscribed to the Google Groups "scrapy-users" group.
> To post to this group, send email to scrapy...@googlegroups.com.
> To unsubscribe from this group, send email to scrapy-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/scrapy-users?hl=en.
>
>

Reply all
Reply to author
Forward
0 new messages