A request with higher priority will be executed before a request with lower
priority. Spider middlewares can adjust the Request.priority attribute.
Pablo.
> --
> You received this message because you are subscribed to the Google Groups "scrapy-users" group.
> To post to this group, send email to scrapy...@googlegroups.com.
> To unsubscribe from this group, send email to scrapy-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/scrapy-users?hl=en.
One thing that can also help is setting CONCURRENT_REQUESTS_PER_SPIDER to 1,
but it won't completely ensure the order either because the downloader has its
own local queue for performance reasons, so the best you can do is prioritize
the requests but not ensure its exact order.
Pablo.
Maybe you have hit a CPU bottleneck?
Pablo.