No results with middle function

[Nick Gilmour]

Hi all,

I wanted to have the response object in the first page, so I have created a new function named middle() between the functions on_start() and index_page(). Like this:

from pyspider.libs.base_handler import *

class Handler(BaseHandler):

    crawl_config = {

    }

     

    @every(minutes=24 * 60)

    def on_start(self):

        self.crawl('http://scrapy.org/', callback=self.middle)

    @every(minutes=24 * 60)

    def middle(self,reponse):

        self.crawl('http://scrapy.org/', callback=self.index_page)

    

        

    @config(age=10 * 24 * 60 * 60)

    def index_page(self, response):

        for each in response.doc('a[href^="http"]').items():

            self.crawl(each.attr.href, callback=self.detail_page)

    def detail_page(self, response):

        return {

            "url": response.url,

            "title": response.doc('title').text(),

        }

When I run this in the debugger mode everythin seems to be working fine - links are generated correctly and I can see the end results. When I set this script in DEBUG mode and let it run, I get no erros but there are no results. Am I missing something or is this a bug?

[Nick Gilmour]

typo...

def middle(self,response):

[Roy Binux]

Are you actually crawling same URL?

--
You received this message because you are subscribed to the Google Groups "pyspider-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyspider-user...@googlegroups.com.
To post to this group, send email to pyspide...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyspider-users/CAH-drozOLtnA9nhpqSyj%2BscTP7YHi6CnJ15wR-2QmZnpLR1Pdw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.