scrapy rules

56 views

Skip to first unread message

Игорь Горобец

unread,

Dec 21, 2016, 6:23:09 PM12/21/16

to scrapy-users

Hello. I'm

newbie in scruppy.
My code:

http://pastebin.com/fMuR0i5P

I am trying parsed all products from the site armani.com,

but I can't understand why second rule not worked after first rule. If i comment first rule - second worked and call parse, but

It does not go on all links on the site, only 'start_urls'.

I need to go through all the links within the site. If i have first rule - scrapy walking

goes on all links. Sample:

2016-12-21 23:17:46 [scrapy] DEBUG: Crawled (200) <GET http://www.armani.com/us/giorgioarmani/suit_cod49215969fb.html> (referer: http://www.armani.com/us/giorgioarmani/women/onlinestore/suits-and-jackets)
2016-12-21 23:17:46 [scrapy] DEBUG: Crawled (200) <GET http://www.armani.com/us/giorgioarmani/suit_cod49215968kj.html> (referer: http://www.armani.com/us/giorgioarmani/women/onlinestore/suits-and-jackets)
2016-12-21 23:17:47 [scrapy] DEBUG: Crawled (200) <GET http://www.armani.com/us/giorgioarmani/blazer_cod41664442ef.html> (referer: http://www.armani.com/us/giorgioarmani/women/onlinestore/suits-and-jackets)
2016-12-21 23:17:47 [scrapy] DEBUG: Crawled (200) <GET http://www.armani.com/us/giorgioarmani/women/onlinestore/suits-and-jackets/blazers> (referer: http://www.armani.com/us/giorgioarmani/women/onlinestore/suits-and-jackets)
2016-12-21 23:17:47 [scrapy] DEBUG: Crawled (200) <GET http://www.armani.com/us/giorgioarmani/blazer_cod41667734nj.html> (referer: http://www.armani.com/us/giorgioarmani/women/onlinestore/suits-and-jackets)

What am I doing wrong? sorry for my English

Sayth Renshaw

unread,

Dec 23, 2016, 3:42:30 AM12/23/16

to scrapy-users

You only have one item returned. If you want more than one item add it to the return object before its returned.