Need help to fix the error in my spider logic using xpath with firebug

6 views
Skip to first unread message

SreeSindhu Sruthi

unread,
Jul 21, 2015, 8:34:01 AM7/21/15
to scrapy...@googlegroups.com
In my spider code I have written the following for parse function:

def parse(self, response):
hxs = Selector(response)
titles = hxs.xpath(".//*[@id='requisitionListInterface.listRequisition']")
items = []
for x in titles:
item =  CrawlsiteItem()
item["title"] = x.xpath(".//@title ='View this job description'/text()").extract()
item["location"] = x.xpath(".//*[@class ='morelocation']/span/text()").extract()
item["date"] = x.xpath(".//*[@class ='jobposted']/text()").extract()
item["reqid"] = x.xpath(".//*[@class ='text']/text()").extract()
items.append(item)
return items



I tried to output the file as csv and it outputs the following:

date reqid location title
Job Posting Requisition ID

 My intention is to get all the items that are inside the .//*[@id='requisitionListInterface.listRequisition'] tag.
And the id I used is a table with in the table I want to get each element with in that.
So to get the text of all the elements that meet the @ class = 'more location' tag I used the above xpath. 
I used firebug to get the xpath. 

Please help me out to fix the error in the logic or in xpath.


Reply all
Reply to author
Forward
0 new messages