In my spider code I have written the following for parse function:
def parse(self, response):
hxs = Selector(response)
titles = hxs.xpath(".//*[@id='requisitionListInterface.listRequisition']")
items = []
for x in titles:
item = CrawlsiteItem()
item["title"] = x.xpath(".//@title ='View this job description'/text()").extract()
item["location"] = x.xpath(".//*[@class ='morelocation']/span/text()").extract()
item["date"] = x.xpath(".//*[@class ='jobposted']/text()").extract()
item["reqid"] = x.xpath(".//*[@class ='text']/text()").extract()
items.append(item)
return items
I tried to output the file as csv and it outputs the following:
date |
reqid |
location |
title |
Job Posting |
Requisition ID |
|
My intention is to get all the items that are inside the .//*[@id='requisitionListInterface.listRequisition'] tag.
And the id I used is a table with in the table I want to get each element with in that.
So to get the text of all the elements that meet the @ class = 'more location' tag I used the above xpath.
I used firebug to get the xpath.
Please help me out to fix the error in the logic or in xpath.