Scrapy 0.24.4 - exports to csv, but the csv file is empty

[ara]

Hi,

Can anyone tell me how to fix this: 

I am using Scrapy 0.24.4, Python 2.7: things are fine when I scrape within Terminal. However, when I try to export it as CSV, it creates a CSV file, but completely empty.

Here's what the terminal shows (note: I had 3 urls that I was scraping as a test:

2014-11-05 10:47:09-0500 [file_nov2014] INFO: Closing spider (finished)

2014-11-05 10:47:09-0500 [file_nov2014] INFO: Dumping Scrapy stats:

{'downloader/request_bytes': 688,

'downloader/request_count': 3,

'downloader/request_method_count/GET': 3,

'downloader/response_bytes': 60910,

'downloader/response_count': 3,

'downloader/response_status_count/200': 3,

'file_count': 6,

'file_status_count/uptodate': 6,

'finish_reason': 'finished',

'finish_time': datetime.datetime(2014, 11, 5, 15, 47, 9, 188837),

'log_count/DEBUG': 11,

'log_count/ERROR': 3,

'log_count/INFO': 7,

'response_received_count': 3,

'scheduler/dequeued': 3,

'scheduler/dequeued/memory': 3,

'scheduler/enqueued': 3,

'scheduler/enqueued/memory': 3,

'start_time': datetime.datetime(2014, 11, 5, 15, 47, 7, 935771)}

2014-11-05 10:47:09-0500 [file_nov2014] INFO: Spider closed (finished)

How do I get Scrapy to write to the CSv file? 

Thanks.

-Ara

[Travis Leleu]

​If that's all the output you're getting, it appears that your spider doesn't actually extract the item data and yield it.  With debug loglevel, you should see the extracted data written to stdout.​

--
You received this message because you are subscribed to the Google Groups "scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users...@googlegroups.com.
To post to this group, send email to scrapy...@googlegroups.com.
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.

[Anjali Arora]

Thanks but I do get output to the terminal (mac) window. I can see all the data is fetched in, images too are getting downloaded fine . It simply won't write anything to csv.

I am noticing this problem since upgrading to the latest Scrapy. Any ideas? 

--
You received this message because you are subscribed to a topic in the Google Groups "scrapy-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/scrapy-users/zHKoujfJvzs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to scrapy-users...@googlegroups.com.

[Capi Etheriel]

the log output indicates your spider is not finding any items, if it did there would be a line like:

'item_scraped_count': 7,

[vivian Y]

In your parse or other functions, did you return items or return item? If not, you will not see any items even if you can print them in your terminal. 

在 2014年11月5日星期三UTC-6上午9时59分59秒，ara写道：

[Tuấn Lê]

Do you return item?

[Anjali Arora]

Thanks everyone. Turns out my spider file was fine, but strangely, my pipeline file had some weird code inserted in that was not fetching rows for which the spider was unable to download images (due to the pipeline.py bug). And since the spider was unable to get imagex for any urls in the file, it was not writing the data to csv. As soon as we removed the strange code in the pipeline file, things are running smoothly.

Thanks again.

--