I have a question about scrapy tutorial and I put 2 questions in the
scrapy group
http://groups.google.com/group/scrapy-users/t/d5afae7d88672e02
http://groups.google.com/group/scrapy-users/t/4d52fa8c589a412a
but there were no answers. Can anyone here help me? The spider works
fine but I couldn't figure out where the items file is so that I can
print it to a file to transfer it to my database.
The same question in stackoverflow
http://stackoverflow.com/questions/1771151/newbie-q-about-scrapy-pipeline-py
Thanks!
i have no idea what scrapy is - but i see, that you try to use
idle with pipes or anything like that. That cannot work.
idle doesn't even handle __file__ correctly. Use a full-fledged
python IDE (PyDev under Eclipse leaves very few wishes open) or
test in a python interpreter shell.
Good luck, DreiJane
By the way, I realized that the spider sends the scraped data to the
pipelines.py. Now everything is working.
Scrapy is a fast high-level screen scraping and web
crawling framework, used to crawl websites and extract
structured data from their pages. It can be used for
a wide range of purposes, from data mining to monitoring
and automated testing.
Also, previously unknown to me
before google-izing ....
--
Stanley C. Kitching
Human Being
Phoenix, Arizona