Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Newbie question about scrapy tutorial

10 views
Skip to first unread message

Zeynel

unread,
Nov 20, 2009, 11:46:30 AM11/20/09
to
Hello,

I have a question about scrapy tutorial and I put 2 questions in the
scrapy group

http://groups.google.com/group/scrapy-users/t/d5afae7d88672e02
http://groups.google.com/group/scrapy-users/t/4d52fa8c589a412a

but there were no answers. Can anyone here help me? The spider works
fine but I couldn't figure out where the items file is so that I can
print it to a file to transfer it to my database.

The same question in stackoverflow
http://stackoverflow.com/questions/1771151/newbie-q-about-scrapy-pipeline-py

Thanks!

DreiJane

unread,
Nov 21, 2009, 11:21:26 AM11/21/09
to
Sorry,

i have no idea what scrapy is - but i see, that you try to use
idle with pipes or anything like that. That cannot work.
idle doesn't even handle __file__ correctly. Use a full-fledged
python IDE (PyDev under Eclipse leaves very few wishes open) or
test in a python interpreter shell.

Good luck, DreiJane

Zeynel

unread,
Nov 21, 2009, 2:00:58 PM11/21/09
to
Now I use EditpadPro and IDLE which appears to be adequate for
beginning level. But PyDev looks fun, I'll try it.

By the way, I realized that the spider sends the scraped data to the
pipelines.py. Now everything is working.

Cousin Stanley

unread,
Nov 21, 2009, 2:10:47 PM11/21/09
to

> i have no idea what scrapy is
> ....

http://scrapy.org/

Scrapy is a fast high-level screen scraping and web
crawling framework, used to crawl websites and extract
structured data from their pages. It can be used for
a wide range of purposes, from data mining to monitoring
and automated testing.

Also, previously unknown to me
before google-izing ....


--
Stanley C. Kitching
Human Being
Phoenix, Arizona

0 new messages