Create a project with Portia and run it in Scrapy

361 views
Skip to first unread message

Christoph

unread,
Jun 5, 2014, 3:52:19 PM6/5/14
to portia-...@googlegroups.com
Hi,

is it possible to create a project with Portia and run this project/spider with scrapy?
I would like to create a project with portia, change some additional spider settings, deploy it to my server where scrapy is installed and run/control the spider.

Simply doing a "scrapy crawl mySpider" didn't work.

Any advice?

Thanks,
Christoph

Marcos Campal

unread,
Jun 5, 2014, 3:58:15 PM6/5/14
to Christoph, portia-...@googlegroups.com
Christoph,

You can't simply do it with scrapy, you need to use portiacrawl (which in turn is based on scrapy). Have a look at:

Cheers


--
You received this message because you are subscribed to the Google Groups "portia-scraper" group.
To unsubscribe from this group and stop receiving emails from it, send an email to portia-scrape...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Christoph

unread,
Jun 5, 2014, 4:30:46 PM6/5/14
to portia-...@googlegroups.com, skag...@googlemail.com
Thanks for the quick reply.
I think I cannot use scrapyd to deploy and control spiders created with portia, right?

I thought Portia was just the visual scraping tool which creates projects that scrapy can use.

Any other limitation that Portia have compared to scrapy?

- Christoph
Reply all
Reply to author
Forward
0 new messages