How to run scrapy with spider.py file created inside the scrapy project folder

151 views
Skip to first unread message

shiva krishna

unread,
Sep 29, 2012, 12:21:53 AM9/29/12
to scrapy...@googlegroups.com
Hi i am working on scrapy,  i created a scrapy folder with `scrapy startproject example`
and written spider to scrape all the data from the url, and
I had run the spider using the command `scrapy crawl spider_name`, its working fine and able to fetch data. 

But i had a requirement that i need to run the scrapy with a single spider file created i mean a single py file something like

    pythpn -u /path/to/spider_file_inside_scrapy_folder_created.py

Is it possible to run a spider without `scrapy crawl` command after creating a scrapy project folder with spider.py file

Tsouras

unread,
Sep 29, 2012, 5:26:33 AM9/29/12
to scrapy...@googlegroups.com
You could make an .sh file (or .bat file for winlows).
example for .sh file

#!/bin/bash  
cd /path/to/your/scrapy/project
scrapy crawl [spider_name]

Steven Almeroth

unread,
Sep 30, 2012, 12:04:53 PM9/30/12
to scrapy...@googlegroups.com
Have you tried the runspider command?
$ scrapy runspider /path/to/spider_file_inside_scrapy_folder_created.py

shiva krishna

unread,
Oct 19, 2012, 8:42:24 AM10/19/12
to scrapy...@googlegroups.com
yeah thats working thankyou.................
Reply all
Reply to author
Forward
0 new messages