how to pass parameters to a spider in __init__ function?

1,638 views
Skip to first unread message

hongleij

unread,
Dec 31, 2010, 7:52:47 AM12/31/10
to scrapy-users
since a spider's __init__ has ''name" and "kwargs" parameters, how
could i pass args to them?

class UserInfoSpider(BaseSpider):
name = "userinfo.sjtu.edu.cn"
#allowed_domains = ["dmoz.org"]
start_urls = [
#"http://pt.sjtu.edu.cn/userdetails.php?id=125", #no user:
]
def __init__(self, name=None, **kwargs):
BaseSpider.__init__(self,name)
self.secret_users = {}

Pablo Hoffman

unread,
Dec 31, 2010, 8:03:54 AM12/31/10
to scrapy...@googlegroups.com
You can pass the arguments with the scrapy crawl command like this:

scrapy crawl myspider -a arg1=val1 -a arg2=val2

In that case, name would be "myspider" and kwargs would be:

{'arg1': 'val1', 'arg2': 'val2'}

Pablo.

> --
> You received this message because you are subscribed to the Google Groups "scrapy-users" group.
> To post to this group, send email to scrapy...@googlegroups.com.
> To unsubscribe from this group, send email to scrapy-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/scrapy-users?hl=en.

hongleij

unread,
Dec 31, 2010, 8:40:34 AM12/31/10
to scrapy-users
many Thanks! i test and the result is:
>>> name
>>> kwargs
{'arg1': 'val1', 'arg2': 'val2'}
it seems name parameter will be None

Pablo Hoffman

unread,
Dec 31, 2010, 8:56:50 AM12/31/10
to scrapy...@googlegroups.com
"name" is the spider name, but I'm not sure where you put that debugging code.
Reply all
Reply to author
Forward
0 new messages