passing arguments to scrapyd

[Anne Schuth]

hi all,

I'm trying to pass arguments to the schedule.json function of scrapyd as described on http://doc.scrapy.org/topics/scrapyd.html#schedule-json:
"any other parameter is passed as spider argument"

So, I've tried the following:

-d set="MYVAR=5"
-d set=MYVAR=5
-d MYVAR=5

None of these options work as I expect (ie, the setting MYVAR is not overwritten). How should I pass in parameters?



Thank you for your time!
Anne

--
Anne Schuth
ILPS - ISLA - FNWI
University of Amsterdam
Science Park 904, C3.230
1098 XH AMSTERDAM
The Netherlands
0031 (0) 20 525 5357

[vitsin]

following worked fine for me:
http://localhost:6800/schedule.json -d project=my_project -d
spider=test001 -d my_custom_param=my_custom_value ... etc

--vs

On Jun 22, 11:31 am, Anne Schuth <anne.sch...@uva.nl> wrote:
> hi all,
>
> I'm trying to pass arguments to the schedule.json function of scrapyd as

> described onhttp://doc.scrapy.org/topics/scrapyd.html#schedule-json:

[Anne Schuth]

Thanks for the quick reply! Good to hear that it can work :)

Can you tell how you read out your parameter "my_custom_param" in your spider?

I'm trying to have the parameter overwrite the settings, just like in this call:

$ scrapy crawl test001 --set MYVAR=5

Best,

Anne

--
Anne Schuth
ILPS - ISLA - FNWI
University of Amsterdam
Science Park 904, C3.230
1098 XH AMSTERDAM
The Netherlands
0031 (0) 20 525 5357

--
You received this message because you are subscribed to the Google Groups "scrapy-users" group.
To post to this group, send email to scrapy...@googlegroups.com.
To unsubscribe from this group, send email to scrapy-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/scrapy-users?hl=en.

[vitsin]

from within spider:

def __init__(self, *args, **kwargs):
super(mySpider, self).__init__(*args, **kwargs)
my_custom_value = kwargs.get('my_custom_param')


--vs

On Jun 22, 11:44 am, Anne Schuth <anne.sch...@uva.nl> wrote:
> Thanks for the quick reply! Good to hear that it can work :)
>
> Can you tell how you read out your parameter "my_custom_param" in your
> spider?
>
> I'm trying to have the parameter overwrite the settings, just like in this
> call:
>
> $ scrapy crawl test001 --set MYVAR=5
>
> Best,
> Anne
>
> --
> Anne Schuth
> ILPS - ISLA - FNWI
> University of Amsterdam
> Science Park 904, C3.230
> 1098 XH AMSTERDAM
> The Netherlands
> 0031 (0) 20 525 5357
>
>
>
>
>
>
>
> On Wed, Jun 22, 2011 at 17:38, vitsin <vitaly.sinit...@gmail.com> wrote:
> > following worked fine for me:

> >http://localhost:6800/schedule.json-d project=my_project -d

[vitsin]

scrapy crawl test001 -a MYVAR=5

On Jun 22, 11:44 am, Anne Schuth <anne.sch...@uva.nl> wrote:
> Thanks for the quick reply! Good to hear that it can work :)
>
> Can you tell how you read out your parameter "my_custom_param" in your
> spider?
>
> I'm trying to have the parameter overwrite the settings, just like in this
> call:
>
> $ scrapy crawl test001 --set MYVAR=5
>
> Best,
> Anne
>
> --
> Anne Schuth
> ILPS - ISLA - FNWI
> University of Amsterdam
> Science Park 904, C3.230
> 1098 XH AMSTERDAM
> The Netherlands
> 0031 (0) 20 525 5357
>
>
>
>
>
>
>
> On Wed, Jun 22, 2011 at 17:38, vitsin <vitaly.sinit...@gmail.com> wrote:
> > following worked fine for me:

> >http://localhost:6800/schedule.json-d project=my_project -d

[Anne Schuth]

right! thanks a lot!

--
Anne Schuth
ILPS - ISLA - FNWI
University of Amsterdam
Science Park 904, C3.230
1098 XH AMSTERDAM
The Netherlands
0031 (0) 20 525 5357