deploy scrapy and encoding difficulties

[gulan]

Hi,

I have two problems that's been bugging me. I think they're pretty
easy to solve, but I have had no luck.

a )Deploying project to scrapyd

1. I have managed to start scrapyd on my local machine and I can see
the UI if i visit http://localhost: 6800

2. In the scrapy.cfg file on my project I have:

[settings]
default = kalender.settings

[deploy:project1]
url = http://localhost:6800/
project = kalender

3. When I type scrapy deploy -l I get the correct response:

project1 http://localhost:6800/


4. But when I try to deploy with "scrapy deploy scrapyd -p project1" I
get the following error:
Usage
=====
scrapy deploy [options] [ [target] | -l | -L <target> ]

deploy: error: Unknown target: scrapyd


I follow the tutorial here:
http://doc.scrapy.org/en/latest/topics/scrapyd.html

Any ideas whta's going on here?

b) Decoding problems
I'm scraping a swedish site and when I export to json the swedish
signs å,ä, ö is not included properly. In the default settings file I
have changed to the following:

DEFAULT_REQUEST_HEADERS = {
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/
*;q=0.8',
'Accept-Language': 'sv',
}

DEFAULT_RESPONSE_ENCODING = 'utf-8'

But this doesn't seem to work. So, how can I get non latin charachters
working properly in the exports?

Any help much appreciated!

/Gustav

[shiva krishna]

Hi Gustav 

i am facing the same problem as u faced in (a) 

i mean 

4. But when I try to deploy with "scrapy deploy scrapyd -p project1" I 
get the following error: 
Usage 
===== 
  scrapy deploy [options] [ [target] | -l | -L <target> ] 

deploy: error: Unknown target: scrapyd 

How did u solved this........

[Steven Almeroth]

a) You deploy targets, and optionally, projects within the target, try:

[deploy:scrapyd] 

url = http://localhost:6800/ 

project = kalender 

$ scrapy deploy scrapyd -p kalender

[shiva krishna]

when i do in the above mentioned format i got the following error

sh-4.2$ scrapy deploy ebsite -p ebsite

Building egg of ebsite-1341808737

'build/scripts-2.7' does not exist -- can't clean it

zip_safe flag not set; analyzing archive contents...

Deploying ebsite-1341808737 to http://localhost:6800/addversion.json

Deploy failed: <urlopen error [Errno 111] Connection refused>

How to solve this..............

--
You received this message because you are subscribed to the Google Groups "scrapy-users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/scrapy-users/-/WzCLlbaH2tMJ.

To post to this group, send email to scrapy...@googlegroups.com.
To unsubscribe from this group, send email to scrapy-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/scrapy-users?hl=en.

[Gustav Julander]

Hello, 

I solved it the way Steven Almeroth shoved in the previous post. I do not recognize your error message, sorry.

/Gustav

2012/7/9 shiva krishna <shivak...@gmail.com>

[shiva krishna]

No problem Gustav,

Actually i too solved that by deploying in to scrapyd, but the problem is  here i had multiple spiders in a single project , so even though we uploaded it in to scrapyd and by using curl command we can run one spider at a time, then there is no use of uploading it in to scrapyd, just scrapy crawl spider_name is enough right?

Thanks in advance......