scrapy error in ubuntu 10.10

794 views
Skip to first unread message

Lx Yu

unread,
Apr 29, 2011, 2:53:54 AM4/29/11
to scrapy...@googlegroups.com
1. error in easy_install
$ easy_install -U scrapy
install_dir /usr/local/lib/python2.6/dist-packages/
Searching for scrapy
Reading http://pypi.python.org/simple/scrapy/
Reading http://scrapy.org
Best match: Scrapy 0.12.0.2541
Processing Scrapy-0.12.0.2541-py2.6.egg
Scrapy 0.12.0.2541 is already the active version in easy-install.pth
Installing scrapy script to /usr/local/bin

Using /usr/local/lib/python2.6/dist-packages/Scrapy-0.12.0.2541-py2.6.egg
Processing dependencies for scrapy
Searching for Twisted>=2.5
Reading http://pypi.python.org/simple/Twisted/
Reading http://twistedmatrix.com/
Reading http://tmrc.mit.edu/mirror/twisted/Twisted/10.0/
Reading http://www.twistedmatrix.com
Reading http://twistedmatrix.com/products/download
Reading http://tmrc.mit.edu/mirror/twisted/Twisted/8.2/
Reading http://tmrc.mit.edu/mirror/twisted/Twisted/8.1/
Reading http://tmrc.mit.edu/mirror/twisted/Twisted/9.0/
Reading http://twistedmatrix.com/projects/core/
Best match: Twisted 11.0.0
Downloading http://pypi.python.org/packages/source/T/Twisted/Twisted-11.0.0.tar.bz2#md5=d7f94a1609a1b8f3b8c8d0146d4cfe54
Processing Twisted-11.0.0.tar.bz2
Running Twisted-11.0.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-ol2QlG/Twisted-11.0.0/egg-dist-tmp-SEDAa3
twisted/runner/portmap.c:10: fatal error: Python.h: No such file or directory
compilation terminated.
error: Setup script exited with error: command 'gcc' failed with exit status 1


Then I try to install using aptitude

2. error after aptitude install, and after a successful install, I get the following error:

$ aptitude install python-scrapy
$ scrapy
Traceback (most recent call last):
File "/usr/local/bin/scrapy", line 4, in <module>
import pkg_resources
File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 2671, in <module>
working_set.require(__requires__)
File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 654, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 552, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: Twisted>=2.5

$ scrapy-ctl
Traceback (most recent call last):
File "/usr/bin/scrapy-ctl", line 3, in <module>
from scrapy.command.cmdline import execute
ImportError: No module named cmdline


I'm new to scrapy and how can I solve these problem

Javier

unread,
Apr 29, 2011, 6:50:02 AM4/29/11
to scrapy-users
Try installing python-dev using 'sudo apt-get install python-dev'.

On Apr 29, 3:53 am, Lx Yu <lixinf...@gmail.com> wrote:
> 1. error in easy_install
> $ easy_install -U scrapy
> install_dir /usr/local/lib/python2.6/dist-packages/
> Searching for scrapy
> Readinghttp://pypi.python.org/simple/scrapy/
> Readinghttp://scrapy.org
> Best match: Scrapy 0.12.0.2541
> Processing Scrapy-0.12.0.2541-py2.6.egg
> Scrapy 0.12.0.2541 is already the active version in easy-install.pth
> Installing scrapy script to /usr/local/bin
>
> Using /usr/local/lib/python2.6/dist-packages/Scrapy-0.12.0.2541-py2.6.egg
> Processing dependencies for scrapy
> Searching for Twisted>=2.5
> Readinghttp://pypi.python.org/simple/Twisted/
> Readinghttp://twistedmatrix.com/
> Readinghttp://tmrc.mit.edu/mirror/twisted/Twisted/10.0/
> Readinghttp://www.twistedmatrix.com
> Readinghttp://twistedmatrix.com/products/download
> Readinghttp://tmrc.mit.edu/mirror/twisted/Twisted/8.2/
> Readinghttp://tmrc.mit.edu/mirror/twisted/Twisted/8.1/
> Readinghttp://tmrc.mit.edu/mirror/twisted/Twisted/9.0/
> Readinghttp://twistedmatrix.com/projects/core/
> Best match: Twisted 11.0.0
> Downloadinghttp://pypi.python.org/packages/source/T/Twisted/Twisted-11.0.0.tar.b...

Pablo Hoffman

unread,
Apr 29, 2011, 8:42:01 AM4/29/11
to scrapy...@googlegroups.com
Don't use the python-scrapy package provided by Ubuntu because it ships a
rather old Scrapy version (0.8). Instead, use the ubuntu packages provided by
the Scrapy project:

http://doc.scrapy.org/topics/ubuntu.html

> --
> You received this message because you are subscribed to the Google Groups "scrapy-users" group.
> To post to this group, send email to scrapy...@googlegroups.com.
> To unsubscribe from this group, send email to scrapy-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/scrapy-users?hl=en.

Lx Yu

unread,
Apr 30, 2011, 4:46:02 AM4/30/11
to scrapy...@googlegroups.com
Do you know why easy_install failed?

余立鑫 (Lx Yu), ELEME Inc.
Email: lx...@ele.me
Mobile: +86-136-0167-6914
http://ele.me 饿了么?

Leonardo Lazzaro

unread,
May 1, 2011, 10:10:44 PM5/1/11
to scrapy...@googlegroups.com
did you try sudo apt-get install python-dev as suggested?

2011/4/30 Lx Yu <lixi...@gmail.com>

Lx Yu

unread,
May 8, 2011, 4:24:58 AM5/8/11
to scrapy...@googlegroups.com
thank you, the problem solved after I installed the following two packages:
libxml2-dev
libxslt1-dev

--
Never say die

John Kabler

unread,
Feb 12, 2012, 1:14:39 AM2/12/12
to scrapy...@googlegroups.com
Thank you Thank you for taking the time to post your solution.  Here it is, 9 months later, and I had the same problem.  The python-dev package installs the libxml2-dev package, but not the libxslt1-dev.  That was what I was missing.  

You are the man.  
Reply all
Reply to author
Forward
0 new messages