Debug Scrapy in Eclipse/PyDev

[tpug]

Hello,

Can anyone explain how to run and debug Scrapy in a Eclipse/PyDev
environnement. I have difficulties debugging because I don't
understand the sequencing, what gets called first etc.

Any help is welcome,
Thanks in advance.

[Guy Gavriely]

Right click one of the files in your project, say, the setup.py and choose
run / debug as -> python run

it will try and fail, then edit the newly created run /debug configuration menu:

Run -> Run / Debug configuration

Then change your Main Module to where Scrapy is located, mine for example is at:

/usr/local/lib/python2.6/dist-packages/Scrapy-0.12.0.2542-py2.6.egg/scrapy/cmdline.py

Last you should set the scrapy arguments in the arguments tab, for example:

crawl my_spider_name

Guy

--
You received this message because you are subscribed to the Google Groups "scrapy-users" group.
To post to this group, send email to scrapy...@googlegroups.com.
To unsubscribe from this group, send email to scrapy-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/scrapy-users?hl=en.

[Pablo Hoffman]

I never used Eclipse/PyDev (I'm a vim user myself) but it'd be nice if we can
put together some notes/tips (like those) on how to do Scrapy development in
Eclipse.

And any ideas for improving development are always welcome. Scrapy (being a
framework that it is, and not a library) has always taken development
productivity as one of its goals.

[Ensnap]

I have been using scrapy with Aptana studio (free & based on eclipse).
Once you create a project and have used the same technique as @Guy.
I've found Aptana very useful for looking up class definitions,
searching across projects but it's heavy on memory so I usually rely
on console.

On Jun 20, 8:56 pm, Pablo Hoffman <pablohoff...@gmail.com> wrote:
> I never used Eclipse/PyDev (I'm a vim user myself) but it'd be nice if we can
> put together some notes/tips (like those) on how to do Scrapy development in
> Eclipse.
>
> And any ideas for improving development are always welcome. Scrapy (being a
> framework that it is, and not a library) has always taken development
> productivity as one of its goals.
>
>
>
>
>
>
>
> On Sun, Jun 19, 2011 at 02:58:37PM +0300, Guy Gavriely wrote:
> > Right click one of the files in your project, say, the setup.py and choose
> > run / debug as -> python run
>
> > it will try and fail, then edit the newly created run /debug configuration
> > menu:
>
> > Run -> Run / Debug configuration
>
> > Then change your Main Module to where Scrapy is located, mine for example is
> > at:
>
> > /usr/local/lib/python2.6/dist-packages/Scrapy-0.12.0.2542-py2.6.egg/scrapy/ cmdline.py
>
> > Last you should set the scrapy arguments in the arguments tab, for example:
>
> > crawl my_spider_name
>
> > Guy
>

[Aniruddha Gaikwad]

Hi,

this might be helpful : http://stackoverflow.com/questions/9809656/scrapy-eclipse-pydev-how-to-setup-the-debugger