Spider Contracts
Pablo Hoffman
We're working on a new Scrapy feature to make it easier to test your spiders.
It's called "Spider Contracts" and here's the SEP:
http://dev.scrapy.org/wiki/SEP-017
We'd love to hear your feedback.
Pablo.
Ping Yin
>
> We're working on a new Scrapy feature to make it easier to test your spiders.
> It's called "Spider Contracts" and here's the SEP:
> http://dev.scrapy.org/wiki/SEP-017
>
Wow, great feature, thanks! However, i expect more tags such as
@expected name foo
@expected email f...@example.com
Not familiar with the contract syntax, i hope my meaning is clear.
Pablo Hoffman
> On Thu, Jun 10, 2010 at 9:44 PM, Pablo Hoffman <pabloh...@gmail.com> wrote:
> >
> > We're working on a new Scrapy feature to make it easier to test your spiders.
> > It's called "Spider Contracts" and here's the SEP:
> > http://dev.scrapy.org/wiki/SEP-017
> >
>
> Wow, great feature, thanks!
You're welcome, thanks for the feedback :)
> However, i expect more tags such as
>
> @expected name foo
> @expected email f...@example.com
>
> Not familiar with the contract syntax, i hope my meaning is clear.
It's clear, and it can be done, but I think checking for the exact value of
scraped fields may be harder to maintain, since you'll have to update those
values every time you update the sample url (for example, when a product goes
unavailable and the page returns 404). So I think checking for scraped fields
present is more reliable and easier to maintain, but it can be done
nevertheless.
Pablo.
Ping Yin
Exact value is just an example. The regex match may be more flexible.
Just checking whether the field exists is not enough for me. The field
may exist, however, the value extracted may be wrong.
