Error with select-method of BeautifulSoup4

[Susannah K.]

Hi,

I've created this scraper and keep getting the following error message on line 42:

TypeError: 'NoneType' object is not callable

However, the code works fine when I execute it in my python console. What's more, the select method of BS4 cannot even return None - it will return an empty list when the selected class does not exist.

The website I'm trying to scrape is http://www.germandeli.com/toblerone10.html. It contains a div with the class I'm searching for.

When I execute the code in my python console I get this data.

It seems that somebody at StackOverflow had a similar problem with the unwrap method. Could it be a bug of ScraperWiki?

[David Jones]

On Fri, Dec 14, 2012 at 2:23 AM, Susannah K. <susann...@gmail.com> wrote:

Hi,

I've created this scraper and keep getting the following error message on line 42:

TypeError: 'NoneType' object is not callable

However, the code works fine when I execute it in my python console. What's more, the select method of BS4 cannot even return None - it will return an empty list when the selected class does not exist.

The version of BeautifulSoup4 that was installed was old even by my conservative standards and it didn't have the .select() method.

I've upgraded it and your scraper seems to be running now.

It would be nice to hear what you are doing with your scraper.  Sounds yummy.

David Jones

ScraperWiki.com

[Susannah K.]

Hi David,

You really have a great support! Thanks for your fast help. I'm working for a university project for robotic perception. The Germandeli website
has high quality photos of German supermarket products that could be used for object recognition of household robots. I wrote the scraper to be able to extend the database regularly with the newly added photos.

Thanks again

Susannah