Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

scraping with urllib2

0 views
Skip to first unread message

Patrick

unread,
Jan 27, 2010, 12:26:14 AM1/27/10
to
I'm trying to scrape the attached link for the price listed $99.99:
http://bananarepublic.gap.com/browse/product.do?cid=41559&vid=1&pid=692392

I can see the price if I view the source(I even turned off java and
javascript), but when I use urllib2, the price doesn't show up.

Is there another library other than urllib2 that would work?


Andre Engels

unread,
Jan 27, 2010, 1:45:48 AM1/27/10
to Patrick, pytho...@python.org

To see that page you need to accept cookies from the site and send them back.

--
André Engels, andre...@gmail.com

Javier Collado

unread,
Jan 27, 2010, 6:58:34 AM1/27/10
to pytho...@python.org
Hello,

To accept cookies, use the HTTPCookieProcessor as explained here:
http://www.nomadjourney.com/2009/03/automatic-site-login-using-python-urllib2/

Best regards,
Javier

2010/1/27 Andre Engels <andre...@gmail.com>:


> On Wed, Jan 27, 2010 at 6:26 AM, Patrick <why...@gmail.com> wrote:

> To see that page you need to accept cookies from the site and send them back.
>
>
>
> --
> André Engels, andre...@gmail.com

> --
> http://mail.python.org/mailman/listinfo/python-list
>

0 new messages