Scrapy Response Body empty

[Andrew Zhou]

Hey I was trying to scrape from a website ("http://www.gap.com") and the response body was always empty. What could possibly be the problem and how could I fix?

[Rolando Espinoza]

This using `scrapy shell`, right?

The default request created by `fetch` does not follow redirects. So, you can create your own request object:

$ scrapy shell

>>> import scrapy

>>> fetch(scrapy.Request('http://www.gap.com'))

Rolando

On Thu, Jun 9, 2016 at 8:34 PM, Andrew Zhou <azho...@gmail.com> wrote:

Hey I was trying to scrape from a website ("http://www.gap.com") and the response body was always empty. What could possibly be the problem and how could I fix?

--
You received this message because you are subscribed to the Google Groups "scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users...@googlegroups.com.
To post to this group, send email to scrapy...@googlegroups.com.
Visit this group at https://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.