Missing header on "fetch --headers" output

11 views
Skip to first unread message

Marcus

unread,
Oct 31, 2013, 11:16:13 AM10/31/13
to scrapy...@googlegroups.com
The command fetch doesn't seem to report all of the response headers as expected.

Scenario: fresh install of scrapy 0.18.4 with no custom settings, run the command fetch with added option --headers on any URL.

Checking the reported response headers and adding the response body size, the sum of their contents (in bytes) does not match the value reported by 'downloader/response_bytes'.
Of course I've ignored the ">" and "<" and added the extra \r\n that the response headers always sends.
So, I've noticed that the difference in the value reported by 'downloader/response_bytes' and the resulting body+header is down to a single missing response.header.
By missing header I mean, a header not output by the fetch command.



Pablo Hoffman

unread,
Apr 18, 2014, 3:19:13 PM4/18/14
to scrapy-users
Which header was that?. Is there a way to reproduce it with a specific url  without any other dependencies?. If this is the case, you should open a defect in github.


--
You received this message because you are subscribed to the Google Groups "scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users...@googlegroups.com.
To post to this group, send email to scrapy...@googlegroups.com.
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/groups/opt_out.

Reply all
Reply to author
Forward
0 new messages