Issue 146 in httplib2: [patch included] multiple headers of the same name overwrite the previous header (cookies)

[codesite...@google.com]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 146 by firefigh...@gmail.com: [patch included] multiple headers
of the same name overwrite the previous header (cookies)
http://code.google.com/p/httplib2/issues/detail?id=146

If a website sends more than one Set-Cookie header, only the last one will
be kept as Response() stores them as individual keys of the same name.

in __init__.py, class Response(dict), about line 1169

CHANGE:
if isinstance(info, http.client.HTTPResponse):
for key, value in info.getheaders():
self[key.lower()] = value

TO:
if isinstance(info, http.client.HTTPResponse):
for key, value in info.getheaders():
key_lower = key.lower()
if key_lower in self and key_lower == 'set-cookie':
""" blindly ignore RFC and append value to existing
header. some headers such as Set-Cookie, are not comma
separated, they are semicolon separated -- just do it
"""
self[key_lower] += '; '+value
else:
self[key_lower] = value

also applicable to the two elifs following.

[codesite...@google.com]

Comment #1 on issue 146 by travis.p...@gmail.com: [patch included] multiple

headers of the same name overwrite the previous header (cookies)
http://code.google.com/p/httplib2/issues/detail?id=146

http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2

"Multiple message-header fields with the same field-name MAY be present
in a message if and only if the entire field-value for that header field is
defined as a comma-separated list [i.e., #(values)]. It MUST be possible to
combine the multiple header fields into one "field-name: field-value" pair,
without changing the semantics of the message, by appending each subsequent
field-value to the first, each separated by a comma."

any http header is allowed to be present more than once as long as its
value can be specified as a comma separated list, so this isn't at all
specific to cookies.

http://tools.ietf.org/html/rfc2965#page-11

cookie = "Cookie:" cookie-version 1*((";" | ",") cookie-value)

cookies within a single header can be separated by semi-colons OR commas,
so they fit the bill for compaction into a single header, but it should be
done with commas.

patch attached (uses commas, applies the rule to any header that shows up
more than once)

Attachments:
multiheader.diff 696 bytes

[codesite...@google.com]

Comment #2 on issue 146 by travis.p...@gmail.com: [patch included] multiple

headers of the same name overwrite the previous header (cookies)
http://code.google.com/p/httplib2/issues/detail?id=146

upload.py'd

http://codereview.appspot.com/4528105/

[codesite...@google.com]

Comment #3 on issue 146 by joe.gregorio: [patch included] multiple headers

of the same name overwrite the previous header (cookies)
http://code.google.com/p/httplib2/issues/detail?id=146

Issue 144 has been merged into this issue.

[codesite...@google.com]

Comment #4 on issue 146 by firefigh...@gmail.com: [patch included] multiple

headers of the same name overwrite the previous header (cookies)
http://code.google.com/p/httplib2/issues/detail?id=146

i'm curious about the use of commas. i've been watching my Set-Cookie
responses as i visit websites etc, and am seeing the use of semi-colons
only. granted, i have my personal tastes in websites, but my experience
suggests that the use of semicolons would be more expected from app
developers.

[codesite...@google.com]

Updates:
Status: Duplicate
Mergedinto: 11

Comment #5 on issue 146 by joe.gregorio: [patch included] multiple headers

of the same name overwrite the previous header (cookies)
http://code.google.com/p/httplib2/issues/detail?id=146

(No comment was entered for this change.)

[codesite...@google.com]

Comment #6 on issue 146 by travis.p...@gmail.com: [patch included] multiple

headers of the same name overwrite the previous header (cookies)
http://code.google.com/p/httplib2/issues/detail?id=146

httplib.HTTPMessage is already doing what my patch does, though joining
by ", " instead of " , ".

I used spaces on both sides because in cursory testing with chrome I found
that was required for it to recognize that they are separate cookie key
value pairs. Semi-colons are much more commonly used for separating cookies
so a cookie-specific solution (rather than "any repeated headers", which
httplib is doing) may be in order for this.

[codesite...@google.com]

Comment #7 on issue 146 by firefigh...@gmail.com: [patch included] multiple

headers of the same name overwrite the previous header (cookies)
http://code.google.com/p/httplib2/issues/detail?id=146

it appears the cookie specific solution is needed because some servers use
commas in their expire time. the use of semicolons does appear to
dominate, probably because of this.