Go 1.1 cookiejar
762 views
Skip to first unread message
Nigel Tao
Jan 25, 2013, 6:17:20 AM1/25/13
to Volker Dobler, Brad Fitzpatrick, Russ Cox, golang-dev
Summary: I am proposing dropping persistent storage and other
programmatic cookie access (outside that needed by package http) from
package cookiejar for Go 1.1.
With help from Volker Dobler, I'm working on getting exp/cookiejar up
for Go 1.1. The design process started with us trading CLs at 10 paces
(e.g. [0], [1], [2], [3]), moved to a shared document ([4] points to
[5]), and then moved to private e-mail exchanges. I'd like to get the
conversation back to the golang-dev list.
All this back-and-forth proves that there does not exist a unique,
obviously correct design.
One interesting design decision revolves around a storage interface.
It would be good if the cookie jar could persist its cookies, to files
on disk, to a database, or something else. Ideally the implementor of
an SQLite-backed cookie jar storage would only have to know about
SQLite, and not the intricacies of RFC 6265 ("HTTP State Management
System", i.e. cookies). On the other hand, the amount of I/O required
can become more efficient the more cookie-aware the storage is. It is
not obvious what the right trade-off is.
A second interesting design decision is how much should a cookie jar
expose its stored cookies, over and above the http.Cookiejar
interface. On representation, there is an existing http.Cookie type,
unchanged in tip since Go 1.0. It does not implement all the RFC 6265
fields; it is missing creation and last-access times, and persistent
and host-only bits. On the other hand, the http.Cookie struct contains
four fields (Raw, RawExpires, MaxAge, Unparsed) that are relevant for
a cookie parsed from an HTTP response but might not apply to one
reconstituted from storage. Should http.Cookie gain more fields or
should there be a separate (exported) cookiejar.Cookie type? If the
latter, should it embed an http.Cookie or should it have similarly
named and typed fields but be otherwise unrelated? On exposure, can
programs explicitly add or delete cookies without going through the
Cookies or SetCookies methods? Can programs enumerate the domains for
which the jar contains cookies? Does the jar just provide that "I have
cookies for example.com", or is it "I have 4KiB of cookies", or is it
"I have 7 cookies and here are all their details"? What is the
enumeration mechanism: an eagerly filled slice, a channel, an
iterator, closure execution, or something else? Note that, by design,
a cookie jar must be usable (and modifiable) concurrently. Do we need
some notion of transactions? How do particular mechanisms restrict
possible Jar and Storage implementations? Will Storage require
explicit locking?
To cut a long story short, it's not clear what the 'right' cookiejar
design is. We will have to maintain backwards compatability with any
net/cookiejar package that we publish in Go 1.1 for years to come. For
any exported interface type, backwards compatibility means that we can
neither add nor remove methods, ever. I am hesitant about prematurely
committing to a particular design, and I don't think we will know the
right design until we actually try to build a variety of programs with
it.
Given that Go 1.1 is only months away, I propose to reduce the scope
of package cookiejar to simply be an in-memory implementation of
http.CookieJar that has no extra functionality: no optional persistent
storage and no additional programmatic cookie access. Richer
implementations can easily live on Github, Google Code Hosting,
Bitbucket, etc., and be just a "go get" away. The proposed Go 1.1 API
is:
--------
// Package cookiejar implements an in-memory RFC 6265-compliant http.CookieJar.
package cookiejar
// Jar implements the http.CookieJar interface from the net/http package.
type Jar struct {
// Contains unexported fields.
}
type PublicSuffixList interface {
PublicSuffix(domain string) string
String() string
}
type Options struct {
// PublicSuffixList is the public suffix list that determines whether an
// HTTP server can set a cookie for a domain. A nil value is valid and
// may be useful for testing but it is not secure: it means that the
// HTTP server for foo.com can set a cookie for bar.com.
PublicSuffixList PublicSuffixList
// Other options may be added in Go 1.2 and beyond.
}
func New(o *Options) (*Jar, error)
func (j *Jar) Cookies(u *url.URL) []*http.Cookie
func (j *Jar) SetCookies(u *url.URL, cookies []*http.Cookie)
// Other *Jar methods may be added in Go 1.2 and beyond.
--------
This cookie jar won't be all things to all people, but it will be
useful (I would have used it, if it existed, earlier this week for
another project), it has an API surface area that I am happy to
maintain (and possibly grow) for years to come, and on my reading it
can be RFC 6265 compliant. On storage, RFC section 5.3 "Storage model"
says that the user agent MUST remove non-persistent cookies at session
end, but doesn't say that persistent cookies must actually persist,
and that's in fact what "Incognito" or "Private Browsing" modes do. On
cookie access, RFC section 7.2 "User Controls" has language like "user
agents SHOULD provide", "might let", and "many user agents... let
users examine the cookies", but SHOULD is not MUST. I'm not saying
that we never want to support such a feature; I'm just saying that
it's out of scope for Go 1.1.
Comments welcome.
[0] https://codereview.appspot.com/6846125/
[1] https://codereview.appspot.com/6854114/
[2] https://codereview.appspot.com/6944067/
[3] https://codereview.appspot.com/6996044/
[4] https://groups.google.com/d/msg/golang-dev/LTs6PRAFcps/B6VZF1TrfSYJ
[5] https://docs.google.com/document/d/1w8VHqIGBmud9huUQ_nBNqOSqufqoxuSs93ZrXVvut_Q/edit
programmatic cookie access (outside that needed by package http) from
package cookiejar for Go 1.1.
With help from Volker Dobler, I'm working on getting exp/cookiejar up
for Go 1.1. The design process started with us trading CLs at 10 paces
(e.g. [0], [1], [2], [3]), moved to a shared document ([4] points to
[5]), and then moved to private e-mail exchanges. I'd like to get the
conversation back to the golang-dev list.
All this back-and-forth proves that there does not exist a unique,
obviously correct design.
One interesting design decision revolves around a storage interface.
It would be good if the cookie jar could persist its cookies, to files
on disk, to a database, or something else. Ideally the implementor of
an SQLite-backed cookie jar storage would only have to know about
SQLite, and not the intricacies of RFC 6265 ("HTTP State Management
System", i.e. cookies). On the other hand, the amount of I/O required
can become more efficient the more cookie-aware the storage is. It is
not obvious what the right trade-off is.
A second interesting design decision is how much should a cookie jar
expose its stored cookies, over and above the http.Cookiejar
interface. On representation, there is an existing http.Cookie type,
unchanged in tip since Go 1.0. It does not implement all the RFC 6265
fields; it is missing creation and last-access times, and persistent
and host-only bits. On the other hand, the http.Cookie struct contains
four fields (Raw, RawExpires, MaxAge, Unparsed) that are relevant for
a cookie parsed from an HTTP response but might not apply to one
reconstituted from storage. Should http.Cookie gain more fields or
should there be a separate (exported) cookiejar.Cookie type? If the
latter, should it embed an http.Cookie or should it have similarly
named and typed fields but be otherwise unrelated? On exposure, can
programs explicitly add or delete cookies without going through the
Cookies or SetCookies methods? Can programs enumerate the domains for
which the jar contains cookies? Does the jar just provide that "I have
cookies for example.com", or is it "I have 4KiB of cookies", or is it
"I have 7 cookies and here are all their details"? What is the
enumeration mechanism: an eagerly filled slice, a channel, an
iterator, closure execution, or something else? Note that, by design,
a cookie jar must be usable (and modifiable) concurrently. Do we need
some notion of transactions? How do particular mechanisms restrict
possible Jar and Storage implementations? Will Storage require
explicit locking?
To cut a long story short, it's not clear what the 'right' cookiejar
design is. We will have to maintain backwards compatability with any
net/cookiejar package that we publish in Go 1.1 for years to come. For
any exported interface type, backwards compatibility means that we can
neither add nor remove methods, ever. I am hesitant about prematurely
committing to a particular design, and I don't think we will know the
right design until we actually try to build a variety of programs with
it.
Given that Go 1.1 is only months away, I propose to reduce the scope
of package cookiejar to simply be an in-memory implementation of
http.CookieJar that has no extra functionality: no optional persistent
storage and no additional programmatic cookie access. Richer
implementations can easily live on Github, Google Code Hosting,
Bitbucket, etc., and be just a "go get" away. The proposed Go 1.1 API
is:
--------
// Package cookiejar implements an in-memory RFC 6265-compliant http.CookieJar.
package cookiejar
// Jar implements the http.CookieJar interface from the net/http package.
type Jar struct {
// Contains unexported fields.
}
type PublicSuffixList interface {
PublicSuffix(domain string) string
String() string
}
type Options struct {
// PublicSuffixList is the public suffix list that determines whether an
// HTTP server can set a cookie for a domain. A nil value is valid and
// may be useful for testing but it is not secure: it means that the
// HTTP server for foo.com can set a cookie for bar.com.
PublicSuffixList PublicSuffixList
// Other options may be added in Go 1.2 and beyond.
}
func New(o *Options) (*Jar, error)
func (j *Jar) Cookies(u *url.URL) []*http.Cookie
func (j *Jar) SetCookies(u *url.URL, cookies []*http.Cookie)
// Other *Jar methods may be added in Go 1.2 and beyond.
--------
This cookie jar won't be all things to all people, but it will be
useful (I would have used it, if it existed, earlier this week for
another project), it has an API surface area that I am happy to
maintain (and possibly grow) for years to come, and on my reading it
can be RFC 6265 compliant. On storage, RFC section 5.3 "Storage model"
says that the user agent MUST remove non-persistent cookies at session
end, but doesn't say that persistent cookies must actually persist,
and that's in fact what "Incognito" or "Private Browsing" modes do. On
cookie access, RFC section 7.2 "User Controls" has language like "user
agents SHOULD provide", "might let", and "many user agents... let
users examine the cookies", but SHOULD is not MUST. I'm not saying
that we never want to support such a feature; I'm just saying that
it's out of scope for Go 1.1.
Comments welcome.
[0] https://codereview.appspot.com/6846125/
[1] https://codereview.appspot.com/6854114/
[2] https://codereview.appspot.com/6944067/
[3] https://codereview.appspot.com/6996044/
[4] https://groups.google.com/d/msg/golang-dev/LTs6PRAFcps/B6VZF1TrfSYJ
[5] https://docs.google.com/document/d/1w8VHqIGBmud9huUQ_nBNqOSqufqoxuSs93ZrXVvut_Q/edit
Dave Cheney
Jan 25, 2013, 6:51:58 AM1/25/13
to Nigel Tao, Volker Dobler, Brad Fitzpatrick, Russ Cox, golang-dev
Hi Nigel,
I understand your concerns about adding a new perpetual API for
cookiejars in Go 1.1. Can I suggest that this package be incubated in
the go.net subrepo instead ?
Cheers
Dave
> --
>
>
>
I understand your concerns about adding a new perpetual API for
cookiejars in Go 1.1. Can I suggest that this package be incubated in
the go.net subrepo instead ?
Cheers
Dave
>
>
>
Russ Cox
Jan 25, 2013, 9:35:00 AM1/25/13
to Dave Cheney, Nigel Tao, Volker Dobler, Brad Fitzpatrick, golang-dev
On Fri, Jan 25, 2013 at 6:51 AM, Dave Cheney <da...@cheney.net> wrote:
Hi Nigel,
I understand your concerns about adding a new perpetual API for
cookiejars in Go 1.1. Can I suggest that this package be incubated in
the go.net subrepo instead ?
I have been pushing to include at least minimal cookie support in Go 1.1. I believe the standard libraries should be able to do a simple sequence of web interactions that depend on cookie state "out of the box" (for example, logging into a web site and downloading a file using that credential), without having to add new packages, even ones in subrepos. There are many non-storage details involved in doing that correctly, and I think we should try to get at least that non-storage functionality into Go 1.1. I wouldn't mind seeing storage too, but I also wouldn't mind if storage is deferred.
Russ
Brad Fitzpatrick
Jan 25, 2013, 3:22:16 PM1/25/13
to Nigel Tao, Volker Dobler, Russ Cox, golang-dev
On Fri, Jan 25, 2013 at 3:17 AM, Nigel Tao <nige...@golang.org> wrote:
Summary: I am proposing dropping persistent storage and other
programmatic cookie access (outside that needed by package http) from
package cookiejar for Go 1.1.
SGTM. You've presented a scary list of questions. In-memory for Go 1.1 seems fine, letting the community handle persistence for now.
The proposed Go 1.1 API
is:
--------
// Package cookiejar implements an in-memory RFC 6265-compliant http.CookieJar.
package cookiejar
// Jar implements the http.CookieJar interface from the net/http package.
type Jar struct {
// Contains unexported fields.
}
type PublicSuffixList interface {
PublicSuffix(domain string) string
String() string
}
type Options struct {
// PublicSuffixList is the public suffix list that determines whether an
// HTTP server can set a cookie for a domain. A nil value is valid and
// may be useful for testing but it is not secure: it means that the
// HTTP server for foo.com can set a cookie for bar.com.
PublicSuffixList PublicSuffixList
It'd be nice if a nil *Options meant a zero Options, and then a zero PublicSuffixList meant something safe. If somebody wants a legitimately zero PublicSuffixList, I'd like them to have to say cookiejar.NoPublicSuffixList (a default impl we provide).
Also document this zero behavior and that it's in-memory only (in the package doc and/or on New)
Volker Dobler
Jan 25, 2013, 4:15:12 PM1/25/13
to Brad Fitzpatrick, Nigel Tao, Russ Cox, golang-dev
Nigel lists all the facts, his arguments are valid, the reasoning is sound:
Thus his conclusion is correct.
On Fri, Jan 25, 2013 at 9:22 PM, Brad Fitzpatrick <brad...@golang.org> wrote:
On Fri, Jan 25, 2013 at 3:17 AM, Nigel Tao <nige...@golang.org> wrote:
Summary: I am proposing dropping persistent storage and other
programmatic cookie access (outside that needed by package http) from
package cookiejar for Go 1.1.SGTM. You've presented a scary list of questions. In-memory for Go 1.1 seems fine, letting the community handle persistence for now
The questions themselves do not scare me, it just seems that any answer
has at least one obvious fault.
It'd be nice if a nil *Options meant a zero Options, and then a zero PublicSuffixList meant something safe. If somebody wants a legitimately zero PublicSuffixList, I'd like them to have to say cookiejar.NoPublicSuffixList (a default impl we provide).
Reply all
Reply to author
Forward
0 new messages