A proposal for an IMAP client API
Andrea Fazzi
I've started implemening an IMAPv4rev1 (RFC3501)[1] client in Go. As
Russ suggested months ago[2], I'm posting an initial proposal for an
API before there is too much implementation. The API is not complete
but - I think - there's enough to figure out how does it look and if
I'm taking a wrong directions. Of course, I can expand on some point
on request. In any case, I'd like to receive your feedback. Thanks in
advance.
Andrea
# API
This initial API is modeled after the smtp[3] package. The package
name for the imap client could be "imap".
## Client/Server interaction
At a basic level, an IMAP client-server interaction is represented by
the exchange of textual messages between the two endpoints. The client
sends a command to the server and waits for the response. The server
responds to the client request with one or more messages. Eventually,
the server asks the client to complete its request sending the
remaining part. Server responses are in three forms: status, data and
continuation request.
### Status responses
A status response can be tagged or untagged.
A tagged response refers to a previous request from the client. In
other words, a tagged status response says if the client command as
succeeded or not. There are three tagged status responses, one
positive and two negative.
OK: command completed successfully (positive response)
NO: command failure (negative response)
BAD: command unknown or argument invalid (negative response)
In the present proposal, a negative tagged status response received
from the server after issuing a method is represented by an os.Error
object returned by that method. For example the following code
if _, err := client.Login("user", "wrong password"); err != nil {
fmt.Println(err)
}
prints the following error on stdout:
NO Invalid credentials
Of course, a positive status response doesn't produce errors (err ==
nil).
Untagged status responses are informative responses received from the
server during operations. From the API point-of-view these responses
are not always relevant. For example, during a logout operation, the
server sends to the client an untagged BYE status message. I think
that this message should not be returned to the user (with user !=
client). In this way, we can keep the signature for Logout as simple
as:
func (c *Client) Logout() os.Error
rather than
func (c *Client) Logout() (Response, os.Error)
Shortly: the API should returns relevant responses only.
### Data responses
There are server responses that produce data. Data responses provide
informations about the server and the mailboxes on the server. The
type of a data response changes according to the request issued by the
client. As an example, let's consider the CAPABILITY command example:
// Capability requests a listing of capabilities that the server supports.
func (c *Client) Capability() ([]string, os.Error)
In the case of Capability the response is a list of server's
capabilities (in the form of a string slice).
As another example, let's consider the List command:
// List retrieves a subset of names from the complete set of all names
// available to the client. reference provides a context (for
// instance, a base directory in a directory-based mailbox hierarchy).
// mailbox specifies a mailbox or (via wildcards) mailboxes under that
// context.
func (c *Client) List(reference, mailbox string) ([]ListData, os.Error)
where ListData is a struct type defined as follow:
type ListData struct {
Attr []string
Delimiter, Name string
}
As a pretty problematic case, let's consider the Fetch method:
// Fetch retrieves data associated with the message in the
// mailbox. The set argument can be a sequence number, an array of
// sequence numbers or a string in the form "n:m" representing a set
// of sequence numbers (see RFC3501 Section 6.4.5). The item
// argument represents the part of the message that should be
// fetched (e.g. BODY, BODYSTRUCTURE, ENVELOPE, etc.)
func (c *Client) Fetch(set interface{}, item string) (?, os.Error)
I'm not sure about what to return from Fetch. Two options come to
mind:
* ? = io.Reader: Leave the server response as is providing a io.Reader
for consuming it later. It's up to the user to parse the data
associated with the message.
* ? = interface{}: Unmarshal the response from the server filling an
interface{} value. For example:
bodyText, _ := imap.Fetch(1, "BODY[TEXT]")
bodyText is a string representing the textual content of the message
as described in RFC2822. However, the imap package will not provide a
full parser for bodyText (this should be address by another package).
Another usage example:
envelopes, _ := imap.Fetch("1:5", "ENVELOPE")
envelopes is an array of EnvelopeData objects representing the
envelopes of the first five messages in the mailbox.
type EnvelopeData struct {
Date string // a string representing the date
Subject string // a string representing the subject
From []AddressData // an array of AddressData objects
// representing the sender(s)
...
}
Finally, let's give a look at the Search method:
// Search searches the selected mailbox for messages that match the
// given searching criteria. It returns a slice of sequence numbers
// corresponding to those messages that match the searching criteria.
func (c *Client) Search(criteria string, charset string) ([]int64,
os.Error)
Note: In addition to the Search method, the package could provide a
family of searching facilities: SearchBody, SearchSubject, etc.
### Continuation request responses
Quoting from RFC3501
Section 7.1:
<<Continuation request responses are sent by the server to indicate
acceptance of an incomplete client command and readiness for the
remainder of the command>>
Section 7.5:
<<This form of response indicates that the server is ready to accept
the continuation of a command from the client>>
For example, a continuation response is returned by the server after
receiving the AUTHENTICATE command. In this case, the server asks the
client for additional data (e.g. a client response to a server
challenge). Continuation responses are handled internally by the
client and they should not be accessible through the API.
## Connection
This covers the basic API for establishing a connection to an IMAP server.
// Return a new client connected to an IMAP server at addr.
func Dial(addr string) (*Client, os.Error)
// Return a new client connected to an IMAP server at addr over a TLS
connection.
func DialTLS(addr string) (*Client, os.Error)
// Return a new client using an existing connection.
func NewClient(conn net.Conn) os.Error
## Authentication
Authentication mechanism is modeled after the smtp package.
// PlainAuth returns an Auth that implements the PLAIN authentication
// mechanism as defined in RFC 4616.
// The returned Auth uses the given username and password to authenticate
// on TLS connections to host and act as identity.
func PlainAuth(identity, username, password, host string) Auth
// XOauth returns an Auth that implements XOAUTH authentication.
func Xoauth(...) Auth
## Types
// Auth is implemented by an IMAP authentication mechanism.
type Auth interface {
// Start begins an authentication with a server.
// It returns the name of the authentication protocol
// and optionally data to include in the initial AUTHORIZE command
// sent to the server. It can return proto == "" to indicate
// that the authentication should be skipped.
// If it returns a non-nil os.Error, the IMAP client aborts
// the authentication attempt and closes the connection.
Start(server *ServerInfo) (proto string, toServer []byte, err
os.Error)
// Next continues the authentication. The server has just sent
// the fromServer data. If more is true, the server expects a
// response, which Next should return as toServer; otherwise
// Next should return toServer == nil.
// If Next returns a non-nil os.Error, the IMAP client aborts
// the authentication attempt and closes the connection.
Next(fromServer []byte, more bool) (toServer []byte, err os.Error)
}
// ListResponse implements Response.
type ListResponse struct {
Attributes []string
Delimiter string
Name string
}
...
type Client struct {
Text *textproto.Conn
// unexported fields omitted
}
// ServerInfo records information about an IMAP server.
type ServerInfo struct {
Name string // IMAP server name
TLS bool // using TLS, with valid certificate for Name
Auth []string // advertised authentication mechanisms
}
## More examples
// Noop does nothing.
func (c *Client) Noop() os.Error
// Logout informs the server that the client is done with the connection.
func (c *Client) Logout() os.Error
// Auth authenticates a client using the provided authentication mechanism.
// A failed authentication closes the connection.
func (c *Client) Auth(a Auth) os.Error
// StartTLS starts a TLS negotiation with the server.
func (c *Client) StartTLS() os.Error
// Login indentifies the client to the server and carries the
// plaintext password authenticating this user. Login returns an error if
// it is issued over an insecure network.
func (c *Client) Login(username, password string) os.Error
// Select selects a mailbox so that messages in the mailbox can be
// accessed.
func (c *Client) Select(mailbox string) os.Error
# Hypothetical roadmap
* Define a preliminary, almost-generally-accepted API for the client
* Start implementing a subset of the commands defined in the RFC3501
* Push the first working code on github and let the people play with
it and contribute
* Eventually - as soon as the package is considered almost stable -
consider the inclusion in the standard library
# References
[1] - http://www.rfc-editor.org/rfc/rfc3501.txt
[2] - http://groups.google.com/group/golang-nuts/browse_thread/thread/3131e119a6a3dc91/eccda9a4ab4de232?lnk=gst&q=imap#eccda9a4ab4de232
[3] - http://golang.org/pkg/smtp/
--
Andrea Fazzi @ alcacoop.it
Read my blog at http://freecella.blogspot.com
Follow me on http://twitter.com/remogatto
Evan Shaw
> I've started implemening an IMAPv4rev1 (RFC3501)[1] client in Go.
Cool!
If this is all going to be repeated from smtp, I'd just as soon make
another package they can both depend on. sasl seems like an
appropriate name. These are well-known authentication mechanisms that
a number of protocols can use, even if they're not part of the core
library.
> // Noop does nothing.
> func (c *Client) Noop() os.Error
I'd probably not expose this unless there's a common use for it.
> // Login indentifies the client to the server and carries the
> // plaintext password authenticating this user. Login returns an error if
> // it is issued over an insecure network.
> func (c *Client) Login(username, password string) os.Error
Is this just a convenient way of saying client.Auth(PlainAuth("",
username, password, host))? It's not a bad idea and probably something
smtp should add if imap will have it.
> # Hypothetical roadmap
>
> * Define a preliminary, almost-generally-accepted API for the client
>
> * Start implementing a subset of the commands defined in the RFC3501
>
> * Push the first working code on github and let the people play with
> it and contribute
>
> * Eventually - as soon as the package is considered almost stable -
> consider the inclusion in the standard library
LGTM
- Evan
Russ Cox
I think it is fine for now to say
type Auth smtp.Auth
and then worry about the IMAP-specific code instead.
It's great to copy the basic client connection, encryption,
and authentication interface from SMTP. I think they're fine.
Once we get to the IMAP proper, there is a question of
how high level an interface the package should expose.
SMTP chooses to be fairly low level, but there's not much
that you can do in SMTP so it ends up being still fairly
understandable for clients of the package. The doc
comments walk prospective clients through the sequence
Mail, Rcpt, Data, and while there is implicit state (the single
message being sent at this moment) it is fairly limited.
IMAP, on the other hand, is a much more complex protocol.
As a prospective client, I would want to use a somewhat
higher level interface to the protocol than one that made me
worry about details like message UIDs and UID validity and
what the current selected mailbox is.
I think that at the least you'd want a Box type to represent
a mailbox, and the commands that operate in the protocol
on the "selected" mailbox would be methods of Box. Then
if there were multiple Boxes in use in a single Client, the
client code would take care of flipping between them as
appropriate, hiding the fact that there is this global state in
the protocol.
Another problem with the IMAP protocol is that almost all
the interesting information comes back not in the actual
protocol responses but as "oh by the way" interim notes
(the so-called unexpected responses).
My recollection is that you have to save this state somewhere,
because a future response might assume you already know
something that was mentioned during a previous one. That's
another argument for having the explicit Box types, but also
for having an explicit Msg type and MsgPart type, treating
the whole thing as a big cache of accumulated information.
Different servers also behave differently as far as what they
send when; it would be good if the API could shield the
package clients from details like that.
The Client should let you enumerate the Boxes or look
up a specific Box by name.
The Box should let you enumerate the Msgs or look up
a specific Msg by IMAP uid.
The Msg should let you enumerate the child MsgParts or
skip to a specific one.
The MsgParts should let you enumerate the child MsgParts
or skip to a specific one.
The commands that are valid in the Authenticated State
would be methods on Client. The commands that are valid
in the Selected State would typically be methods on Box
or Msg or MsgPart.
The various methods that inspect a Box or a Msg or a
MsgPart would need to page in data on demand, if that
data had not already been sent as an unexpected response
by the server.
It would be great if the whole interface let you later come
back and implement transparent server reconnection but
that's probably something to skip on first draft as long as
the API doesn't preclude it later. Typically this is pretty easy
as long as the data structures record UID and UID validity
instead of relative message numbers.
There's definitely some tension between trying to do something
that lets the package client control how much data gets sent
over the wire (for example, the method on Box that lists the
mailbox and returns a []*Msg should probably take lo and hi bounds
so that clients can control how many messages are considered).
and doing something that makes IMAP actually pleasant to use,
but we should be able to strike a nice balance.
A good next step would be to try to design an API for these
higher level concepts.
Russ
Andrea Fazzi
> > // Noop does nothing.
> > func (c *Client) Noop() os.Error
>
> I'd probably not expose this unless there's a common use for it.
Hi Evan,
sorry for the late reply.
Quoting from RFC3501 section 6.1.2:
<<Since any command can return a status update as untagged data, the
NOOP command can be used as a periodic poll for new messages or
message status updates during a period of inactivity (this is the
preferred method to do this). The NOOP command can also be used
to reset any inactivity autologout timer on the server.>>
But since Russ proposed an higher lever API, I'll propably hide this to
the user.
> > // Login indentifies the client to the server and carries the
> > // plaintext password authenticating this user. Login returns an error if
> > // it is issued over an insecure network.
> > func (c *Client) Login(username, password string) os.Error
>
> Is this just a convenient way of saying client.Auth(PlainAuth("",
> username, password, host))? It's not a bad idea and probably something
> smtp should add if imap will have it.
It's different: the IMAP client protocol provides itself a LOGIN command
which is not related with the SASL PLAIN mechanism. Unless I'm wrong, the
SMTP protocol doesn't.
Cheers,
Andrea
Andrea Fazzi
> A good next step would be to try to design an API for these
> higher level concepts.
Thanks for your feedback, Russ. I'll sketch that API and I'll post here
next days.
Andrea