Ugh. We've WONTFIX'd adding OAuth 2.0 support to IMAP in the past. The
problem with OAuth is that the authorization steps are completely
separate from the low-level SASL steps in IMAP and friends, and the SASL
portion gives you no insight into how to perform the HTTP steps of
authorization (i.e., it doesn't tell you which HTTP server to talk to).
It's worse than GSSAPI in this regard because at least GSSAPI has a
standardized platform C API that allows you to pass off the
communication steps to the third party.
Oh, and IIRC, OAuth requires you to provision a "secret" identifier for
your application, which is rather hostile for open-source applications
since checking it into our public repository is not likely to go over
well. :-(
OAuth 2.0 makes some amount of sense if your application only cares
about talking to Google's server. For a generic IMAP client, it is
really hostile to your userbase. I find Google's attempt to make
traditional authentication steps (even relatively secure SASL steps?)
more difficult for users to use to be rather hostile to users and clients.
[Granted, they also seem to like the idea of locking you out of your
account if you try accessing it from a different country, as I found
much to my chagrin when I tried checking email in Toronto... or was that
Facebook? or both?]
--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist
Would it make sense to comment on the blog post, raising these issues
(in particular, the problems for open source clients)?
Gerv
But Gmail is so important that I don't think we have much choice other
than to support access through whatever access mechanism they throw at
us. It would be good to keep it separate though from public-spec IMAP so
that we are not encouraging proliferation of stuff that we don't believe in.
On OAuth itself I don't know enough to have an opinion whether it is
good or bad.
:rkent
From https://developers.google.com/accounts/docs/OAuth2#installed:
"The process results in a client ID and, in some cases, a client secret,
which you embed in the source code of your application. (In this
context, the client secret is obviously not treated as a secret.) "
The Firefox OS Gaia calendar app uses oauth2 for accessing google
calendar. You can find its code at
https://github.com/mozilla-b2g/gaia/tree/master/apps/calendar including
the secret it uses to talk to Google. Grep on 'oauth' for the good stuff.
> OAuth 2.0 makes some amount of sense if your application only cares
> about talking to Google's server. For a generic IMAP client, it is
> really hostile to your userbase. I find Google's attempt to make
> traditional authentication steps (even relatively secure SASL steps?)
> more difficult for users to use to be rather hostile to users and
> clients.
I think this is unfair. A user's Google account is potentially a
gateway to a mountain of personal and private information and a
username/password is a fairly blunt instrument. While it's very
possible to make a case that Google's application-specific passwords
should support scoping so you can say "this password is just for gmail",
that's effectively what their oauth2 implementation accomplishes, and
with arguably better UX since it allows Google to be the entity
describing to the user what abilities they are granting to Thunderbird
if the user says yes.
(Not that Thunderbird wouldn't try and describe what it needs/wants very
accurately, but in an adversarial system, you can't trust things that
aren't Thunderbird to be honest about what they are going to get up to.
And they are indistinguishable from Thunderbird for our purposes here.)
Andrew
On 04/25/2014 12:35 PM, Joshua Cranmer 🐧 wrote:From https://developers.google.com/accounts/docs/OAuth2#installed:
Oh, and IIRC, OAuth requires you to provision a "secret" identifier for your application, which is rather hostile for open-source applications since checking it into our public repository is not likely to go over well. :-(
"The process results in a client ID and, in some cases, a client secret, which you embed in the source code of your application. (In this context, the client secret is obviously not treated as a secret.) "
The Firefox OS Gaia calendar app uses oauth2 for accessing google calendar. You can find its code at https://github.com/mozilla-b2g/gaia/tree/master/apps/calendar including the secret it uses to talk to Google. Grep on 'oauth' for the good stuff.
On 25/04/2014 17:52, Gervase Markham wrote:http://googleonlinesecurity.blogspot.co.uk/2014/04/new-security-measures-will-affect-older.html Is this relevant to Thunderbird accessing Gmail?It looks like it - plz file a bug ..... Ludo
I know that Firefox was requested by Google to hide its Google API keys.
>> OAuth 2.0 makes some amount of sense if your application only cares
>> about talking to Google's server. For a generic IMAP client, it is
>> really hostile to your userbase. I find Google's attempt to make
>> traditional authentication steps (even relatively secure SASL steps?)
>> more difficult for users to use to be rather hostile to users and
>> clients.
>
> I think this is unfair. A user's Google account is potentially a
> gateway to a mountain of personal and private information and a
> username/password is a fairly blunt instrument. While it's very
> possible to make a case that Google's application-specific passwords
> should support scoping so you can say "this password is just for
> gmail", that's effectively what their oauth2 implementation
> accomplishes, and with arguably better UX since it allows Google to be
> the entity describing to the user what abilities they are granting to
> Thunderbird if the user says yes.
(Reads documentation a little more closely)
Okay, at the very least, Google's documentation is completely horrid: I
didn't realize that it was possible to recover the bearer token without
having to scrape UI from a webpage.
I don't disagree that external authorization mechanisms are necessarily
a bad thing. However, I think that OAuth fails to be an effective mechanism:
1. The mechanism is trivial to internalize: you need to be able to
control an HTML form and http[s] calls manually, which isn't a terribly
hard task for many applications [e.g., Thunderbird's current OAuth 2
accesses do this]. Once you internalize the authorization, you still get
the username and password and effectively complete access.
2. Critical authorization tokens are passed in plaintext on URIs. So if
you can sniff this URI somehow, even without full control over the
authorization, you can still do large worlds of hurt. And URIs are
generally handled far more carelessly than, say, POST data.
3. There's no standardized way to access the authorization mechanism. In
GSSAPI, to request access an IMAP access token, you load a platform
library [typically], and you call an RFC-standardized function with
RFC-standardized parameters that indicate that you're looking for an
IMAP access token. In contrast, the SASL OAUTH step just has the client
access with a Bearer token it got from... who knows? The server, pages,
request parameters, etc., are never hinted at in the authentication
step, so the only way you can know these values are to manually hard
code them for every site.
It's #3 that makes me so resistant to this feature.
--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist
_______________________________________________
You receive a scoped-to-email access credential that is different from
the user's normal site-wide Google login credentials. Compromise of the
token is a pretty big deal given the importance of email, but it's less
bad than compromise of the entire account. But if it is compromised:
1) Google has a better chance of detecting the compromise and only
cutting off that specific token, rather than trying to lock-out the
entire account/etc. (Noting that they can already do this with
application-specific passwords. Those are just more dangerous. And
require even more manual intervention on the part of the user.)
2) Compromise of this token does not require the user to change their
password, it just requires them to revoke the access credential and have
Thunderbird request a new token.
> 2. Critical authorization tokens are passed in plaintext on URIs. So
> if you can sniff this URI somehow, even without full control over the
> authorization, you can still do large worlds of hurt. And URIs are
> generally handled far more carelessly than, say, POST data.
The URIs travel over https. If you can sniff the URI over the network
then you can sniff everything else.
If you're talking about Thunderbird being careless with URIs, then that
sounds like a Thunderbird problem, not an oauth problem.
> 3. There's no standardized way to access the authorization mechanism.
> In GSSAPI, to request access an IMAP access token, you load a platform
> library [typically], and you call an RFC-standardized function with
> RFC-standardized parameters that indicate that you're looking for an
> IMAP access token. In contrast, the SASL OAUTH step just has the
> client access with a Bearer token it got from... who knows? The
> server, pages, request parameters, etc., are never hinted at in the
> authentication step, so the only way you can know these values are to
> manually hard code them for every site.
As documented at
https://developers.google.com/accounts/docs/OAuth2Login#discovery,
Google exposes their openid data at
https://accounts.google.com/.well-known/openid-configuration
openid-configuration is documented in the IANA Well-Known URIs registry:
https://www.iana.org/assignments/well-known-uris/well-known-uris.xhtml
Now, there is the big question of how to generically end up at the
accounts.google.com domain; that webpage says you should hardcode it. I
agree that that hard-coding is not great. It would be better if that
file was also accessible at
https://gmail.com/.well-known/openid-configuration (404). There is a
https://gmail.com/.well-known/host-meta from the webfinger experiments
from a few years back, so it seems possible to request that such a file
be added.
There is also the question of how we know what scope to use.
https://developers.google.com/gmail/xoauth2_protocol tells us to use the
scope "https://mail.google.com/". This is not an inherently obvious
transformation for an automated mechanism. Ideally this could also be
put in something under https://gmail.com/.well-known/.
So I would say:
1) It makes sense to work with Google to avoid anyone having to hardcode
Google-specific paths in their app. They provide SRV records for
_imaps._tcp.gmail.com, so they clearly are already trying to do this.
2) Gmail is already a special-case inside Thunderbird in many places,
oauth2 is not a Google-specific invention, and Thunderbird already has
an ISP database for this exact purpose (recently extended to provide
login URLs! https://bugzil.la/973005).
Andrew
What I mean is that the page which Google uses to ask you for your
username and password is effectively under the control of the
application, so it can still be possible to intercept the password.
> If you're talking about Thunderbird being careless with URIs, then
> that sounds like a Thunderbird problem, not an oauth problem.
Thunderbird itself isn't impacted by this problem, but it is a potential
security hole in websites if they aren't careful.
> 1) It makes sense to work with Google to avoid anyone having to
> hardcode Google-specific paths in their app. They provide SRV records
> for _imaps._tcp.gmail.com, so they clearly are already trying to do this.
I'd rather see Google's documentation indicate how to discover the OAuth
information to use instead of giving you the hard-coded strings, and I
would like to see a solution rolled out before adding support for SASL
OAuth2 to Thunderbird. I've been made cynical so that I don't believe
it's going to get done until I see it get done.
--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist
_______________________________________________
heh. That's a very nice point.
All that steems from the fact that OAuth was designed for websites: One
website wants access to webservices from another website. All the
choices - client secrets (kept by the server), entering the password on
the service site (protected by cross-origin barriers of the browser),
making it a webpage in the first place - make sense in that context.
Many of them make absolutely no sense in the context of desktop client
applications (e.g. Thunderbird) or even non-interactive processes (e.g.
sync).
Gervase Markham wrote, On 28.04.2014 16:58:
But the sort of questions I would want to find answers to are:1. They block login attempts from a new country. Presumably that's anti-account-theft.
* What does Google hope to gain by making this change? Is it an
anti-spam/anti-fraud measure?
2. When that triggers, they demand a working phone number, where they send an activation code. Strangely, that can be any phone number. They pretend that's for "security", but the "nice" side effect for them is that using a phone number, they can link the account to a real life identity. Given that they also link the account to all searches I make on Google, that's a privacy invasion for me. But for Google, that means $$.
3. Long-term, their goal is to move everything (Internet and offline) to the web, and to their servers. They want to kill MS Office, email, phone etc., moving it to gmail, google cloud etc. They are not doing all this for fun, after all.
* Can the additional data about logins that Google hopes to obtain be
obtained in other ways for IMAP?
You can't ask for a phone number via IMAP. But I reject that premise and interest.
If a suspicious login attempt shows up via IMAP or SMTP, they can return an error (in IMAP/SMTP) *with* an error message that mentions reason and remedy, e.g.
"You are logging in from a new country. Please log in via https://www.gmail.com first and approve this connection."
This is (more or less) how some German freemail ISPs do it.
This is a manual hand-over, but a) would happen only in really problematic cases b) give them the same possibilities as now.
* What does Google hope to gain by making this change? Is it an anti-spam/anti-fraud measure?
* Can the additional data about logins that Google hopes to obtain be obtained in other ways for IMAP?
* Why can't single-service passwords continue to be used instead?
* Do they understand the ramifications of the idea that clients of all these protocols will need to contain a browser? (Is that actually so?)
* Are there any ramifications on open source software in particular? (Seems from the discussion that the answer is actually no, but we need to be sure.)
* Who are the other desktop email software providers, and what are their opinions?
Use of single-factor authentication (just a password) that is used by both the user and all their applications is
I think Google's goal is to be able to control the entire authentication
process rather than delegating it to the normal protocol authentication
(i.e., SASL) procedures. You might be able to argue that it's merely a
means to other ends, but I get the impression that Google is too fixated
on the OAuth 2 mechanism in particular to fail to ponder how best their
goals, whatever they may be, may be best served in the normal SASL
mechanisms. The more I think about the security implications of OAuth
2-via-SASL, the more I see it as ultimately a weakening of security
guarantees as far as IMAP is concerned.
Looking at OAuth 2 itself, it seems to me like a really, really insecure
variant of Kerberos. I'm guessing the main reason they don't want to
attempt to reuse prior art here is
a) They want to authorize the client in particular instead of the
mechanism of access (which to me is bogus).
b) Using Kerberos may be "too hard" for people (despite having
well-established, cross-platform support for at least a decade).
c) They want to eliminate the ability for the client to
control/intercept authentication (which OAuth 2.0 only appears to do).
> * Can the additional data about logins that Google hopes to obtain be
> obtained in other ways for IMAP?
It probably shouldn't be hard to get most of the same functionality via
Kerberos:
1. KDC servers can already be looked up via DNS SRV records.
2. SASL/GSSAPI already gives a well-defined mapping to look up how to
request a service ticket (e.g., imap/mail.go...@GMAIL.COM).
3. If they want to authenticate client id, they could arguably do this
by having users login not with Pidg...@GMAIL.COM but with
Pidgeot18/Clie...@GMAIL.COM.
4. Acquiring kerberos tickets is typically handled outside of the client
anyways (if you select GSSAPI as the auth option, we don't bother asking
for a password, although there is a bug on us popping up dialogs if the
ticket expired).
> * Do they understand the ramifications of the idea that clients of all
> these protocols will need to contain a browser? (Is that actually so?)
The main ramification to me is not that clients need to contain a
browser but that they need to encode gmail-specific logic with little
pretense of genericity. Note that the "contain a browser" portion of it
automatically implies in great detail that clients have complete control
over the authentication/authorization process anyways, one of the
reasons I think the security gains are rather illusory.
--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist
_______________________________________________
If tokens can expire and be refreshed by Lightning without (!) user interaction, I don't know how that would work. Can you expand, please?
Google's OAuth documentation is extremely heavily geared to web
applications. It's confusing even when you know what you're looking for. :-(
>
> If tokens can expire and be refreshed by Lightning without (!) user
> interaction, I don't know how that would work. Can you expand, please?
The authorization procedure looks like this:
1. Client makes authorization request to Google.
2. Google does authentication using its own UI and pages.
3. Google sends client an authorization code.
4. Client uses authorization code to gain an access token and refresh token.
5. Client sends access token to actually get access [this is the only
part that actually goes over IMAP].
6. Access tokens expire after a "short" amount of time (hours?)
7. A client can obtain a new access token with its refresh token. This
is meant to be saved in long-term storage.
There's this gem of a note:
"Note that there are limits on the number of refresh tokens that will be
issued... You should save refresh tokens in long-term storage and
continue to use them as long as they remain valid. If your application
requests too many refresh tokens, it may run into these limits, in which
case older refresh tokens will stop working."
[which, incidentally, turns out to contradict earlier statements on that
page about refresh tokens. Google's OAuth documentation is less than
stellar.]
It's implied that refresh tokens both don't expire and will expire over
a "long" period of time, but the documentation is really vague on this
matter.
--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist
_______________________________________________
As Patrick said, the refresh token probably expires after weeks.
Even if it doesn't, Google may kill it for other reasons, e.g. you
changing IP addresses.
This, an app needs to be prepared to refresh (re-gain) the refresh token
and start from scratch at any time.
And yes, Google is *intentionally* vague. That's policy: They want to be
in full control.
That means we'd need to handle 4 different tokens: Auth token, refresh
token, access token. Plus the password that the user enters towards Google.
Any of them can suddenly go invalid at any time for any reason. If
you've ever implemented something like that, it's gruesome. Esp. given
that you need to be able to go back to UI at any time, and the
documentation and error codes are intentionally vague, it's hard to make
a proper design. Often, you can't even distinguish between expiry and
auth failure, due to collation of error codes, so it's hard to know what
to do, and you risk doing loops.
(And given that we display the browser and have control over it, we can
get the password, making the whole thing moot, as you pointed out. For
that exact reason, for me, token-based auth mechs are all security theater.)
Ben
On 29.04.2014 18:17, Ben Bucksch wrote:
nea...@gmx.de wrote, On 28.04.2014 20:10:
Lightning hides that process from the user.
That access key will be used to generate a token to work with your mail/calendar. That token will expire after a certain time and your application (mail/calendar) needs to generate a new token. Normally the user should not notice about that for any session.
Expiry indeed is a core problem. One-time setup - if it's really once and then never again - can be handled differently than something that can up at any random time.
How does that process work? All readable documentation I found about OAuth speaks about webpages. That is: not HTTP URLs, but random HTML served by Google, containing arbitrary challenges to the end user (e.g. enter phone number, Google sending a code there, enter that code) that only the allowed human can fulfill.
If tokens can expire and be refreshed by Lightning without (!) user interaction, I don't know how that would work. Can you expand, please?
[...]
Lightning does that first process to get the very first access and refresh code a bit more elegant. I think Philipp (Fallen) could give a more detailed description here.
This was brought up in the status meeting, and we resolved to reach out
to Gmail to clarify some questions. Here's the status of as right now:
1. The clarification from GMail IMAP folks is:
> The bottom line is that GMail would really like Thunderbird to use
> OAuth2 for imap/smtp/pop access. If it doesn't, there's an increased
> possibility that GMail will suspect the login attempt is unauthorized.
> If you keep using the same IP address, or have two factor auth turned
> on, you'll most likely be OK. Otherwise, the users run the risk of
> having to jump through some hoops to get imap access again (I don't
> know the exact details of that...)
2. I've made a post to the IMAP-protocol list about this topic (it felt
more relevant there than the Kitten working group):
<http://mailman13.u.washington.edu/pipermail/imap-protocol/2014-April/002243.html>.
From responses in the past 12 hours, it does seem like there is
agreement by client implementers that some of these issues need to be
resolved.
3. I've been told by both Bienvenu and Brandon that the OAuth people
have been brought into the discussion, although they haven't responded
publicly yet.
It looks to me that it will be possible to see many of the concerns I
have about OAuth discussed and addressed.
As a side note, it also looks like other IMAP servers are planning on
supporting OAuth 2.0. Outlook.com recently rolled out support for it as
well, and I think there was another server the name of which I don't
recall right now.
--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist
_______________________________________________
1. The OAuth working group is currently producing a document on dynamic
client registration. This satisfies one of my two principle concerns.
Reading up on ancillary information, Thunderbird was explicitly
mentioned as a use case. I've read the current draft, and my thoughts
are best summarized as:
* We may need to provision a public/private key pair for Thunderbird. I
don't know how secret this would need to be, but since a key pair is all
that's needed, it's a relatively simple matter to generate one locally
for developers. Perhaps even automatically ^_^
** [Note: keys here are JWT keys, if that means anything to anybody else.]
* There may be some slight issues with localization. There's provisions
for displaying localized strings, but it's not quite clear to me what
the recommended best practice is. Worst case scenario may require us to
carry several localizations at once.
* We'll get a client id and a client secret. May be useful for other
OAuth 2.0 uses in our tree.
* Yet another endpoint
I don't see any major issues, but I'll read it more thoroughly when a
last call is put out.
2. There was some discussion of the issues at the Internet Identity
Workshop, which apparently happened basically the week after this thread
started. This gives a little bit of background context on some of the
design choices, and Thunderbird is explicitly mentioned as the IMAP/SASL
dynamic client registration use case. \o/
3. Endpoint discovery is mentioned several times as a concern... and
mentioned several times as a deferral. I'm concerned that the kitten
[SASL] and OAuth working groups aren't necessarily talking to each
other, because there's an off-hand comment by someone that perhaps SASL
could elucidate the endpoint information, yet the OAuth SASL spec is in
last call and explicitly disclaims responsibility for endpoint notification.
I don't follow the working groups myself, but I've binged on their
archives for relevant details.
So, what should we be doing?
1. This has been discussed in another circumstance, but moving the OAuth
2.0 to shared code or toolkit is probably a good first step.
2. We ought to tweak our SASL code to not assume that SASL is
synchronous. This might also be an excellent time to attempt to
consolidate the SASL code into a better framework than DoGSSAPIStep1,
DoGSSAPIStep2, etc.
3. OAuth 2.0 as a SASL backend is almost certainly going to be in JS.
IMAP protocols are off-main-thread, and hence are forbidden from direct
JS access. We need to make sure that we do a main-thread-proxy here for
IMAP.
I'm guessing that usable endpoint discovery won't roll out for 6-9
months, which means we may need to temporarily hardcode the endpoint
locations for, say, gmail. :-( OTOH, it also depends on how quickly
Google or other providers roll out implementations of the current
standardized variants.
-- Dave Koelmeyer http://blog.davekoelmeyer.co.nz
Just looking at the Gmail account warning in this help doc:
My understanding was that by enabling the less secure apps setting for the Gmail account in question Thunderbird would continue to function with Gmail in the same way it always has. I created a free Gmail account myself a week or two ago, and by making this setting I can connect using Thunderbird fine. I also noted information from a Google developer on the matter at http://mailman13.u.washington.edu/pipermail/imap-protocol/2014-September/002326.html.
Is stating that it won't work at all until April 2015 not really accurate?
The article can probably be updated to reflect users will need to toggle this on. Google did not have a lot of information earlier on I think when this article was created.
I think many will assume it just doesn't work unless they happen to stumble across the setting.