Google API Key

[Doug Turner]

This is a heads up:

Google has been moving many services to require an API key. Mozilla's
contract with Google for geolocation support requires that we use an API
key similar to that of Chromium:
http://www.chromium.org/developers/how-tos/api-keys

I suspect other services like Safe-Browsing will eventually require an
API key as well.

The reasons behind this move are not clear/public. I suspect that
service providers, like Google, want to prevent unauthorized access. In
order to do this, they need some way to identify clients that are
authorized. The way they have chosen to do this is to embed a secret
into the client and have that client present that secret during the API
usage. I know -- this isn't a great technical solution. However, it is
what we're stuck with. There isn't much wiggle room here. Google, and
other service providers, use API Keys. (Please don't make it difficult
by pointing out how easy it is to find the secret in Chrome.)

People that build their own versions of Firefox can either provide their
own keys and these services will work... or they can do nothing, and
these services will not work. Or course, all of our official builds
(nightly, aurora, beta, final) will continue working and there will be
no impact to users.

More details will follow.

Doug Turner

[Mike Hommey]

On Thu, Jun 13, 2013 at 09:13:29PM -0700, Doug Turner wrote:
> People that build their own versions of Firefox can either provide
> their own keys and these services will work... or they can do
> nothing, and these services will not work. Or course, all of our
> official builds (nightly, aurora, beta, final) will continue working
> and there will be no impact to users.

Does this mean the key won't be in the mozilla-* branches, but
integrated on the releng side?
Does it mean each and every linux distro will need its own key?

Mike

[Jonathan Kew]

On 14/6/13 08:07, Mike Hommey wrote:
> On Thu, Jun 13, 2013 at 09:13:29PM -0700, Doug Turner wrote:

>> People that build their own versions of Firefox can either provide
>> their own keys and these services will work... or they can do
>> nothing, and these services will not work. Or course, all of our
>> official builds (nightly, aurora, beta, final) will continue working
>> and there will be no impact to users.
>

> Does this mean the key won't be in the mozilla-* branches, but
> integrated on the releng side?
> Does it mean each and every linux distro will need its own key?

And does it mean each and every developer will need keys for any and all
of these services, if they're to test these features in their own builds
as they're working on them?

[Gavin Sharp]

On Fri, Jun 14, 2013 at 4:32 AM, Jonathan Kew <jfkt...@googlemail.com> wrote:
>>> People that build their own versions of Firefox can either provide
>>> their own keys and these services will work... or they can do
>>> nothing, and these services will not work.

> And does it mean each and every developer will need keys for any and all of
> these services, if they're to test these features in their own builds as
> they're working on them?

Yes. How this works for the Chromium project is described at
http://www.chromium.org/developers/how-tos/api-keys .

This is obviously a burden, and not ideal, but given the relatively
small set of people working on these features specifically, the
ability to change the API URLs for local testing, and the ability to
request your own keys to the real APIs, it seems manageable.

Gavin

[Gavin Sharp]

On Fri, Jun 14, 2013 at 3:07 AM, Mike Hommey <m...@glandium.org> wrote:
> Does this mean the key won't be in the mozilla-* branches, but
> integrated on the releng side?

Yes.

> Does it mean each and every linux distro will need its own key?

I imagine so.

Gavin

[Mike Hommey]

On Fri, Jun 14, 2013 at 04:07:00PM +0900, Mike Hommey wrote:
> On Thu, Jun 13, 2013 at 09:13:29PM -0700, Doug Turner wrote:

> > People that build their own versions of Firefox can either provide
> > their own keys and these services will work... or they can do
> > nothing, and these services will not work. Or course, all of our
> > official builds (nightly, aurora, beta, final) will continue working
> > and there will be no impact to users.
>

> Does this mean the key won't be in the mozilla-* branches, but
> integrated on the releng side?

> Does it mean each and every linux distro will need its own key?

BTW, are these keys required to be "hidden"? (as in, only present in the
binaries, not in the source) If they are, then it's not possible to use
them for (at least some) linux distros.

Mike

[Doug Turner]

Yes, this will be integrated on the releng side. Details TBD. I have
reached out to John Oduinn last night.

Yes, every linux distro will need its own key. This isn't a change.
The existing Geolocation contract required non Mozilla produced builds
to have their own API, iirc. Also, I should be clear about the future
-- application provided geolocation systems are dying. The OS provides
this in many cases. On the mac, we are going to use CoreLocation. On
Windows 7, we are looking at their system services. Linux has something
called GeoClue that we'd like to look it. (if you know about geoclue,
contact me plz)

Thanks!
Doug

> Mike Hommey <mailto:m...@glandium.org>
> June 14, 2013 12:07 AM

>
> Does this mean the key won't be in the mozilla-* branches, but
> integrated on the releng side?
> Does it mean each and every linux distro will need its own key?
>

> Mike
> Doug Turner <mailto:doug....@gmail.com>
> June 13, 2013 9:13 PM

[Ben Hearsum]

On 06/14/13 12:25 PM, Doug Turner wrote:
> Yes, this will be integrated on the releng side. Details TBD. I have
> reached out to John Oduinn last night.

It's probably better to file a bug in mozilla.org:RelEng for this.
John's generally pretty backlogged, and this seems like it will need me
or someone else who delves in the details to figure out, anyways.

[Gervase Markham]

On 14/06/13 12:44, Gavin Sharp wrote:
> This is obviously a burden, and not ideal, but given the relatively
> small set of people working on these features specifically, the
> ability to change the API URLs for local testing, and the ability to
> request your own keys to the real APIs, it seems manageable.

Perhaps we should consider requiring this to be true for APIs we support
in Firefox? That is to say, we should require that it be possible for
any random community member to get a (low volume) API key for testing
purposes.

Gerv

[Matt Brubeck]

On 6/14/2013 4:57 AM, Mike Hommey wrote:
> BTW, are these keys required to be "hidden"? (as in, only present in the
> binaries, not in the source) If they are, then it's not possible to use
> them for (at least some) linux distros.

Some programs in similar situations (for example, programs like Calibre
that use Amazon Web Services) solve this by allowing users to specify
their own keys at runtime. This isn't ideal, but it might be a feasible
way for Linux distros to enable these features without building secrets
into their packages.

Chromium allows users to provide keys at runtime through environment
variables. If Firefox included a similar capability, then Linux distros
could create add-ons that expose it in the UI. (This would also require
Google to provide a way for individuals to request such keys, which they
may or may not be willing to do.)

[Gavin Sharp]

On Fri, Jun 14, 2013 at 12:40 PM, Gervase Markham <ge...@mozilla.org> wrote:
> Perhaps we should consider requiring this to be true for APIs we support
> in Firefox? That is to say, we should require that it be possible for
> any random community member to get a (low volume) API key for testing
> purposes.

Indeed, it would be great if we could have a process like
http://www.chromium.org/developers/how-tos/api-keys (just with a
different "step 1"). That might be somewhat complicated to set up, but
we should try.

Doug, do you have the right contacts to ask?

Gavin

[Hubert Figuière]

On 14/06/13 12:25 PM, Doug Turner wrote:
> On Windows 7, we are looking at their system services. Linux has
> something called GeoClue that we'd like to look it. (if you know about
> geoclue, contact me plz)

Doug,

For GeoClue there is already a bugzilla open

https://bugzilla.mozilla.org/show_bug.cgi?id=485472

With a patch.

Not sure how much it has rot. The OP for that bug is still involved on
the GeoClue part on the Gnome side.

Hub

[Doug Turner]

Yes, that is a great idea. Details will follow!

What I think is going to happen is something like:

You'll add this to your mozconfig:
ac_add_options --with-google-api-keyfile=/builds/google-key.txt



> Gavin Sharp <mailto:ga...@gavinsharp.com>
> June 14, 2013 9:55 AM

>
> Indeed, it would be great if we could have a process like
> http://www.chromium.org/developers/how-tos/api-keys (just with a
> different "step 1"). That might be somewhat complicated to set up, but
> we should try.
>
> Doug, do you have the right contacts to ask?
>
> Gavin

> Mike Hommey <mailto:m...@glandium.org>
> June 14, 2013 12:07 AM
>
> Does this mean the key won't be in the mozilla-* branches, but
> integrated on the releng side?
> Does it mean each and every linux distro will need its own key?
>
> Mike
> Doug Turner <mailto:doug....@gmail.com>
> June 13, 2013 9:13 PM

> This is a heads up:
>
> Google has been moving many services to require an API key. Mozilla's
> contract with Google for geolocation support requires that we use an
> API key similar to that of Chromium:
> http://www.chromium.org/developers/how-tos/api-keys
>
> I suspect other services like Safe-Browsing will eventually require an
> API key as well.
>
> The reasons behind this move are not clear/public. I suspect that
> service providers, like Google, want to prevent unauthorized access.
> In order to do this, they need some way to identify clients that are
> authorized. The way they have chosen to do this is to embed a secret
> into the client and have that client present that secret during the
> API usage. I know -- this isn't a great technical solution. However,
> it is what we're stuck with. There isn't much wiggle room here.
> Google, and other service providers, use API Keys. (Please don't make
> it difficult by pointing out how easy it is to find the secret in
> Chrome.)
>

> People that build their own versions of Firefox can either provide
> their own keys and these services will work... or they can do nothing,

[Robert Kaiser]

Doug Turner schrieb:

> People that build their own versions of Firefox can either provide their
> own keys and these services will work... or they can do nothing, and
> these services will not work.

This is IMHO a big argument to, one by one, try to get rid of using
those services and build on free-to-use alternatives instead.

Robert Kaiser

[Doug Turner]

Awesome. I'll take a look!

[Doug Turner]

[Doug Turner]

I believe that in chromium, this is actually the case now. I'll see if
we can do exactly that for our google-api key -- allow developers to get
a low volume api key.

[Doug Turner]

Jonathan Kew wrote:
> And does it mean each and every developer will need keys for any and all
> of these services, if they're to test these features in their own builds
> as they're working on them?

Yes. Just like chromium.

I should also point out that there are like 3 people that need to worry
about this at this point. Most of them work for me and are all squared
away. :)

[Philip Chee]

On 15/06/2013 00:25, Doug Turner wrote:
> Yes, this will be integrated on the releng side. Details TBD. I have
> reached out to John Oduinn last night.
>
> Yes, every linux distro will need its own key. This isn't a change.
> The existing Geolocation contract required non Mozilla produced builds
> to have their own API, iirc. Also, I should be clear about the future

Is SeaMonkey considered a Mozilla produced build? Because some time back
someone in mozilla-central unconditionally turned on Geolocation for
anything building off mozilla-central/aurora/beta/release (by replacing
a pref with a hardcoded string (I assume "browser=firefox&sensor=true"
is the API key because I can't find anything else that resembles a API
key). When we asked if this was intended we never got a definitive reply.

Phil

--
Philip Chee <phi...@aleytys.pc.my>, <phili...@gmail.com>
http://flashblock.mozdev.org/ http://xsidebar.mozdev.org
Guard us from the she-wolf and the wolf, and guard us from the thief,
oh Night, and so be good for us to pass.

[Philip Chee]

Plus:

SeaMonkey: Uses Geolocation and Safe Browsing APIs from Google.

Thunderbird: Has code to use Safe Browsing API from Google but this code
is currently disabled.

[Doug Turner]

Philip Chee wrote:
> On 15/06/2013 06:20, Doug Turner wrote:
>>
>> Jonathan Kew wrote:
>>> And does it mean each and every developer will need keys for any and all
>>> of these services, if they're to test these features in their own builds
>>> as they're working on them?
>> Yes. Just like chromium.
>>
>> I should also point out that there are like 3 people that need to worry
>> about this at this point. Most of them work for me and are all squared
>> away. :)
>
> Plus:
>
> SeaMonkey: Uses Geolocation and Safe Browsing APIs from Google.

For Geo - this is unauthorized access as near as I can tell. I think
that we use to use a safe browser API that was 'public'. In any case,
you should inform the owners of seamonkey to double check their usage.
Is that still Kairo?

> Thunderbird: Has code to use Safe Browsing API from Google but this code
> is currently disabled.

That's good ;)

[Doug Turner]

Philip Chee wrote:
> Currently the code uses a hard coded key. There is no way of specifying
> your own.


It would be an addon, i suppose. File a bug?

[Doug Turner]

Philip Chee wrote:
> On 15/06/2013 00:25, Doug Turner wrote:
>> Yes, this will be integrated on the releng side. Details TBD. I have
>> reached out to John Oduinn last night.
>>
>> Yes, every linux distro will need its own key. This isn't a change.
>> The existing Geolocation contract required non Mozilla produced builds
>> to have their own API, iirc. Also, I should be clear about the future
>
> Is SeaMonkey considered a Mozilla produced build?

No. SeaMonkey is a community project.

[Philip Chee]

On 16/06/2013 03:52, Doug Turner wrote:
>
>
> Philip Chee wrote:
>> On 15/06/2013 06:20, Doug Turner wrote:
>>>
>>> Jonathan Kew wrote:
>>>> And does it mean each and every developer will need keys for any and all
>>>> of these services, if they're to test these features in their own builds
>>>> as they're working on them?
>>> Yes. Just like chromium.
>>>
>>> I should also point out that there are like 3 people that need to worry
>>> about this at this point. Most of them work for me and are all squared
>>> away. :)
>>
>> Plus:
>>
>> SeaMonkey: Uses Geolocation and Safe Browsing APIs from Google.
>
> For Geo - this is unauthorized access as near as I can tell. I think
> that we use to use a safe browser API that was 'public'. In any case,
> you should inform the owners of seamonkey to double check their usage.
> Is that still Kairo?

To clarify, SeaMonkey uses our own API key for Safe Browsing. We don't
use our own API key for Geolocation because there is no way of
specifying a different key (short of forking a whole bunch of code).

>> Thunderbird: Has code to use Safe Browsing API from Google but this code
>> is currently disabled.
>
> That's good ;)
>
>

--
-==-

[Doug Turner]

Philip Chee wrote:
> To clarify, SeaMonkey uses our own API key for Safe Browsing. We don't
> use our own API key for Geolocation because there is no way of
> specifying a different key (short of forking a whole bunch of code).

Good news here though: We are moving to use native system geolocation
systems.

Also, the new API allows you to just specify a key when you build. :)

[Joe Drew]

On 2013-06-15 3:55 PM, Doug Turner wrote:
>> Is SeaMonkey considered a Mozilla produced build?
>
> No. SeaMonkey is a community project.

Not being flippant here - what's the difference between something
Mozilla produces and something the Mozilla community produces?

[Boris Zbarsky]

On 6/17/13 10:54 AM, Joe Drew wrote:
> Not being flippant here - what's the difference between something
> Mozilla produces and something the Mozilla community produces?

At a guess, the difference is whether something is produced by the
entity that has the contract with Google to use the API key (that
presumably being the Mozilla Corporation) or not....

But I don't know anything about the actual restrictions on our usage of
these API keys; the above is a total guess.

-Boris

[Doug Turner]

For a better answer, I would ask on .governance and cc' the big guns.

Doug

[Gavin Sharp]

Seems kind of like a leading question, because there are many
differences between Firefox and SeaMonkey, and it's not clear which
you care about :)

"Mozilla produced build" in this context is just an imprecise way of
saying "the products for which the Mozilla Corporation has reached an
agreement with Google for API keys, on behalf of the Mozilla project",
i.e. Firefox.

Gavin

On Mon, Jun 17, 2013 at 10:54 AM, Joe Drew <j...@mozilla.com> wrote:
>
> On 2013-06-15 3:55 PM, Doug Turner wrote:
>>>

>>> Is SeaMonkey considered a Mozilla produced build?
>>
>>
>> No. SeaMonkey is a community project.
>
>

> Not being flippant here - what's the difference between something Mozilla
> produces and something the Mozilla community produces?
>

> _______________________________________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-planning

[Mike Hommey]

On Sat, Jun 15, 2013 at 12:52:34PM -0700, Doug Turner wrote:
>
>
> Philip Chee wrote:
> >On 15/06/2013 06:20, Doug Turner wrote:
> >>
> >>Jonathan Kew wrote:
> >>>And does it mean each and every developer will need keys for any and all
> >>>of these services, if they're to test these features in their own builds
> >>>as they're working on them?
> >>Yes. Just like chromium.
> >>
> >>I should also point out that there are like 3 people that need to worry
> >>about this at this point. Most of them work for me and are all squared
> >>away. :)
> >
> >Plus:
> >
> >SeaMonkey: Uses Geolocation and Safe Browsing APIs from Google.
>
> For Geo - this is unauthorized access as near as I can tell. I think
> that we use to use a safe browser API that was 'public'.

Until recently, using the public safe browser shavars required code
changes. Now it's only a pref away, but the default is still to use the
shavar that requires Google's approval, not the public one. Bug 557752
should be put forward, and, in fact, should not be about unbranded vs
branded, but Mozilla vs. not.

I think most third party API usage is in the same vein, requiring code
changes to not use them when you're not authorized to, which, as you
imply, you're not unless you're Mozilla. It's fine-ish that some
features might not be available when you're not authorized, but it's not
fine that the code a) does unauthorized things by default and b) doesn't
allow to do the right thing without modifying the code.

Mike

[Mike Hommey]

The tradeoff would be that individual users would be identified
uniquely. That sucks.

Mike

[Philip Chee]

On 16/06/2013 09:04, Mike Hommey wrote:
> On Sat, Jun 15, 2013 at 12:52:34PM -0700, Doug Turner wrote:
>>
>>
>> Philip Chee wrote:
>> >On 15/06/2013 06:20, Doug Turner wrote:
>> >>
>> >>Jonathan Kew wrote:
>> >>>And does it mean each and every developer will need keys for any and all
>> >>>of these services, if they're to test these features in their own builds
>> >>>as they're working on them?
>> >>Yes. Just like chromium.
>> >>
>> >>I should also point out that there are like 3 people that need to worry
>> >>about this at this point. Most of them work for me and are all squared
>> >>away. :)
>> >
>> >Plus:
>> >
>> >SeaMonkey: Uses Geolocation and Safe Browsing APIs from Google.
>>
>> For Geo - this is unauthorized access as near as I can tell. I think
>> that we use to use a safe browser API that was 'public'.
>
> Until recently, using the public safe browser shavars required code
> changes. Now it's only a pref away, but the default is still to use the
> shavar that requires Google's approval, not the public one. Bug 557752
> should be put forward, and, in fact, should not be about unbranded vs
> branded, but Mozilla vs. not.

I fixed this in Bug 825417 so now each project can specify what shavars
to use:
http://hg.mozilla.org/mozilla-central/rev/740cf756cf33

> I think most third party API usage is in the same vein, requiring code
> changes to not use them when you're not authorized to, which, as you
> imply, you're not unless you're Mozilla. It's fine-ish that some
> features might not be available when you're not authorized, but it's not
> fine that the code a) does unauthorized things by default and b) doesn't
> allow to do the right thing without modifying the code.
>
> Mike

[Gervase Markham]

On 17/06/13 19:51, Gavin Sharp wrote:
> "Mozilla produced build" in this context is just an imprecise way of
> saying "the products for which the Mozilla Corporation has reached an
> agreement with Google for API keys, on behalf of the Mozilla project",
> i.e. Firefox.

Did we even attempt to get an agreement for all Mozilla software (or all
browser software), or was that not discussed? (If you don't know, who
would?)

One of the goals Mozilla has with source code licensing is that other
people should be able to take our code and build a browser which works
just as well (but without our branding). It would be a shame if that
laudable goal, which has been met for many years, was undermined by
increasing use of web services with Firefox-only contractual arrangements.

Not that I can see an obvious solution to the problem - these services
are important competitively - but this is a change which should not
pass without comment.

Gerv

[cka...@floodgap.com]

On Monday, June 24, 2013 2:22:17 AM UTC-7, Gervase Markham wrote:
> > "Mozilla produced build" in this context is just an imprecise way of
> > saying "the products for which the Mozilla Corporation has reached an
> > agreement with Google for API keys, on behalf of the Mozilla project",
> > i.e. Firefox.
>
> Did we even attempt to get an agreement for all Mozilla software (or all
> browser software), or was that not discussed? (If you don't know, who
> would?)

I've been waiting to find out a final determination for TenFourFox. Because it's limited to Power Macs, we can't use CoreLocation (no support in the OS). It's not a high priority feature, so disabling it entirely isn't a dealbreaker, but it would be nice to know if there was another option. I doubt Google would just give us a high-volume API key out of the goodness of their hearts; we still have in the ballpark of 20,000 active users and some small portion of them probably do use geolocation in some manner.

Cameron Kaiser

[Philip Chee]

On 24/06/2013 17:22, Gervase Markham wrote:
> On 17/06/13 19:51, Gavin Sharp wrote:
>> "Mozilla produced build" in this context is just an imprecise way of
>> saying "the products for which the Mozilla Corporation has reached an
>> agreement with Google for API keys, on behalf of the Mozilla project",
>> i.e. Firefox.
>
> Did we even attempt to get an agreement for all Mozilla software (or all
> browser software), or was that not discussed? (If you don't know, who
> would?)
>
> One of the goals Mozilla has with source code licensing is that other
> people should be able to take our code and build a browser which works
> just as well (but without our branding). It would be a shame if that
> laudable goal, which has been met for many years, was undermined by
> increasing use of web services with Firefox-only contractual arrangements.

IIRC that GPLv3 has clauses which address TIVOization. Did this issue
crop up during the MPLv2 discussion?

> Not that I can see an obvious solution to the problem - these services
> are important competitively - but this is a change which should not
> pass without comment.
>
> Gerv

Re: Geolocation:
Did Doug forget to fix B2G? It looks like it's still using the old uri.
Compare:
http://mxr.mozilla.org/mozilla-central/source/b2g/app/b2g.js#166
with
http://mxr.mozilla.org/mozilla-central/source/browser/app/profile/firefox.js#1287
q.v. Bug 882485.

Also how does FirefoxOS fit into this? Do all those phone makers have to
get their own keys from Google?

How about all those developers who have bought the Keon/Peak phones?