New Web Analytics Vendor

[Stacy Martin]

Mozilla has a wide range of websites and Web properties. To understand
how users interact with our pages, and improve the user experience, we
utilize Web analytics tools. As with any tools at Mozilla, its
crucial to our success that user privacy is carefully considered.

We've changed vendors over time, as needs and capabilities change.
Decisions are based on a benefits analysis, combined with a privacy
review, to ensure that a new vendor matches Mozilla’s business and
privacy needs.

In 2008, Mitchell blogged about Mozilla’s proposed use of Web
analytics and the benefits of such tools as well as the safeguards
they provide. We want to take a moment to revisit the points she made
and discuss why we have decided to go with a new vendor.

In 2008, Mozilla made the following commitments:

1) We will use the Web analytics data only to determine aggregate
usage patterns for our websites.
2) We will not seek to determine personal information from this
data.
3) Our Web analytics provider will use the data from Mozilla websites
only to provide and maintain the service for Mozilla. It will not
share the information with others or use the information for other
purposes.
4) Our analytics provider will not correlate and report on any
customer data with any other data collected through other products,
services or Web properties.
5) The domain names in Mozilla cookies will clearly identify their
affiliation with Mozilla and the Web analytics service.

In 2012, these commitments have not changed, but Google Analytics
has.

Google Analytics Premium Service now offers clients the ability to use
the settings page to block any secondary use of the information
collected through the service. This is a significant change, and one
that enables Mozilla to consider Google as our Web analytics provider.

Google Analytics will not correlate or report on any customer data
with any other data, they will use Mozilla data only to provide and
maintain the service for Mozilla, and they will not share or use it
for any other purpose. Google Analytics will also allow us to
increase the ease and accuracy with which we can access the
information crucial to our daily operations and business insights.
Without sacrificing user privacy, we will gain knowledge that will
enable us to better serve our users and create a better, more
effective, and more user-friendly Mozilla Web universe.

As Mitchell noted in 2008, we live in a world of data; we should be
thinking carefully about that data and its impact. This is even more
true today, and we believe we have maintained our vigilance.

[Ralph Giles]

Great news! One question:

> 4) Our analytics provider will not correlate and report on any
> customer data with any other data collected through other products,
> services or Web properties.

Does this apply to our current use of WebTrends? Ghostery and the
earlier, more judgement Collusion addons both identify it as a tracking
site.

Their privacy policy says they anonymize user data before sharing it,
and also that they use it for targetted advertising. It doesn't appear
they implement Do-Not-Track.

-r

[Byron Jones]

webtrends doesn't use the data it collects for its clients for targeted
advertising, nor does it share it with other sites.
see the "Information we collect for our clients" section of
http://webtrends.com/privacy-policy/



--
byron - irc:glob - bugzilla.mozilla.org team -

[Ralph Giles]

On 12-05-07 9:06 PM, Byron Jones wrote:

> webtrends doesn't use the data it collects for its clients for targeted
> advertising, nor does it share it with other sites.

Thanks for clarifying. I was not clear which categories of use are
included in the "Services" the other sections refer to.

> see the "Information we collect for our clients" section of
> http://webtrends.com/privacy-policy/

They do share the data, it's just anonymized and binned first. "...we
may share anonymized, aggregated data that does not identify any client
or any personal information included in ‘Customer Data’."

-r

[Pascal Chevrel]

Le 08/05/2012 00:53, Stacy Martin a écrit :

> As Mitchell noted in 2008, we live in a world of data; we should be
> thinking carefully about that data and its impact. This is even more
> true today, and we believe we have maintained our vigilance.

If we are thinking of moving to Google Analytics, why not host our own
analytics software such as Piwik? It's an excellent open source
equivalent to Google Analytics and it removes all the privacy concerns.
Furthermore, since it's open source software, we can send them patches
if we have specific needs.

Pascal

[Nukeador]

2012/5/8 Pascal Chevrel <pascal....@free.fr>

> If we are thinking of moving to Google Analytics, why not host our own
> analytics software such as Piwik? It's an excellent open source equivalent
> to Google Analytics and it removes all the privacy concerns. Furthermore,
> since it's open source software, we can send them patches if we have
> specific needs.
>

+1 to evaluating self hosted and open source tools.

Regards.
--
Rubén Martín (Nukeador)
Mozilla Reps Council member
http://mozilla-hispano.org
http://twitter.com/mozilla_hispano
http://facebook.com/mozillahispano

[Stacy Martin]

On May 8, 4:51 am, Nukeador <nukea...@mozilla-hispano.org> wrote:
> 2012/5/8 Pascal Chevrel <pascal.chev...@free.fr>

>
> > If we are thinking of moving to Google Analytics, why not host our own
> > analytics software such as Piwik? It's an excellent open source equivalent
> > to Google Analytics and it removes all the privacy concerns. Furthermore,
> > since it's open source software, we can send them patches if we have
> > specific needs.
>
> +1 to evaluating self hosted and open source tools.

Thank you for these thoughtful questions. Here are some responses
from the team. I will also reach out to a couple others for more
information. - Stacy

- Maintenance: Piwik is self hosted and for massive installations like
ours this is headcount intensive. People I talked to mentioned that
this was one of the reasons we moved away from Urchin (the original
and self hosted solution that Google bought to build Google
Analytics).
- Training: A lot of people at the company are already familiar with
GA. Using Piwik would require additional hours to gain familiarity.
- Features: While it seems that everything important is there, it's
unclear to me how powerful the segmentation capabilities are.
- Multidomain tracking: From the documentation and the online demo, it
seems that the multidomain tracking is not comparable.

[:mrz]

> - Maintenance: Piwik is self hosted and for massive installations like
> ours this is headcount intensive. People I talked to mentioned that
> this was one of the reasons we moved away from Urchin (the original
> and self hosted solution that Google bought to build Google
> Analytics).

Stacy's right - Urchin was difficult to manage at scale and difficult to maintain when upgrades came out.

[:mrz]

> - Maintenance: Piwik is self hosted and for massive installations like
> ours this is headcount intensive. People I talked to mentioned that
> this was one of the reasons we moved away from Urchin (the original
> and self hosted solution that Google bought to build Google
> Analytics).

[Henri Sivonen]

On Thu, May 17, 2012 at 8:10 PM, Stacy Martin <stacyc...@gmail.com> wrote:
> - Multidomain tracking: From the documentation and the online demo, it
> seems that the multidomain tracking is not comparable.

It seems a bit weird for Mozilla to be pushing Do Not Track and doing
multi-domain tracking at the same time. Do we have some kind of
organization-wide theory of what sort of multi-domain tracking is
okay?

--
Henri Sivonen
hsiv...@iki.fi
http://hsivonen.iki.fi/

[David Bruant]

Le 18/05/2012 09:13, Henri Sivonen a écrit :
> On Thu, May 17, 2012 at 8:10 PM, Stacy Martin<stacyc...@gmail.com> wrote:
>> - Multidomain tracking: From the documentation and the online demo, it
>> seems that the multidomain tracking is not comparable.
> It seems a bit weird for Mozilla to be pushing Do Not Track and doing
> multi-domain tracking at the same time.

If by "domain" you mean "domain name" in the Internet (DNS) sense, then,
I disagree.
Regardless of the domain name, the user interacts with Mozilla, so I
don't understand what the moral problem is.
As long as Mozilla (the same "authority domain" if I can put it this
way) is behind the different domains, what is the difference between
interacting with several pages within the same domain or with pages
spread across different domains?

David

[Henri Sivonen]

On Fri, May 18, 2012 at 11:15 AM, David Bruant <brua...@gmail.com> wrote:
> Regardless of the domain name, the user interacts with Mozilla, so I don't
> understand what the moral problem is.
> As long as Mozilla (the same "authority domain" if I can put it this way) is
> behind the different domains, what is the difference between interacting
> with several pages within the same domain or with pages spread across
> different domains?

A user who is logged into Gmail might be uncomfortable with the same
authority connecting YouTube accesses to the same person. Likewise,
it's not unthinkable that people might not like having what
lightweight themes they install from getpersonas.com associated with
their MDN or SUMO login.

[David Bruant]

Le 18/05/2012 12:58, Henri Sivonen a �crit :

And on wiki.mozilla.org, maybe I don't care about being tracked on most
pages, but don't want to be tracked when I'm looking at other user
pages. Both being under the same wiki.mozilla.org domain, how do you
differenciate?

I understand and agree with your point, but it leads to a complicated
situation, because the granularity to which people might not want to be
tracked can be arbitrary.

Regardless of what is being decided for Mozilla websites, I hope that
the tracking decisions will be clearly documented, saying:
* Exactly which information is being collected (and I wish to see a
technical as well as a non-technical explanation here)
* From which website
* How long are the data being kept
* For what purpose
* Are informations from different sources bundled? how? for what
purpose? (once again, being technical as well as non-technical in these
explanations would be appropriate for the sake transparency)
* Maybe put all this information on mozilla.org/websites/privacy (since
/privacy is taken already)
It could be interesting if /privacy became a de facto standard, very
much like /about in websites.

Having all this information does not guarantee no tracking, but makes
cristal clear how information is being handled. Being ready to receive
feedback on this documentation and being reactive would be the next step.

David

[Henri Sivonen]

On Fri, May 18, 2012 at 3:02 PM, David Bruant <brua...@gmail.com> wrote:
> I understand and agree with your point, but it leads to a complicated
> situation, because the granularity to which people might not want to be
> tracked can be arbitrary.

Going back to my question "It seems a bit weird for Mozilla to be

pushing Do Not Track and doing multi-domain tracking at the same time.

Do we have some kind of
organization-wide theory of what sort of multi-domain tracking is

okay?", I take it that the answer is "We don't". :-(

[Asa Dotzler]

On 5/21/2012 1:45 AM, Henri Sivonen wrote:
> On Fri, May 18, 2012 at 3:02 PM, David Bruant<brua...@gmail.com> wrote:
>> I understand and agree with your point, but it leads to a complicated
>> situation, because the granularity to which people might not want to be
>> tracked can be arbitrary.
>
> Going back to my question "It seems a bit weird for Mozilla to be
> pushing Do Not Track and doing multi-domain tracking at the same time.
> Do we have some kind of
> organization-wide theory of what sort of multi-domain tracking is
> okay?", I take it that the answer is "We don't". :-(

I don't agree wit your implied premise. Mozilla is not trying to end
tracking. We're trying to put users in control. Multi-domain tracking,
IMO, is completely OK in certain scenarios so long as the user is in
control. Mozilla is a non-profit organization trying to promote a
positive mission for people and the Web. Mozilla can achieve that
mission more effectively by understanding how I use Mozilla websites. I
have no problem with Mozilla wanting to know about my movement around
different Mozilla domains. It's obvious to users that these are all
Mozilla websites (we're not putting Facebook or Twitter style beacons on
non-Mozilla pages here) and users can opt out using DNT if they think
Mozilla understanding their movements on Mozilla sites is a problem.

So, to answer your question directly, no, I don't think it's a bit weird
for Mozilla to be pushing DNT and doing multi-domain tracking.

- A

- A

[sabine.micha...@gmail.com]

> I don't agree wit your implied premise. Mozilla is not trying to end
> tracking. We're trying to put users in control.

If we want to put users in control of tracking then surely the minimum criteria for our use of analytics companies should be that they honor our own anti-tracking system.

> It's obvious to users that these are all
> Mozilla websites (we're not putting Facebook or Twitter style beacons on
> non-Mozilla pages here) and users can opt out using DNT if they think
> Mozilla understanding their movements on Mozilla sites is a problem.
>

mozilla.org uses webtrends.com and if you look at http://www.privacychoice.org/companies/index/66/webtrends you will see that webtrends.com fail to honor do not track.

[Rubén Martín]

El 21/05/12 18:28, sabine.micha...@gmail.com escribió:

> If we want to put users in control of tracking then surely the minimum criteria for our use of analytics companies should be that they honor our own anti-tracking system.

That's a really good point now that we are seeing more and more
companies supporting DNT and most browsers support it too.

(And yes, Google is the only one not supporting it yet)

Regards.

--
Rubén Martín [Nukeador]
Mozilla Reps Council Member
http://www.mozilla-hispano.org
http://twitter.com/mozilla_hispano
http://facebook.com/mozillahispano

[David Ascher]

On May 21 '12 1:45 AM, Henri Sivonen wrote:
> On Fri, May 18, 2012 at 3:02 PM, David Bruant <brua...@gmail.com> wrote:
>> I understand and agree with your point, but it leads to a complicated
>> situation, because the granularity to which people might not want to be
>> tracked can be arbitrary.
> Going back to my question "It seems a bit weird for Mozilla to be
> pushing Do Not Track and doing multi-domain tracking at the same time.
> Do we have some kind of
> organization-wide theory of what sort of multi-domain tracking is
> okay?", I take it that the answer is "We don't". :-(
>

My understanding is that DNT is actually not about multi-domain tracking
but about cross-org tracking.

--da

[xio...@gmail.com]

On Thursday, May 17, 2012 6:10:41 PM UTC+1, Stacy Martin wrote:
>
> Thank you for these thoughtful questions. Here are some responses
> from the team. I will also reach out to a couple others for more
> information. - Stacy
>
> - Maintenance: Piwik is self hosted and for massive installations like
> ours this is headcount intensive. People I talked to mentioned that
> this was one of the reasons we moved away from Urchin (the original
> and self hosted solution that Google bought to build Google
> Analytics).
> - Training: A lot of people at the company are already familiar with
> GA. Using Piwik would require additional hours to gain familiarity.
> - Features: While it seems that everything important is there, it's
> unclear to me how powerful the segmentation capabilities are.
> - Multidomain tracking: From the documentation and the online demo, it
> seems that the multidomain tracking is not comparable.

I find a bit sad that we're directly going toward Google products/proprietary softwares without even trying Open Source alternatives first.
According to the documentation, we can scale it to our needs, and even if we encounter some problems, we should work with their developers, the community, or even some of our devs. It's not because Urchin (not sure but I think it was a proprietary tool) was difficult to manage at scale, that Piwik will.
If we want some additional features we can offer a bounty to whoever do it (thing that we can't have with GA) so we can raise it to our exact needs.
That way the data stays within Mozilla, we help and contribute to open source projects, we get more freedom about the tool we're using and how we want to customize it and we get some good karma :)
About the training, if it requires a couple of hours to learn a tool that we're going to use for many years, I don't see any problem.

--
Arzhel

[xio...@gmail.com]

On Thursday, May 17, 2012 6:10:41 PM UTC+1, Stacy Martin wrote:
>

> Thank you for these thoughtful questions. Here are some responses
> from the team. I will also reach out to a couple others for more
> information. - Stacy
>
> - Maintenance: Piwik is self hosted and for massive installations like
> ours this is headcount intensive. People I talked to mentioned that
> this was one of the reasons we moved away from Urchin (the original
> and self hosted solution that Google bought to build Google
> Analytics).
> - Training: A lot of people at the company are already familiar with
> GA. Using Piwik would require additional hours to gain familiarity.
> - Features: While it seems that everything important is there, it's
> unclear to me how powerful the segmentation capabilities are.
> - Multidomain tracking: From the documentation and the online demo, it
> seems that the multidomain tracking is not comparable.

[:mrz]

> I find a bit sad that we're directly going toward Google products/proprietary softwares without even trying Open Source alternatives first.

Where should Mozilla (and specifically, Mozilla IT) spend it's limited resources?

Would you argue that we shift resources off Marketplace or RelEng or BrowserID to stand up and manage a high volume analytics solution?

(btw, we did try open source. I was here when we did it and recall first hand how much of a resource drain it was for something that isn't our core competency).

[Rubén Martín]

El 22/05/12 19:30, :mrz escribió:

> Would you argue that we shift resources off Marketplace or RelEng or BrowserID to stand up and manage a high volume analytics solution?

I will repeat myself here about new tools and mozilla resources :P

Maybe these thinks are great opportunities to ask volunteers to help and
get involved with paid staff on IT, webdev... etc

Most of the times when we speak about mozilla resources we are
forgetting the huge amount of volunteers that could be coordinated to
get in charge of these things.

[:mrz]

On Tuesday, May 22, 2012 1:45:48 PM UTC-7, Rubén Martín wrote:
> El 22/05/12 19:30, :mrz escribió:
> > Would you argue that we shift resources off Marketplace or RelEng or BrowserID to stand up and manage a high volume analytics solution?
> I will repeat myself here about new tools and mozilla resources :P
>
> Maybe these thinks are great opportunities to ask volunteers to help and
> get involved with paid staff on IT, webdev... etc

When I speak of Mozilla I do not differentiate between paid and non-paid staff.

In both cases, you're diverting resources from other projects to something that is not Mozilla's core competency or strength. Running a high volume analytics solution is not easy and requires a lot of physical resources that divert resources from where we should be focusing our time/efforts.

[David Ascher]

I think it would be great if there was an existing OSS analytics were
Good Enough to do what we need for our own usage. If that's not the
case (which I believe is true, having tried various ones myself, and
compared them to GA), I think it'd be great to contribute to (or build
from scratch) something that was competitive. I've been muttering
something about that for a while now to those who will listen.

That said, I very much don't think we should let ideal hypothetical
outcomes get in the way of getting data we need yesterday. So I don't
think it's an either/or discussion.

- it's great that there's now a way to get the amazing analytics that
GA provides under a privacy regime that is consistent with our values.

- if that catalyzes action towards building or improving OSS systems to
be better, even better, and once the OSS system gets good enough, we
can consider switching.

--david

[:mrz]

On Tuesday, May 22, 2012 1:45:48 PM UTC-7, Rubén Martín wrote:

> El 22/05/12 19:30, :mrz escribió:
> > Would you argue that we shift resources off Marketplace or RelEng or BrowserID to stand up and manage a high volume analytics solution?
> I will repeat myself here about new tools and mozilla resources :P
>
> Maybe these thinks are great opportunities to ask volunteers to help and
> get involved with paid staff on IT, webdev... etc

[Rubén Martín]

El 22/05/12 23:14, :mrz escribió:

> In both cases, you're diverting resources from other projects to something that is not Mozilla's core competency or strength. Running a high volume analytics solution is not easy and requires a lot of physical resources that divert resources from where we should be focusing our time/efforts.

It depends, for example we have a Labs area on Mozilla Hispano with a
lot of developers that want to help with something, in fact we have more
volunteers than projects, and I assume other communities have a huge
number of web devs too. We can point some of them to a person in charge
of coordinating this or other webdev efforts.

If we don't ask all the community we are not going to be sure if we have
or not the resources, most volunteers don't follow this list for
example. Let's ping a key person from each community in charge of
development and let them tell us if there are or there are not resources
from the community ;)

[:mrz]

> If we don't ask all the community we are not going to be sure if we have
> or not the resources, most volunteers don't follow this list for
> example.

This is very true and I'll let you take point on that. Could very easily have two platforms running side-by-side.

But even so, hosting this platform (servers & storage, data center) and running it the SLA we'd want for this is not trivial and takes paid-staff resources (unfortunately).

The Socorro cluster is two 45u racks of HP SL chassis and one HP c7000 chassis. Sync is 35 racks of 2u servers. The existing blocklist metrics infrastructure is several tens of computers and a whole team. All of these run at a fairly high availability and none have the volume that a web analytics platform would need.

I completely advocate for open source when it make sense.

I'm not convinced, right now, that writing & supporting a web analytics platform is a good use of Mozilla resources, paid or non-paid. I'm also not convinced it further's Mozilla's mission right now.

I would also repeat everything David said.

[:mrz]

> If we don't ask all the community we are not going to be sure if we have
> or not the resources, most volunteers don't follow this list for
> example.

[Gijs Kruitbosch]

On 22/05/2012 19:44 PM, :mrz wrote:
>
>> If we don't ask all the community we are not going to be sure if we have
>> or not the resources, most volunteers don't follow this list for
>> example.
>
> This is very true and I'll let you take point on that. Could very easily have two platforms running side-by-side.
>

> But even so, hosting this platform (servers& storage, data center) and running it the SLA we'd want for this is not trivial and takes paid-staff resources (unfortunately).

>
> The Socorro cluster is two 45u racks of HP SL chassis and one HP c7000 chassis. Sync is 35 racks of 2u servers. The existing blocklist metrics infrastructure is several tens of computers and a whole team. All of these run at a fairly high availability and none have the volume that a web analytics platform would need.
>
> I completely advocate for open source when it make sense.
>

> I'm not convinced, right now, that writing& supporting a web analytics platform is a good use of Mozilla resources, paid or non-paid. I'm also not convinced it further's Mozilla's mission right now.

>
> I would also repeat everything David said.

I am definitely not an expert, but aren't there platforms to take care of the
server, storage and data center stuff? Amazon AWS/EC2/S3 is well-known, but I'm
sure there are others... Perhaps we could run the OSS alternative on that and
reduce the running cost to the software maintenance and any brokering with the
third-party hosting that we'd need to do?

~ Gijs

[Mitchell Baker]

On 5/22/12 3:59 PM, Rubén Martín wrote:
>
> It depends, for example we have a Labs area on Mozilla Hispano with a
> lot of developers that want to help with something, in fact we have more
> volunteers than projects, and I assume other communities have a huge
> number of web devs too. We can point some of them to a person in charge
> of coordinating this or other webdev efforts.
>

Nukeador --

This is a really interesting topic. More developers than projects --
what a treasure!

One question is whether the projects Mozilla is already focused on
aren't of interest -- and here I include things like Popcorn, popcorn
maker, hackasaurus, etc. Or maybe these developers are web developers?

Or that general open source tools (an OS version of web analytics, or X
or Y or Z software) is a greater draw than the things Mozilla is focused
on now?

Or is it that there don't seem to be enough ways to get involved in the
Mozilla projects?

I'm not trying to judge or say one thing is better than another. Trying
to understand how to think about the developers.

Mitchell
>

[Nukeador]

2012/5/23 Mitchell Baker <mitc...@mozilla.com>

>
> One question is whether the projects Mozilla is already focused on aren't
> of interest -- and here I include things like Popcorn, popcorn maker,
> hackasaurus, etc. Or maybe these developers are web developers?
>
> Or that general open source tools (an OS version of web analytics, or X or
> Y or Z software) is a greater draw than the things Mozilla is focused on
> now?
>
> Or is it that there don't seem to be enough ways to get involved in the
> Mozilla projects?
>
> I'm not trying to judge or say one thing is better than another. Trying
> to understand how to think about the developers.
>

In our case we are trying to coordinate efforts between our Labs area with
the rest of the community but most of the times language is a barrier for
most contributors, so we need an English speaking person to act as bridge
between them and the person in charge of this developments.

I will ping Felipe Lerena, who is in charge of our Labs/Dev area, to get
more information about his plans for this year about this.