Google Groups

Re: Proposal to Change Talos Data Calculations


slew...@mozilla.com Nov 4, 2011 5:29 PM
Posted in group: mozilla.dev.platform
Some additional graphs/insights.  The proposed changes are expected to slightly increase the noise and variance in the data as high valued anomalies may no longer be discarded.  The benefit however is the elimination of systematic biases in the data. The proposed scheme increases the output by a similar amount across most of the builds in the week of Oct 24-30, with a few larger changes (88cd8e9287c8 for fedora64 and 532d27c289de for snowleopard-r4) [1].  In the week of October 24 through 30, the same set of small pages were discarded meaning the existing scheme hides the impact of these tests [2].

[1] http://people.mozilla.org/~slewchuk/graphs/tp5_aggregation_differences.pdf
First graph shows the existing aggregation scheme (blue) and proposed scheme (red).  Second graph shows the difference between the two schemes.

[2] http://people.mozilla.org/~slewchuk/graphs/tp5_discarded_pages.pdf
This graph shows the counts of the pages being discarded by the current scheme.  Raw counts below:

                                 fedora fedora64 leopard snowleopard snowleopard-r4 win7  xp
  alipay.com/.../index.html           8       11       3           3              4    2   1
  bild.de/.../index.html              0       25       0          24             62   22   0
  dailymail.co.uk/.../index.html    118      102     158         149            110  117 135
  xinhuanet.com/.../index.html        0        0      14           3              3    0   0

If any one has questions about the graphs/data/proposals please ping me and I'll happily answer them,

Stephen