Re: New UseCounters

48 views
Skip to first unread message

Rick Byers

unread,
Jan 31, 2017, 9:32:06 AM1/31/17
to Philip Jägenstedt, blink-api-owners-discuss, platform-predictability, Hiroki Nakagawa
I've looked a little, but can't explain everything yet. You can see the raw data here.  A couple interesting ones:
  • DeprecatedFlexboxChromeExtension: fell from 0.002% to 0.00005%
    presumably because chrome-extensions:// URLs are now excluded.  That's clearly a problem for something that is trying to measure usage in extensions.  Also curious that it's not now zero, but I haven't looked at the code.

  • ApplicationCacheAPISecureOrigin: grew from 0.1% to 4%
    I'm not sure what could cause this.  Maybe this counter somehow tended to be associated with fast shutdown (where we were loosing data in the old scheme)?

  • SharedWorkerStart grew from 0.1% to 3%
    There has been (and continues to be) work to fix UseCounter behavior in workers.  But that impacts both the old and new histograms the same, so shouldn't be involved (the data here is from the exact same chrome versions - the two most recent dev channel releases).  

  • SVGSVGElementInXMLDocument fell from 8% to 0.08%
    SVG Images are very special - each has their own isolated 'Page' instance, and a single image can be used by multiple other Pages.  Usage within SVG Images is now tracked in a separate set of histograms - Blink.UseCounter.SVGImage.*


On Mon, Jan 30, 2017 at 9:58 PM, Philip Jägenstedt <foo...@chromium.org> wrote:
Great work, Rick! Do you happen to have a few of the counters where the difference was the largest? Those might tell an interesting story.

On Mon, Jan 30, 2017 at 5:32 PM Rick Byers <rby...@chromium.org> wrote:
Hey,
I've landed a new version of our UseCounters (under the Blink.UseCounter histograms) in Chrome 57, while (temporarily) preserving the original semantics in WebCore.FeatureObserver.  This fixes a number of bugs in how the counters were computed.

From a quick check of recent dev channel data (raw data Google internal - sorry), it seems that PageVisits was over-represented by about 3x (because we were counting every SVG Image as a "page").  The absolute fraction-of-PageVisit values are now about 3x-7x higher than what we were seeing before (with a few outliers with a larger difference).

Since this is still only in dev, we should keep using the old FeatureObserver data (it's still tells roughly the same story for the vast majority of metrics).  But sometime in Q2 I plan to switch chromestatus.com over to the new data and we should re-calibrate how we interpret the numbers in intent threads.


​
​

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.
To post to this group, send email to blink-api-owners-discuss@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAFUtAY-WXCpwBv8FmSGuvBQjHOeh3hScFtgnNW9oEh12M47BxA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.
To post to this group, send email to blink-api-owners-discuss@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAARdPYcRnjDDj%2BwogDgLVT%2BFihyhQXV0VrsLPHQ5bQhAQagooQ%40mail.gmail.com.

Mike West

unread,
Jan 31, 2017, 9:54:57 AM1/31/17
to Rick Byers, Philip Jägenstedt, blink-api-owners-discuss, platform-predictability, Hiroki Nakagawa
Thanks for this, Rick!

On Tue, Jan 31, 2017 at 6:31 AM, Rick Byers <rby...@chromium.org> wrote:
I've looked a little, but can't explain everything yet. You can see the raw data here.  A couple interesting ones:
  • DeprecatedFlexboxChromeExtension: fell from 0.002% to 0.00005%
    presumably because chrome-extensions:// URLs are now excluded.  That's clearly a problem for something that is trying to measure usage in extensions.  Also curious that it's not now zero, but I haven't looked at the code.
I commented on the bug, but I think it's worth commenting here too: discounting extensions entirely seems like the wrong long-term plan. For better or worse, users use extensions, and the behavior of that environment is something we really should be collecting data on, as it does constrain our ability to make certain changes.

Are we just excluding counters from `chrome-extension://` pages? Or are you also excluding things that content scripts do inside their isolated world?
  • ApplicationCacheAPISecureOrigin: grew from 0.1% to 4%
    I'm not sure what could cause this.  Maybe this counter somehow tended to be associated with fast shutdown (where we were loosing data in the old scheme)?
!!! 4% is a lot of percent. I hope this is a bug, as it would significantly impact our ability to get rid of appcache. :(

None of the triggers for this metric seem to be particularly related to shutdown: we poke the counter in `ApplicationCache::recordAPIUseType`, which is called from various IDL-bound APIs. I don't see much difference between those calls and any other, so the jump in usage really surprises me.

Perhaps we could extract a list of sites that trigger this counter from HTTP Archive, and see if it's a false positive or a real representation of what's going on? I'm happy to dig in a bit if you can run your magic scripts again with the new histogram. :)
  • SharedWorkerStart grew from 0.1% to 3%
    There has been (and continues to be) work to fix UseCounter behavior in workers.  But that impacts both the old and new histograms the same, so shouldn't be involved (the data here is from the exact same chrome versions - the two most recent dev channel releases).  

  • SVGSVGElementInXMLDocument fell from 8% to 0.08%
    SVG Images are very special - each has their own isolated 'Page' instance, and a single image can be used by multiple other Pages.  Usage within SVG Images is now tracked in a separate set of histograms - Blink.UseCounter.SVGImage.*

On Mon, Jan 30, 2017 at 9:58 PM, Philip Jägenstedt <foo...@chromium.org> wrote:
Great work, Rick! Do you happen to have a few of the counters where the difference was the largest? Those might tell an interesting story.

On Mon, Jan 30, 2017 at 5:32 PM Rick Byers <rby...@chromium.org> wrote:
Hey,
I've landed a new version of our UseCounters (under the Blink.UseCounter histograms) in Chrome 57, while (temporarily) preserving the original semantics in WebCore.FeatureObserver.  This fixes a number of bugs in how the counters were computed.

From a quick check of recent dev channel data (raw data Google internal - sorry), it seems that PageVisits was over-represented by about 3x (because we were counting every SVG Image as a "page").  The absolute fraction-of-PageVisit values are now about 3x-7x higher than what we were seeing before (with a few outliers with a larger difference).

Since this is still only in dev, we should keep using the old FeatureObserver data (it's still tells roughly the same story for the vast majority of metrics).  But sometime in Q2 I plan to switch chromestatus.com over to the new data and we should re-calibrate how we interpret the numbers in intent threads.


​
​

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsubscr...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsubscr...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "platform-predictability" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-predictability+unsub...@chromium.org.
To post to this group, send email to platform-predictability@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/platform-predictability/CAFUtAY8czsC4wFVpoycAVOKrHVQwV2PtGWeQRtM3A8aFUYdsyA%40mail.gmail.com.

Alex Komoroske

unread,
Feb 6, 2017, 1:22:41 PM2/6/17
to Mike West, Rick Byers, Philip Jägenstedt, blink-api-owners-discuss, platform-predictability, Hiroki Nakagawa
Exciting stuff!

Just wanted to check: when we switch over to using this, will the numbers on chromestatus.com update to use them?

Thanks for this, Rick!

To unsubscribe from this group and stop receiving emails from it, send an email to platform-predictability+unsubscr...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "platform-predictability" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-predictability+unsub...@chromium.org.
To post to this group, send email to platform-predictability@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/platform-predictability/CAKXHy%3Dc_AjiZOMQ0mNMFPX-ianjCTqy5cNDka%3Dz1Wfvo2PFJfw%40mail.gmail.com.

Philip Jägenstedt

unread,
Feb 6, 2017, 1:27:17 PM2/6/17
to Alex Komoroske, Mike West, Rick Byers, blink-api-owners-discuss, platform-predictability, Hiroki Nakagawa
Yes, at the end of Rick's first email, he wrote "sometime in Q2 I plan to switch chromestatus.com over to the new data and we should re-calibrate how we interpret the numbers in intent threads."

I'm not sure if we should just throw away the existing historical data. Even if there's a mysterious-looking discontinuity at the switch date, being able to see older trends would still be useful.

Thanks for this, Rick!

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To post to this group, send email to blink-api-ow...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To post to this group, send email to blink-api-ow...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "platform-predictability" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-predicta...@chromium.org.
To post to this group, send email to platform-pr...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "platform-predictability" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-predicta...@chromium.org.
To post to this group, send email to platform-pr...@chromium.org.

Alex Komoroske

unread,
Feb 6, 2017, 1:34:16 PM2/6/17
to Philip Jägenstedt, Mike West, Rick Byers, blink-api-owners-discuss, platform-predictability, Hiroki Nakagawa
On Mon, Feb 6, 2017 at 10:27 AM, Philip Jägenstedt <foo...@chromium.org> wrote:
Yes, at the end of Rick's first email, he wrote "sometime in Q2 I plan to switch chromestatus.com over to the new data and we should re-calibrate how we interpret the numbers in intent threads."

Whoops! 

Carry on. :-)  

I'm not sure if we should just throw away the existing historical data. Even if there's a mysterious-looking discontinuity at the switch date, being able to see older trends would still be useful.
On Mon, Feb 6, 2017 at 1:22 PM Alex Komoroske <komo...@chromium.org> wrote:
Exciting stuff!

Just wanted to check: when we switch over to using this, will the numbers on chromestatus.com update to use them?
Thanks for this, Rick!

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.
To post to this group, send email to blink-api-owners-discuss@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.
To post to this group, send email to blink-api-owners-discuss@chromium.org.

--
You received this message because you are subscribed to the Google Groups "platform-predictability" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-predictability+unsub...@chromium.org.
To post to this group, send email to platform-predictability@chromium.org.

--
You received this message because you are subscribed to the Google Groups "platform-predictability" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-predictability+unsub...@chromium.org.
To post to this group, send email to platform-predictability@chromium.org.
Reply all
Reply to author
Forward
0 new messages