UseCounters on methods suffer from feature detection spam

[Vincent Scheib]

I've noticed that UseCounter UMA are fragile to web-app feature detection.  These UMA are easy to enable via the IDL Measure/MeasureAs attributes, and used widely.   However, code that tests for the existence of methods counts as a use.

E.g. we recently saw a spike in MeasureAs=WebBluetoothRequestDevice, but not a matching result in RecordRequestDeviceOutcome. (googlers only link to see charts)

It seems to offer little value to measure feature detection.  I haven't yet looked into the practicality of changing what is measured, but presume we could change behavior such that we only count method calls and not just access to methods.

Would this be a reasonable change to make?

[Philip Jägenstedt]

Looking at the generated code in V8Bluetooth.cpp, the use counter is only hit when the method is invoked, although there are a few possible exceptions before Bluetooth::requestDevice is reached.

It could be that there's been an increase in calls, but that many/most of them are failing. Perhaps it's tracking scripts, and they often run over HTTP and thus fail the secure context checks?

P.S. Feature detection spam is a big problem for attributes, and there sometimes the only fix is to analyze httparchive to see what usage patterns are most common. That might help here, at least if it's fingerprinting scripts and not for "real" usage.

--
You received this message because you are subscribed to the Google Groups "platform-predictability" group.
To unsubscribe from this group and stop receiving emails from it, send an email to platform-predicta...@chromium.org.
To post to this group, send email to platform-pr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/platform-predictability/CAK-EfXnEwuPEOedWRRfJV0NF-f7sHmJsGS-QKCM5obj-Hmv1Qw%40mail.gmail.com.

[Vincent Scheib]

Ah, I didn't understand the current impl of UseCounter on methods /me hangs head in shame of not reading the code/.

Re: HTTPArchive... wow, 30min in and I still can't do a simple grep for a string on web pages. I found this article which was overkill for a grep.  Is there a place for basic intro to using httparchive?

On Tue, Aug 29, 2017 at 1:44 AM, Philip Jägenstedt <foo...@google.com> wrote:

Looking at the generated code in V8Bluetooth.cpp, the use counter is only hit when the method is invoked, although there are a few possible exceptions before Bluetooth::requestDevice is reached.

It could be that there's been an increase in calls, but that many/most of them are failing. Perhaps it's tracking scripts, and they often run over HTTP and thus fail the secure context checks?

P.S. Feature detection spam is a big problem for attributes, and there sometimes the only fix is to analyze httparchive to see what usage patterns are most common. That might help here, at least if it's fingerprinting scripts and not for "real" usage.

On Tue, Aug 29, 2017 at 1:44 AM Vincent Scheib <sch...@chromium.org> wrote:

I've noticed that UseCounter UMA are fragile to web-app feature detection.  These UMA are easy to enable via the IDL Measure/MeasureAs attributes, and used widely.   However, code that tests for the existence of methods counts as a use.

E.g. we recently saw a spike in MeasureAs=WebBluetoothRequestDevice, but not a matching result in RecordRequestDeviceOutcome. (googlers only link to see charts)

It seems to offer little value to measure feature detection.  I haven't yet looked into the practicality of changing what is measured, but presume we could change behavior such that we only count method calls and not just access to methods.

Would this be a reasonable change to make?

--
You received this message because you are subscribed to the Google Groups "platform-predictability" group.

To unsubscribe from this group and stop receiving emails from it, send an email to platform-predictability+unsub...@chromium.org.
To post to this group, send email to platform-predictability@chromium.org.

[Philip Jägenstedt]

Oh, sorry, I should have pointed you at https://www.chromium.org/blink/platform-predictability/compat-tools and HTTP Archive for web compat decision making from the beginning. I'm not sure these are instructions are easy to follow, but they are geared towards greping the content as opposed to all of the other data that one can query httparchive about.

Filtering and understanding all results can be very time consuming, so if you don't immediately find it used in some widespread library/analytics, maybe the "Find pages that trigger a UseCounter and sort by page rank" template could help you figure out if some big site is tickling this use counter on every page load.

On Tue, Aug 29, 2017 at 8:01 PM Vincent Scheib <sch...@chromium.org> wrote:

Ah, I didn't understand the current impl of UseCounter on methods /me hangs head in shame of not reading the code/.

Re: HTTPArchive... wow, 30min in and I still can't do a simple grep for a string on web pages. I found this article which was overkill for a grep.  Is there a place for basic intro to using httparchive?

On Tue, Aug 29, 2017 at 1:44 AM, Philip Jägenstedt <foo...@google.com> wrote:

Looking at the generated code in V8Bluetooth.cpp, the use counter is only hit when the method is invoked, although there are a few possible exceptions before Bluetooth::requestDevice is reached.

It could be that there's been an increase in calls, but that many/most of them are failing. Perhaps it's tracking scripts, and they often run over HTTP and thus fail the secure context checks?

P.S. Feature detection spam is a big problem for attributes, and there sometimes the only fix is to analyze httparchive to see what usage patterns are most common. That might help here, at least if it's fingerprinting scripts and not for "real" usage.

On Tue, Aug 29, 2017 at 1:44 AM Vincent Scheib <sch...@chromium.org> wrote:

I've noticed that UseCounter UMA are fragile to web-app feature detection.  These UMA are easy to enable via the IDL Measure/MeasureAs attributes, and used widely.   However, code that tests for the existence of methods counts as a use.

E.g. we recently saw a spike in MeasureAs=WebBluetoothRequestDevice, but not a matching result in RecordRequestDeviceOutcome. (googlers only link to see charts)

It seems to offer little value to measure feature detection.  I haven't yet looked into the practicality of changing what is measured, but presume we could change behavior such that we only count method calls and not just access to methods.

Would this be a reasonable change to make?

--
You received this message because you are subscribed to the Google Groups "platform-predictability" group.

To unsubscribe from this group and stop receiving emails from it, send an email to platform-predicta...@chromium.org.
To post to this group, send email to platform-pr...@chromium.org.