Feature deprecation: measuring usage, and notifying developers.

Showing 1-14 of 14 messages
Feature deprecation: measuring usage, and notifying developers. Mike 4/26/13 5:06 AM
When we actively want to remove something from the platform, I'd suggest that it's valuable to both measure its usage in the wild, and notify developers of its impending doom. This should now be fairly simple to implement via either UseCounter::countDeprecation or [DeprecateAs]

Three steps:
  1. Add your feature to the UseCounter::Feature enum.
  2. Add a clever deprecation message to the big switch in UseCounter::deprecationMessage.
  3. Instrument your code by either:
Note that DeprecateAs is intended to replace MeasureAs in the IDL file. Specifying both is redundant.

Let me know if these options don't map well to your favorite deprecation. Once we're reasonably sure that this API makes sense, I'll throw it up on our wiki somewhere.

Thanks!

-mike
Re: [blink-dev] Feature deprecation: measuring usage, and notifying developers. Julien Chaffraix 4/26/13 8:03 AM
> When we actively want to remove something from the platform, I'd suggest
> that it's valuable to both measure its usage in the wild, and notify
> developers of its impending doom. This should now be fairly simple to
> implement via either UseCounter::countDeprecation or [DeprecateAs].
>
> Three steps:
>
> Add your feature to the UseCounter::Feature enum.
> Add a clever deprecation message to the big switch in
> UseCounter::deprecationMessage.
> Instrument your code by either:
>
> Adding DeprecateAs=[your enum value here] to the feature's IDL definition
> (see window.performance.webkitGetEntries, for example), or
> Adding a call to UseCounter::countDeprecation somewhere relevant (as we're
> dong for the prefixed Content Security Policy headers).
>
> Note that DeprecateAs is intended to replace MeasureAs in the IDL file.
> Specifying both is redundant.

I would add a warning based on my experience with trying to deprecate
UIEvent's layerX / layerY: your proposal seems to be to just go ahead
and deprecate some API while counting usage at the same time. Unless
you want to get developers' wrath on bugs or this mailing list, you
should probably first measure the use and then do the deprecation (ie
split the 2 steps). Making assumption about the brokenness of an API
from Blink code's point of view doesn't really translate to real-world
use.

Also does [DeprecateAs] only dumps once per page or is it for each use
of the deprecated API?

Thanks,
Julien
Re: [blink-dev] Feature deprecation: measuring usage, and notifying developers. Mike 4/26/13 8:38 AM
On Fri, Apr 26, 2013 at 5:03 PM, Julien Chaffraix <jchaf...@chromium.org> wrote:
I would add a warning based on my experience with trying to deprecate
UIEvent's layerX / layerY: your proposal seems to be to just go ahead
and deprecate some API while counting usage at the same time. Unless
you want to get developers' wrath on bugs or this mailing list, you
should probably first measure the use and then do the deprecation (ie
split the 2 steps). Making assumption about the brokenness of an API
from Blink code's point of view doesn't really translate to real-world
use.

To some extent, I agree. Jumping directly to deprecation is probably best suited for unprefixing not-so-widely-used bits and pieces of API. For other APIs, it might make sense to measure for an extended period before moving to deprecation.

That said, measurement takes ~12-18 weeks to hit stable; if we actively intend to remove support for a feature, telling users about that earlier rather than later is valuable both for them and for us. Content Security Policy is a good example: we unprefixed the header in M25, but see developers launching new sites using 'X-WebKit-CSP'. I'd like to prevent that from becoming a de facto standard.

Also does [DeprecateAs] only dumps once per page or is it for each use
of the deprecated API?

Warnings hit the console only once per page, and all come from DeprecationMessageSource; if we end up with tons, we'll be able to somehow filter them in the console.

Thanks!

-mike
Re: [blink-dev] Feature deprecation: measuring usage, and notifying developers. Julien Chaffraix 4/26/13 9:21 AM
>> I would add a warning based on my experience with trying to deprecate
>> UIEvent's layerX / layerY: your proposal seems to be to just go ahead
>> and deprecate some API while counting usage at the same time. Unless
>> you want to get developers' wrath on bugs or this mailing list, you
>> should probably first measure the use and then do the deprecation (ie
>> split the 2 steps). Making assumption about the brokenness of an API
>> from Blink code's point of view doesn't really translate to real-world
>> use.
>
>
> To some extent, I agree. Jumping directly to deprecation is probably best
> suited for unprefixing not-so-widely-used bits and pieces of API. For other
> APIs, it might make sense to measure for an extended period before moving to
> deprecation.
>
> That said, measurement takes ~12-18 weeks to hit stable; if we actively
> intend to remove support for a feature, telling users about that earlier
> rather than later is valuable both for them and for us. Content Security
> Policy is a good example: we unprefixed the header in M25, but see
> developers launching new sites using 'X-WebKit-CSP'. I'd like to prevent
> that from becoming a de facto standard.

There seem to be 2 cases to consider:
1) We know we want to remove support for an API (unprefixing, dead-end
API, ...).
2) We consider the feature to be a burden in the code and would like
to remove it.

For 1), it definitely makes sense to give developers a head start by
putting the warning about the deprecation earlier rather than later,
especially for young API that didn't have time to spread to a lot of
pages.
For 2), it's a good idea to split the 2 parts.

Content Security Policy falls into 1) so it's fine to just do both at
the same time. I would just not make usage measurement and deprecation
the default and only way to go.

>> Also does [DeprecateAs] only dumps once per page or is it for each use
>> of the deprecated API?
>
>
> Warnings hit the console only once per page, and all come from
> DeprecationMessageSource; if we end up with tons, we'll be able to somehow
> filter them in the console.

That's very good thanks.

Julien
Re: [blink-dev] Feature deprecation: measuring usage, and notifying developers. Mike West 4/26/13 9:38 AM


--
Mike West <mk...@google.com>, Developer Advocate
Google Germany GmbH, Dienerstrasse 12, 80331 München, Germany
Google+: https://mkw.st/+, Twitter: @mikewest, Cell: +49 162 10 255 91


This sounds quite reasonable (and, FWIW, I think #1 is more common than #2). I think we'd be on the same page if my original step 1 had read something like:

1. Find your feature in the UseCounter::Feature enum. If it doesn't exist, add it (and carefully consider whether you can jump straight to deprecation without preexisting measurement).

Does that more or less match your mental model?

-mike

Re: [blink-dev] Feature deprecation: measuring usage, and notifying developers. Ben Vanik 4/26/13 10:02 AM
Could it be encouraged to cherry pick new measurement counters into stable/beta releases? Waiting 12-18 weeks just to *start* to get numbers feels extremely low velocity.
Re: [blink-dev] Feature deprecation: measuring usage, and notifying developers. Julien Chaffraix 4/26/13 11:05 AM
> Could it be encouraged to cherry pick new measurement counters into
> stable/beta releases? Waiting 12-18 weeks just to *start* to get numbers
> feels extremely low velocity.

I would be supportive of that but that's because I don't see the
downsides to the approach (the measurement code has been around for a
long time so stability shouldn't be an issue). Would love to be
educated on them though!

Having hand-rolled some usage histograms, you usually forget you added
them by the time they start gathering data on stable (which are
usually the data you want to make your final decision).

>>> There seem to be 2 cases to consider:
>>> 1) We know we want to remove support for an API (unprefixing, dead-end
>>> API, ...).
>>> 2) We consider the feature to be a burden in the code and would like
>>> to remove it.
>>>
>>> For 1), it definitely makes sense to give developers a head start by
>>> putting the warning about the deprecation earlier rather than later,
>>> especially for young API that didn't have time to spread to a lot of
>>> pages.
>>> For 2), it's a good idea to split the 2 parts.
>>
>>
>> This sounds quite reasonable (and, FWIW, I think #1 is more common than
>> #2).

I don't see it this way as we have lots of proprietary legacy APIs
(think CSS reflection, overflow: -webkit-marquee, ...) that will need
to be taken care of - either by standardization or measurement +
deprecation - but that's beside the point of this discussion.

> I think we'd be on the same page if my original step 1 had read
>> something like:
>>
>> 1. Find your feature in the UseCounter::Feature enum. If it doesn't exist,
>> add it (and carefully consider whether you can jump straight to deprecation
>> without preexisting measurement).
>>
>> Does that more or less match your mental model?

That's fine by me.

Julien
Re: [blink-dev] Feature deprecation: measuring usage, and notifying developers. Vincent Scheib 4/26/13 12:58 PM
Re: [blink-dev] Feature deprecation: measuring usage, and notifying developers. Max Heinritz 4/26/13 4:00 PM
My original intention was to have deprecations and feature removals go through the ordinary launch process, but after reading this thread I think it might be better to define distinct paths for new features vs deprecations. How does this sound:

How to Deprecate/Remove a Web Platform Feature

  1. Carefully consider whether you can jump straight to deprecation without first measuring the feature's usage in the wild.
    • If you'd rather measure first:
      • Add your feature to the UseCounter::Feature enum.
      • Add MeasureAs=[your enum value here] to the feature's IDL definition.*
  2. Email blink-dev using the "Intent to Deprecate" template when you're ready.
  3. Measure usage and notify developers.
    • If you haven't, add it to the UseCounter::Feature enum.
    • Instrument your code by either:
      • Adding DeprecateAs=[your enum value here] to the feature's IDL definition.*^ See window.performance.webkitGetEntries, for example.
      • Adding a call to UseCounter::countDeprecation somewhere relevant (as we're dong for the prefixed Content Security Policy headers).
    • Notify developers by adding a clever deprecation message to the big switch in UseCounter::deprecationMessage. This will print a deprecation warning in the DevTools console.
    • Note: it takes 12-18 weeks to hit Stable once you enable instrumentation.
  4. Email blink-dev using the "Intent to Remove" template when you're ready. Wait for LGTMs from ≥3 API OWNERS.
  5. Remove.
* It takes 12-18 weeks to hit Stable once you enable instrumentation.
DeprecateAs is intended to replace MeasureAs in the IDL file. Specifying both is redundant. 

See this blink-dev thread for more information on deprecation.

Also, every time a new Beta is released, we announce major web-facing API changes (including deprecations) on the Chromium blog. Here's the post for Chrome 27: http://blog.chromium.org/2013/04/chrome-27-beta-speedier-web-and-new.html. We'll continue using that channel as a heads up.
Re: [blink-dev] Feature deprecation: measuring usage, and notifying developers. Paweł Hajdan, Jr. 4/29/13 9:51 AM
On Fri, Apr 26, 2013 at 4:00 PM, Max Heinritz <m...@chromium.org> wrote:
My original intention was to have deprecations and feature removals go through the ordinary launch process, but after reading this thread I think it might be better to define distinct paths for new features vs deprecations. How does this sound:

How to Deprecate/Remove a Web Platform Feature

  1. Carefully consider whether you can jump straight to deprecation without first measuring the feature's usage in the wild.
    • If you'd rather measure first:
      • Add your feature to the UseCounter::Feature enum.
      • Add MeasureAs=[your enum value here] to the feature's IDL definition.*


It's not obvious (maybe it's just me), what are the criteria for measuring and not measuring the usage before proceeding further.

I think it would be useful to have some examples of when it is necessary to measure the usage, and a few examples of when it is not needed.

Paweł

Re: [blink-dev] Feature deprecation: measuring usage, and notifying developers. Adam Barth 4/29/13 10:05 AM
On Mon, Apr 29, 2013 at 9:51 AM, Paweł Hajdan, Jr. <phajd...@chromium.org> wrote:
On Fri, Apr 26, 2013 at 4:00 PM, Max Heinritz <m...@chromium.org> wrote:
My original intention was to have deprecations and feature removals go through the ordinary launch process, but after reading this thread I think it might be better to define distinct paths for new features vs deprecations. How does this sound:

How to Deprecate/Remove a Web Platform Feature

  1. Carefully consider whether you can jump straight to deprecation without first measuring the feature's usage in the wild.
    • If you'd rather measure first:
      • Add your feature to the UseCounter::Feature enum.
      • Add MeasureAs=[your enum value here] to the feature's IDL definition.*


It's not obvious (maybe it's just me), what are the criteria for measuring and not measuring the usage before proceeding further.

I think it would be useful to have some examples of when it is necessary to measure the usage, and a few examples of when it is not needed.

It comes down to a judgement call.  If we're sure it's not going to cause compatibility problems, then we don't need to measure.  If we're unsure, then measuring is probably a good idea.

Adam
Re: [blink-dev] Feature deprecation: measuring usage, and notifying developers. Dimitri 5/7/13 1:48 PM
I am okay if we try what Max proposed for a little bit. We can tweak as we go.

:DG<
Re: [blink-dev] Feature deprecation: measuring usage, and notifying developers. Eric Seidel 5/7/13 1:53 PM
This process is OK for big things (say one wanted to remove
XSLTProcessor for example).  But for small things (especially quirks
which are exclusive to webkit), I would encourage skipping this
process all together.

Controlling what we ship, is controlling what we commit to continuing
to support for the next N years.

Controlling what we remove is only about avoiding shipping Chrome
stable and having the whole world scream.  Hopefully Dev and Beta
already catch those sorts of cases anyway w/o needing much extra
process to avoid that. :)
Re: [blink-dev] Feature deprecation: measuring usage, and notifying developers. Max Heinritz 5/10/13 6:41 PM
Thanks for your comments! Just to close the loop -- I added the following to the Blink Project Page under https://sites.google.com/a/chromium.org/dev/blink?pli=1#TOC-Launch-Process:-Deprecation:

  1. Decide if your feature needs to go through this process.

  • You can skip 2-5 if you’re removing a trivial feature (e.g. a quirk that is exclusive to WebKit, like RangeException).
  • Canary, Dev, and Beta users will complain if it breaks the web. But if your reviewer suggests going through the rest of this process, do it.
  1. Carefully consider whether you can jump straight to deprecation without first measuring the feature's usage in the wild.

  • If you're sure you're not going to cause compatibility problems, then you don't need to measure.  If you're unsure, then measuring is probably a good idea.
  • If you'd rather measure first:
    • Add your feature to the UseCounter::Feature enum.
    • Add MeasureAs=[your enum value here] to the feature's IDL definition.*
  1. Email blink-dev using the "Intent to Deprecate" template.


  1. Deprecate: notify developers and measure usage.

  • If you haven't, add your feature to the UseCounter::Feature enum.
  • Instrument your code by either:
    • Adding DeprecateAs=[your enum value here] to the feature's IDL definition.*^ See window.performance.webkitGetEntries, for example.
    • Adding a call to UseCounter::countDeprecation somewhere relevant (as we did for the prefixed Content Security Policy headers).
  • Notify developers by adding a clever deprecation message to the big switch in UseCounter::deprecationMessage. This will print a deprecation warning in the DevTools console.
  1. Email blink-dev using the "Intent to Remove" template. Wait for LGTMs from ≥3 API OWNERS.


  1. Remove.

^ DeprecateAs is intended to replace MeasureAs in the IDL file. Specifying both is redundant. 

* It takes 12-18 weeks to hit Stable once you enable instrumentation.

See this blink-dev thread for more information on deprecation.