All new/removed web APIs guarded by flags

[Rick Byers]

Hey folks,

In thinking about post-mortem for the window.Report fire, I recalled Chromium's flag guarding guidelines. I had forgotten that this says:

When is a flag required?

• ...
• Every deprecation/addition of web platform APIs

In practice I think this means that we want every new API to be at least RuntimeEnabled with a base::Feature which can be used as a killswitch (hmm text fragment URLs don't work here - search for "kill switch"). This was communicated internally within Google, but (unless I missed it), not publicly. I think it's generally trivial to keep new APIs as status=stable [RuntimeEnabled] for a milestone, and likely pretty common already. How do y'all feel about communicating this broadly on blink-dev and API owners keeping our eyes out to help remind folks about this policy?

Note that in this specific case of window.Report I'm not sure how practical it is to make the [LegacyNoInterfaceObject] behavior RuntimeEnabled (might have required bindings generator changes), but let's keep that discussion to the other thread.

I've updated Chrome's release process in our compat principles to indicate that we can't rely on beta channel as much as we'd like, but that we should be relying on the availability of kill switches. Current text is below, any feedback?

Thanks,

   Rick

We depend on Chrome’s release process (finch and canary, dev, beta, stable releases) and Blink’s reliable bug triaging to reduce the cost of mistakes, and allow us to still make breaking change decisions in the face of uncertainty. In particular, if a user or developer (or two) files a bug about a breaking change before that change has hit stable, that probably means the change has a high risk of being disruptive (expect ~100x the pain when the change hits stable). Unfortunately the inverse is not true, just because a change causes no reports of issues in beta, it still may cause a major problem when it reaches stable - especially for environments like enterprises which rarely have any beta user deployments. Increasingly it’s important that changes with any real risk be guarded by flags where possible so that they can be disabled in the field without requiring a respin.

[Yoav Weiss]

LGTM to communicate this to the broader blink-dev audience! I completely missed that risk angle, and I think the cost of adding a runtime flag is not huge in most cases.

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAFUtAY_E0kKT5KV0B7dGv4oiWmb6aYjB6768_ya5qDYCENv92A%40mail.gmail.com.

[Mike Taylor]

+1

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAL5BFfVJ1Q7bJaZQTHMXyq28v-aTJW6U8KJWg1ZZj2t8U8EpHw%40mail.gmail.com.

[Mike West]

Does this mean that every runtime feature should generate a base::Feature as well (e.g. making something like https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/RuntimeEnabledFeatures.md#:~:text=Generate%20a%20base%3A%3AFeature%20instance%20from%20a%20Blink%20Feature happen by default)?

-mike

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAKNNSBmeGtp5jj5ia7iJ8DmjKYjcqMV3zsM3BT9RXXU-ss9tsQ%40mail.gmail.com.

[David Baron]

On Wed, Feb 15, 2023 at 9:46 AM Mike West <mk...@chromium.org> wrote:

Does this mean that every runtime feature should generate a base::Feature as well (e.g. making something like https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/RuntimeEnabledFeatures.md#:~:text=Generate%20a%20base%3A%3AFeature%20instance%20from%20a%20Blink%20Feature happen by default)?

For what it's worth, an unrelated (I think) discussion yesterday came to that conclusion (that RuntimeEnabledFeatures should generate a base::Feature by default) and led to crbug.com/1416399 being filed, although I'm not sure if there's a plan to do it.

-David

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAKXHy%3Ddpw4M96%2BAkN2kw9%2BYpnF6fegg%3D_BE--ormRsu2rRvvhQ%40mail.gmail.com.

[Johnny Stenback]

I'm not an API owner but I strongly support this change as well. I will also add that there's been a discussion ongoing about requesting insight into flag availability (including the flag name) in chromestatus.com feature entries so we can help encourage engineers to think about this early in the process, and track where a flag was in place vs not. I'll continue to push for that change to increase the visibility into this, and help encourage more things shipping with a kill switch in place.

Cheers,

Johnny

- jstenback (he/him)

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAG0MU3j4h0qRA9YDKK3P%2BK0FxACJNnKH73mMO-Oi-D42A2QsRQ%40mail.gmail.com.

[Ian Clelland]

One thing that's come to mind here that I'd like to understand better is the effect of corporate and enterprise policies on this, as window.Report was a specifically-enterprise-breaking situation.

Anecdotally, I've heard that enterprise managers often disable Finch, likely for the same reasons that they'd disable UMA/UKM sharing, and I'm concerned that if that were the case, that a Finch kill switch wouldn't have been effective here.

A Finch kill switch might still work, as long as enterprise policy managers have the ability to enable/disable arbitrary Finch experiments across their fleet. I'm just not sure if that's generally true, or if it would require additional work (for every API) to link the finchable flag to an enterprise policy item. Or whether that additional step also implies, for instance, translations, extra testing load, or a commitment to maintaining a policy item for some length of time. (Not that we couldn't do that, but it might change the effort-to-reward calculations)

Ian

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CACZRgz6o-K6%3DXutBqBjeNiRKwG80xHn0YYiGOOB%2BmXaSwzkJXQ%40mail.gmail.com.

[Rick Byers]

On Wed, Feb 15, 2023 at 9:56 AM David Baron <dba...@chromium.org> wrote:

On Wed, Feb 15, 2023 at 9:46 AM Mike West <mk...@chromium.org> wrote:

Does this mean that every runtime feature should generate a base::Feature as well (e.g. making something like https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/RuntimeEnabledFeatures.md#:~:text=Generate%20a%20base%3A%3AFeature%20instance%20from%20a%20Blink%20Feature happen by default)?

For what it's worth, an unrelated (I think) discussion yesterday came to that conclusion (that RuntimeEnabledFeatures should generate a base::Feature by default) and led to crbug.com/1416399 being filed, although I'm not sure if there's a plan to do it.

Cool! I wondered about this but thought there might be a name collision risk. We should be able to find a way to make that an error though. Can you summarize what the other reason for this was (or share with me internally if necessary)? 

I don't think we have an owner for this anymore. If nobody else volunteers I'll just do it myself before sending the blink-dev e-mail.

[Chris Harrelson]

On Wed, Feb 15, 2023 at 11:59 AM 'Rick Byers' via blink-api-owners-discuss <blink-api-ow...@chromium.org> wrote:

On Wed, Feb 15, 2023 at 9:56 AM David Baron <dba...@chromium.org> wrote:

On Wed, Feb 15, 2023 at 9:46 AM Mike West <mk...@chromium.org> wrote:

Does this mean that every runtime feature should generate a base::Feature as well (e.g. making something like https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/RuntimeEnabledFeatures.md#:~:text=Generate%20a%20base%3A%3AFeature%20instance%20from%20a%20Blink%20Feature happen by default)?

For what it's worth, an unrelated (I think) discussion yesterday came to that conclusion (that RuntimeEnabledFeatures should generate a base::Feature by default) and led to crbug.com/1416399 being filed, although I'm not sure if there's a plan to do it.

Cool! I wondered about this but thought there might be a name collision risk. We should be able to find a way to make that an error though. Can you summarize what the other reason for this was (or share with me internally if necessary)? 

It was about avoiding accidentally forgetting to have base features associated with RuntimeEnabledFeatures.

 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAFUtAY8CVuXa1ODgOzH0OMRaT5kH4ii3qmCiDsPowa98sMo9%2Bg%40mail.gmail.com.

[John Abd-El-Malek]

Thanks all, ensuring we can undo site-visible changes via Finch for most users is a big improvement :)

@Ian: my understanding is that most enterprises do not disable Finch/UMA. Of course there are some that do, but at least with a policy like this we can still help most users recover faster.

[Rick Byers]

I found the docs for the enterprise policy here. Critical fixes (presumably kill-switches) is a separate and highly discouraged option from variations generally. So hopefully that's good enough. Enterprises can always just enable the critical fix option so I don't see a particular argument for requiring a feature-specific policy unless we know there is a particular compat risk.

[Rick Byers]

+Brandon Heenan who is OOO this week. Brandon let me know if you agree that in most regular risk cases it should be sufficient for enterprise risk reduction to have web platform API changes behind a simple finch kill switch. Obviously we should continue with the guidelines for changes identified as potentially risky for enterprises, but we're talking here about how we communicate expectations around ALL web platform changes.

Thanks,

   Rick

[Brandon Heenan]

> most enterprises do not disable Finch/UMA.

Most enterprises have UMA disabled (and it's disabled by default with the enterprise installer), which means we have to be cautious about interpreting data from UMA for enterprises.

That said, according to UMA, very few enterprises disable Finch (or even set it to critical only). That seems to line up with 1:1 conversations I've had with enterprises as well.

Rick, I agree with what you said there. Requiring a bespoke enterprise policy for every new API feels like a bad tradeoff (unless we could make doing that super easy?). I'd still recommend them for API removals and breaking changes to their behavior, but the Report API doesn't fall into those categories.

[Rick Byers]

On Thu, Feb 16, 2023 at 7:28 AM Brandon Heenan <bhe...@google.com> wrote:

> most enterprises do not disable Finch/UMA.

Most enterprises have UMA disabled (and it's disabled by default with the enterprise installer), which means we have to be cautious about interpreting data from UMA for enterprises.

We used to say we were blind to enterprises in UMA, but based on some conversations and this analysis, I updated our compat guidelines with this sentence implying that we could still trust that we get some UMA data from enterprises. Perhaps we should tweak the wording to be more nuanced? Feel free to suggest edits. Note this list and the doc are public - a resource for API owners who aren't all Googlers. 

That said, according to UMA, very few enterprises disable Finch (or even set it to critical only). That seems to line up with 1:1 conversations I've had with enterprises as well.

Excellent, thanks!

Rick, I agree with what you said there. Requiring a bespoke enterprise policy for every new API feels like a bad tradeoff (unless we could make doing that super easy?). I'd still recommend them for API removals and breaking changes to their behavior, but the Report API doesn't fall into those categories.

Thanks! We'll focus on enforcing the use of a base::Feature (killswitch) for every platform change possible but continue to keep our eyes out for cases where having a custom policy knob would have been better. Presumably we will have some cases where we use the killswitch to undo a change for everyone, then add an enterprise policy knob in the next release in order to re-enable the behavior by default. That seems like a reasonable and relatively low-cost emergency compat handling path to me.

[John Abd-El-Malek]

On Thu, Feb 16, 2023 at 7:24 AM Rick Byers <rby...@google.com> wrote:

On Thu, Feb 16, 2023 at 7:28 AM Brandon Heenan <bhe...@google.com> wrote:

> most enterprises do not disable Finch/UMA.

Most enterprises have UMA disabled (and it's disabled by default with the enterprise installer), which means we have to be cautious about interpreting data from UMA for enterprises.

Thanks, I confused Finch not being disabled with UMA not being disabled as well. Good to know!

[Yoav Weiss]

And +Kent Tamura just made it so that Chromium devs only need to add a runtime enabled flag in order to make their feature Finchable!

[John Abd-El-Malek]

Thank you Kent!