Intent to Deprecate: Shadow DOM v0

[Hayato Ito]

Primary eng (and PM) emails

hay...@chromium.org

Summary

Deprecate all Shadow DOM v0 APIs.

Note that /deep/ combinators and multiple shadow roots were already deprecated. This "intent to deprecate" is aiming to deprecate Shadow DOM v0 entirely.

Motivation

Blink shipped Shadow DOM v1 at M53. Shadow DOM v1 has been stable enough since then. It would be good to stat to deprecate old Shadow DOM v0 APIs now so that we can encourage web developers to switch from v0 to v1.

Regarding code health, it has been hard to maintain both v0 and v1 because they have totally different node distribution systems in Blink. That makes distribution lifecycle complex, and prevents me from optimizing code. If we could remove v0 code, that would give us a good amount of opportunity to reduce the complexity.

Compatibility Risk

Only Blink supports Shadow DOM v0. Other browsers are getting to support Shadow DOM v1.

Alternative implementation suggestion for web developers

Use Shadow DOM v1 for interoperability.

See http://hayato.io/2016/shadowdomv1/ as a guide to switch from v0 to v1.

Usage information from UseCounter

ElementCreateShadowRoot: 2.2%

https://www.chromestatus.com/metrics/feature/timeline/popularity/456

Though we have not isolated the usage numbers, AdBlock is one of the big users of Shadow DOM v0 APIs, as far as I know.

Entry on chromestatus.com

https://www.chromestatus.com/features/4507242028072960

Requesting approval to remove too?

No

[Philip Jägenstedt]

Thanks for driving this, it's important to find a path to removal for Shadow DOM v0, as otherwise the APIs may be used by accident or deliberately in a way that could eventually put pressure on other to implement v0 as well.

Usage here is very high. Lately we have preferred to set a removal date at deprecation time and include that in the deprecation message. Is that feasible in this case, what would need to happen before attempting a removal?

[Rune Lillesveen]

On Thu, Nov 24, 2016 at 7:18 AM, Hayato Ito <hay...@chromium.org> wrote:
> Primary eng (and PM) emails
> hay...@chromium.org
>
> Summary
> Deprecate all Shadow DOM v0 APIs.

\o/

> Regarding code health, it has been hard to maintain both v0 and v1 because
> they have totally different node distribution systems in Blink. That makes
> distribution lifecycle complex, and prevents me from optimizing code. If we
> could remove v0 code, that would give us a good amount of opportunity to
> reduce the complexity.

This goes for the style code as well. We can remove a bunch of
complexity and code once v0 support is no longer needed.

--
Rune Lillesveen

[Hayato Ito]

On Thu, Nov 24, 2016 at 6:14 PM Philip Jägenstedt <foo...@chromium.org> wrote:

Thanks for driving this, it's important to find a path to removal for Shadow DOM v0, as otherwise the APIs may be used by accident or deliberately in a way that could eventually put pressure on other to implement v0 as well.

Usage here is very high. Lately we have preferred to set a removal date at deprecation time and include that in the deprecation message. Is that feasible in this case, what would need to happen before attempting a removal?

Thanks. Including a removal date would be a good idea to decrease the usage. I can include a removal date in the Shadow DOM v0 deprecation message, but is that a commitment to remove? For example, if the usage is still high enough at the removal date, we would remove it unconditionally? If so, I have to be careful to set a removal date, one year later or more than it.

What need to happen before attempting a removal is:

- Chrome/Blink itself has to remove the internal usage of Shadow DOM v0. I guess devtools and setting pages are still using Shadow DOM v0 (please correct me if I am wrong). They have to rewrite it if they are still using v0.

- Regarding Polymer, Polymer has not used native Shadow DOM v0 by default recently. I do not worry much about it.

- Maybe there are other users which use Shadow DOM v0, outside of Blink, like an AdBlock Extension, ATOM Editor (which is removing Shadow DOM). They have to stop using v0.

Starting to deprecation would reveal hidden users, and encourage them to switch to v1, I expect.

As of now, I can not think of any better way to decrease the usage of Shadow DOM v0, other than deprecating it (with a removal date).

 

[Philip Jägenstedt]

On Thu, Nov 24, 2016 at 12:10 PM Hayato Ito <hay...@chromium.org> wrote:

On Thu, Nov 24, 2016 at 6:14 PM Philip Jägenstedt <foo...@chromium.org> wrote:

Thanks for driving this, it's important to find a path to removal for Shadow DOM v0, as otherwise the APIs may be used by accident or deliberately in a way that could eventually put pressure on other to implement v0 as well.

Usage here is very high. Lately we have preferred to set a removal date at deprecation time and include that in the deprecation message. Is that feasible in this case, what would need to happen before attempting a removal?

Thanks. Including a removal date would be a good idea to decrease the usage. I can include a removal date in the Shadow DOM v0 deprecation message, but is that a commitment to remove? For example, if the usage is still high enough at the removal date, we would remove it unconditionally? If so, I have to be careful to set a removal date, one year later or more than it.

What need to happen before attempting a removal is:

- Chrome/Blink itself has to remove the internal usage of Shadow DOM v0. I guess devtools and setting pages are still using Shadow DOM v0 (please correct me if I am wrong). They have to rewrite it if they are still using v0.

- Regarding Polymer, Polymer has not used native Shadow DOM v0 by default recently. I do not worry much about it.

- Maybe there are other users which use Shadow DOM v0, outside of Blink, like an AdBlock Extension, ATOM Editor (which is removing Shadow DOM). They have to stop using v0.

Starting to deprecation would reveal hidden users, and encourage them to switch to v1, I expect.

As of now, I can not think of any better way to decrease the usage of Shadow DOM v0, other than deprecating it (with a removal date).

There are indeed calls to createShadowRoot in chrome/, extensions/ and third_party/WebKit/Source/devtools/. UseCounter::countIfNotPrivateScript is used, but I wouldn't assume that all of these cases are excluded, so usage could be inflated by internal usage. Rewriting all of those would be a good first step.

Beyond that, if the true usage is anywhere near 2%, then it's probably used on some pretty big sites. As an experiment, can you make it crash and browse some of http://www.alexa.com/topsites? With any luck, fixing just one case will cause usage to drop sharply.

There's also httparchive, where there's >100k hits for 'createShadowRoot'. It's a lot of work, but categorizing those results should tell you what the problematic usage out there is, and give you a list of sites to reach out to. Happy to provide some tips off-list.

When usage is this high, I think you're right that just deprecating will reduce usage, but if anything like 0.1% would currently be broken by removal and if it's in the long tail of sites, then I wouldn't expect deprecation alone to drive down the breaking usage to a significant degree.

Could you begin by looking into top sites and httparchive to understand usage? It could be that removal is already entirely safe, in which case we should have a short deprecation period. Or it could be more depressing, that a lot of sites would have to change. I think it's still an option to deprecate and clearly communicating that the date is for an attempted removal, if no other path seems available.

[Rick Byers]

On Thu, Nov 24, 2016 at 6:39 AM, Philip Jägenstedt <foo...@chromium.org> wrote:

On Thu, Nov 24, 2016 at 12:10 PM Hayato Ito <hay...@chromium.org> wrote:

On Thu, Nov 24, 2016 at 6:14 PM Philip Jägenstedt <foo...@chromium.org> wrote:

Thanks for driving this, it's important to find a path to removal for Shadow DOM v0, as otherwise the APIs may be used by accident or deliberately in a way that could eventually put pressure on other to implement v0 as well.

Usage here is very high. Lately we have preferred to set a removal date at deprecation time and include that in the deprecation message. Is that feasible in this case, what would need to happen before attempting a removal?

Thanks. Including a removal date would be a good idea to decrease the usage. I can include a removal date in the Shadow DOM v0 deprecation message, but is that a commitment to remove? For example, if the usage is still high enough at the removal date, we would remove it unconditionally? If so, I have to be careful to set a removal date, one year later or more than it.

What need to happen before attempting a removal is:

- Chrome/Blink itself has to remove the internal usage of Shadow DOM v0. I guess devtools and setting pages are still using Shadow DOM v0 (please correct me if I am wrong). They have to rewrite it if they are still using v0.

- Regarding Polymer, Polymer has not used native Shadow DOM v0 by default recently. I do not worry much about it.

- Maybe there are other users which use Shadow DOM v0, outside of Blink, like an AdBlock Extension, ATOM Editor (which is removing Shadow DOM). They have to stop using v0.

Starting to deprecation would reveal hidden users, and encourage them to switch to v1, I expect.

As of now, I can not think of any better way to decrease the usage of Shadow DOM v0, other than deprecating it (with a removal date).

There are indeed calls to createShadowRoot in chrome/, extensions/ and third_party/WebKit/Source/devtools/. UseCounter::countIfNotPrivateScript is used, but I wouldn't assume that all of these cases are excluded, so usage could be inflated by internal usage. Rewriting all of those would be a good first step.

+1.  We should always clean up our own house before asking others to do so ;-) 

Beyond that, if the true usage is anywhere near 2%, then it's probably used on some pretty big sites. As an experiment, can you make it crash and browse some of http://www.alexa.com/topsites? With any luck, fixing just one case will cause usage to drop sharply.

There's also httparchive, where there's >100k hits for 'createShadowRoot'. It's a lot of work, but categorizing those results should tell you what the problematic usage out there is, and give you a list of sites to reach out to. Happy to provide some tips off-list.

If static analysis is proving problematic here, note that we should very soon be able to query use counters for the HTTPArchive data.

When usage is this high, I think you're right that just deprecating will reduce usage, but if anything like 0.1% would currently be broken by removal and if it's in the long tail of sites, then I wouldn't expect deprecation alone to drive down the breaking usage to a significant degree.

Yeah, and I'd worry that introducing a deprecation warning on 2% of all page views would cause more harm (in terms of warning blindness) than benefit (in terms of actually reducing usage by the ~2 orders of magnitude necessary to remove it). 

[Hayato Ito]

Thank you for the advice. I appreciate it. Let me follow the advice to figure out what is causing 2.2% at first.

I agree that 2.2% is too high to deprecate. We would hesitate to deprecate it because that would spam the console. People are likely to ignore the spam.

Given the usecounter chart (https://www.chromestatus.com/metrics/feature/timeline/popularity/456), I think 2.2% number is coming from a limited big sources. There are two big spikes in the chart; Sep 2014 and Jun 2016.

Regarding a spike at Jun 2016, it could be caused by twitter (https://twittercommunity.com/t/upcoming-change-to-embedded-tweet-display-on-web/66215). But this is just a guess. I have not tried to figure out the details of the usage so far.

It sounds that there is no silver bullets to analyze the details. Let me start with http://www.alenxa.com/topsites and httparchive at first.

I have one question; Does a usecounter include a usage from a chrome extension? I am afraid that AdBlock Plus occupies the majority of the number.

[Rick Byers]

On Fri, Nov 25, 2016 at 2:11 AM, Hayato Ito <hay...@chromium.org> wrote:

Thank you for the advice. I appreciate it. Let me follow the advice to figure out what is causing 2.2% at first.

I agree that 2.2% is too high to deprecate. We would hesitate to deprecate it because that would spam the console. People are likely to ignore the spam.

Given the usecounter chart (https://www.chromestatus.com/metrics/feature/timeline/popularity/456), I think 2.2% number is coming from a limited big sources. There are two big spikes in the chart; Sep 2014 and Jun 2016.

Regarding a spike at Jun 2016, it could be caused by twitter (https://twittercommunity.com/t/upcoming-change-to-embedded-tweet-display-on-web/66215). But this is just a guess. I have not tried to figure out the details of the usage so far.

It sounds that there is no silver bullets to analyze the details. Let me start with http://www.alenxa.com/topsites and httparchive at first.

I have one question; Does a usecounter include a usage from a chrome extension? I am afraid that AdBlock Plus occupies the majority of the number.

Yes, I believe they do today.  I'm working to eliminate cases like chrome-extension://.  These are still a type of compat risk of course (could break a lot of page views for users that have popular extensions installed) but I argue that the risk is of an entirely different form (which we shouldn't conflate with "web compat risk") because the user can uninstall the extension, popular extensions would likely be updated quickly, etc.

The harder case is extensions that inject code into pages.  I don't think there's anything practical to do about those - just continue treating them the same as code on the page.  Do you know if AdBlock Plus does that?

Another way you can get some idea of where the usage is coming from is by slicing the UMA data along different axes.  Eg. here's (internal only - sorry) a breakdown of the metric by OS and low and behold we see ~2.5% on windows but 0.015% on Android!  That is some strong evidence supporting your AdBlock Plus theory ;-)

​

[PhistucK]

On Fri, Nov 25, 2016 at 6:16 PM, Rick Byers <rby...@chromium.org> wrote:

The harder case is extensions that inject code into pages.  I don't think there's anything practical to do about those - just continue treating them the same as code on the page.  Do you know if AdBlock Plus does that?

​Unless the code for the injected script comes from an external source, you can just query the web store and notify the authors.

☆PhistucK

[Philip Jägenstedt]

On Fri, Nov 25, 2016 at 5:17 PM Rick Byers <rby...@chromium.org> wrote:

On Fri, Nov 25, 2016 at 2:11 AM, Hayato Ito <hay...@chromium.org> wrote:

Thank you for the advice. I appreciate it. Let me follow the advice to figure out what is causing 2.2% at first.

I agree that 2.2% is too high to deprecate. We would hesitate to deprecate it because that would spam the console. People are likely to ignore the spam.

Given the usecounter chart (https://www.chromestatus.com/metrics/feature/timeline/popularity/456), I think 2.2% number is coming from a limited big sources. There are two big spikes in the chart; Sep 2014 and Jun 2016.

Regarding a spike at Jun 2016, it could be caused by twitter (https://twittercommunity.com/t/upcoming-change-to-embedded-tweet-display-on-web/66215). But this is just a guess. I have not tried to figure out the details of the usage so far.

It sounds that there is no silver bullets to analyze the details. Let me start with http://www.alenxa.com/topsites and httparchive at first.

I have one question; Does a usecounter include a usage from a chrome extension? I am afraid that AdBlock Plus occupies the majority of the number.

Yes, I believe they do today.  I'm working to eliminate cases like chrome-extension://.  These are still a type of compat risk of course (could break a lot of page views for users that have popular extensions installed) but I argue that the risk is of an entirely different form (which we shouldn't conflate with "web compat risk") because the user can uninstall the extension, popular extensions would likely be updated quickly, etc.

The harder case is extensions that inject code into pages.  I don't think there's anything practical to do about those - just continue treating them the same as code on the page.  Do you know if AdBlock Plus does that?

Another way you can get some idea of where the usage is coming from is by slicing the UMA data along different axes.  Eg. here's (internal only - sorry) a breakdown of the metric by OS and low and behold we see ~2.5% on windows but 0.015% on Android!  That is some strong evidence supporting your AdBlock Plus theory ;-)

​

That's a pretty striking difference, thanks for checking Rick!

Looks like the source code for ABP is available, so you can probably just submit a pull request:

https://github.com/adblockplus/adblockpluschrome/blob/918892096eb249c49c73e32fd73b111fc5217c28/include.preload.js#L506

The code is also in AdBlock in a slightly different form, and they seem to accept patches:

http://code.getadblock.com/

These extensions, even if fixed, will probably taint the data for a long time, so it would be worth investigating if there's any way of detecting this, an excluding from the metrics. Otherwise use counters won't be very useful.

[Rick Byers]

Yeah, these are in content scripts so my fix of excluding chrome-extension:// pages won't help.  Definitely you should file issues / PRs to switch to ShadowDOMv1.  Extensions can be updated quickly, so perhaps we could get them changed?

Alternately perhaps you could add some custom code (maybe custom binding) for a use counter which counts createShadowRoot cases where the calling JS is in a script without a chrome-extension:// URL.  I'm guessing we wouldn't want that complexity/overhead for ALL use counters, but adding such a check specifically for createShadowRoot would probably be practical, right?

[Hayato Ito]

Thank you for the comments. I really appreciate that.

>  we see ~2.5% on windows but 0.015% on Android! 

That would be very helpful information! Thanks!

It looks the next actions for me are:

- I will not deprecate Shadow DOM v0 unless we have a clear (hopefully) evidence for what caused the high usage number.

- File issues / PRs to switch to ShadowDOMv1 for AdBlock Plus / AdBlock.

- In addition to that, try other ways to figure out the cause, as you suggested.

Let me update this thread once I figure out something new.

[Dan Beam]

On Thu, Nov 24, 2016 at 3:10 AM, Hayato Ito <hay...@chromium.org> wrote:

On Thu, Nov 24, 2016 at 6:14 PM Philip Jägenstedt <foo...@chromium.org> wrote:

Thanks for driving this, it's important to find a path to removal for Shadow DOM v0, as otherwise the APIs may be used by accident or deliberately in a way that could eventually put pressure on other to implement v0 as well.

Usage here is very high. Lately we have preferred to set a removal date at deprecation time and include that in the deprecation message. Is that feasible in this case, what would need to happen before attempting a removal?

Thanks. Including a removal date would be a good idea to decrease the usage. I can include a removal date in the Shadow DOM v0 deprecation message, but is that a commitment to remove? For example, if the usage is still high enough at the removal date, we would remove it unconditionally? If so, I have to be careful to set a removal date, one year later or more than it.

What need to happen before attempting a removal is:

- Chrome/Blink itself has to remove the internal usage of Shadow DOM v0. I guess devtools and setting pages are still using Shadow DOM v0 (please correct me if I am wrong). They have to rewrite it if they are still using v0.

Downloads, history, settings, extensions, and Chrome's PDF UI are still using Shadow DOM v0 (via Polymer 1).

-- Dan

[Philip Jägenstedt]

Thanks, better understanding the usage would be great.

I would like to add, though, that we don't need to wait until usage actually drops before going ahead with deprecation and removal, we just need to have a fairly good idea that the removal will work out. So, if AdBlock (Plus) is very slow to update, but httparchive shows very low risk in web content, I think it's fine to go ahead. Console spam isn't great, but leaving Shadow DOM v0 around is also not without risk.

[da...@adblockplus.org]

On Tuesday, November 29, 2016 at 10:34:53 AM UTC, Philip Jägenstedt wrote:

I would like to add, though, that we don't need to wait until usage actually drops before going ahead with deprecation and removal, we just need to have a fairly good idea that the removal will work out. So, if AdBlock (Plus) is very slow to update, but httparchive shows very low risk in web content, I think it's fine to go ahead. Console spam isn't great, but leaving Shadow DOM v0 around is also not without risk.

Please do not remove the Shadow DOM v0 APIs without first addressing Chromium issue #632009. We do not use the deprecated API out of choice, and would be very happy to migrate away if there was a viable alternative. (Also see the discussion in our issue tracker.)

Thanks, Dave.