Blog posts on the motivations for webRequest changes

[Simeon Vincent]

Hey extensions devs,

I wanted to give this distro list a heads up that we shared two blog posts this morning that attempt to address some of the recent confusion in the press around the webRequest API changes.

Improving Security and Privacy for Extensions Users

https://security.googleblog.com/2019/06/improving-security-and-privacy-for.html

Web Request and Declarative Net Request: Explaining the impact on Extensions in Manifest V3

https://blog.chromium.org/2019/06/web-request-and-declarative-net-request.html

Happy coding!

Simeon - @dotproto

Extensions Developer Advocate

[Luke Greib]

Pretty poor attempt at damage control, if I say so myself. The privacy claim makes absolutely zero sense. As you've repeatedly stated, only the blocking webRequest API is being targeted for removal, yet the observational webRequest remains fully intact and extensions will still be able to read all your requests. And again, I see no numbers or empirical evidence to support the performance claims, in contrast to the defenders of blocking webRequest, who have demonstrated a very minimal performance impact.

[Bennett Cyphers]

> Additionally, we are currently planning to change the rule limit from maximum of 30k rules per extension to a global maximum of 150k rules.

I might have missed an update, but can you clarify what this means? Is the maximum per extension being raised to 150k, or will each browser have a maximum of 150k rules distributed among all extensions? For example, if HTTPS Everywhere requires 100k rules, and AdBlock Plus requires another 60k rules, will a user have to choose between installing one or the other?

Best,

Bennett

[Rajaram Gaunker]

Where can we test latest manifest v3 extension?. Is it available on chromium master branch?. 

[Daniel Glazman]

Hi Simeon,

I still feel this is not the right way to do things and I still think this harms the industry, and my own company with it. The 150k limit will be hit by adblockers, tracker blockers and other legit tools quite fast. Hard-wired limits of that kind feel like the hard-wired limits of CSS rules in Internet Explorer 6.

I would do differently: I would create a new permission called "adblocking", restricted to extensions performing real blocking w/o any undesirable data capturing. That permission would preserve webRequest and/or "open" declarativeNetNequest to extra methods for dynamic list changes and more.

</Daniel>

[Andrey Meshkov]

Hi Simeon,

150k rules limit is better than 30k indeed, but not enough.

Also, what bothers me is that you're talking about the "static" rules limit.

The dynamic rules limit is more important to content blockers.

We are interested in increasing the dynamic rules limit which is ridiculously low now.

Otherwise, you will hurt not just some content blockers, but the ad blocking community.

Let me please explain what's the problem.

1. Content blockers work is based on using different filter lists which are created by volunteers. 
2. Filter lists generally have different purposes, and users are free to choose which of them to use.
3. Here are some examples of the popular filter lists:
• EasyList, AdGuard Base filter, Peter Lowe's list -- ad blocking filter lists
• EasyPrivacy, AdGuard Tracking Protection filter -- blocking trackers and such
• Fanboy's Social Blocking list, AdGuard Social Media filter -- blocking all kinds of social widgets
• Fanboy's Annoyances, "I don't care about cookies", AdGuard Annoyances filter -- cookie notices, sign up pop-ups, etc
• There are many more lists, just take a look here: https://filterlists.com

Static rules do not allow having lots of different filter lists. You are forcing content blockers to have a single pre-defined filter list, and this not just greatly limits the functionality but also hurts the huge community of people who maintain all these lists.

Q: But filter lists maintainers will be able to easily create their own extensions instead?

A: Actually, they won't, and here's why:

• Cosmetic rules isn’t a part of this new API
• The lists will be completely independent, so:
• You won’t be able to override a rule from a different list. For instance, it's often necessary to override some rules from EasyList.
• Whitelisting a website will be the real hell. You’ll need to disable each one of these extensions.
• Debugging lists will be quite hard for the same reason.

Here's another question - why setting the limit to 150k? The current ad blocking engines implementations work equally fast with 150k or 300k rules, it does not matter. So what is the logic behind setting the limit to 150k? As I explained in another thread, the artificial limitations of this kind will not help with the overall performance.

To summarize:

• The current draft will hurt the content blocking community. We need the dynamic rules limit to be the same as the static rules limit.
• The new limit is better than it was before, but this is still less than what's needed.

--

Regards,

Andrey Meshkov

CTO at AdGuard

[Vasco Gomes]

Hi Simeon, first of all, thanks for the posts.

I have a simple question and an appeal. 

Will this scenario be possible on Manifest V3?

Things like change User-Agent or remove some headers like x-frame-options or content-security policy? Because I have read that some sensitive headers will not be possible to change/remove.

Example:

const requestFilter = {

 urls: ['*://*.example.com/*']

}

const userAgent = 'Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25';

chrome.webRequest.onBeforeSendHeaders.addListener(

 function(details) {

   if (details.tabId && (details.tabId === tabId || details.tabId === -1 || tabMultiId.includes(details.tabId))) {

     const { requestHeaders } = details

     requestHeaders.forEach(header => {

       if (header.name === 'User-Agent') {

         header.value = userAgent

       }

     })

     const headers = requestHeaders.filter(header => {

       return header.name.toLowerCase() !== 'referer'

     })

     return { requestHeaders: headers }

   }

 },

 requestFilter,

 ['blocking', 'requestHeaders']

)

chrome.webRequest.onHeadersReceived.addListener(

 function(details) {

   const toBeRemoved = [

     'x-frame-options',     'content-security-policy-report-only',     'content-security-policy'

   ]

   if (details.tabId && (details.tabId === tabId || details.tabId === -1 || tabMultiId.includes(details.tabId))) {

     const headers = details.responseHeaders.filter(header => {

       return toBeRemoved.indexOf(header.name.toLowerCase()) === -1

     })

     return {

       responseHeaders: headers

     }

   }

 },

 requestFilter,

 ['blocking', 'responseHeaders']

)

Besides this question I have also an appeal:

If this simple operation will not be possible on the new Manifest V3 It will cripple and cease to function thousands of Extensions. And the most relevant ones like Email, Social Media silent notifications counter on the App icon in the browser toolbar.

I make here an appeal to Chrome team to keep at least some basic Webrequest functionality like the previously showed and also the Background pages. Background pages are not only used because of WebRequest but also because they store information, they have a full Html and Javascript page where we can store an Iframe for example. And none of this have is related to sensitive content, it's just how programmers program to make things work.

Much of today Chrome 60% market share was made because of Extensions. Please don't make a step back and cripple extensions functionality. 

Android never did this, they kept adding more and more features, not removing. Computer power doubles every 2 years way go back 10 years in time?

For the uses cases that Google is referring in the documentation you can go the way Android did, that is the creation of "Lite" Apps. Google, Facebook, Twitter, ... all have gone this route. They provide the full standalone Apps and the Lite Apps.

If Chrome want to go Lite, please open a Lite section on the Chrome Web Store and force this functionalities there. And Developers will build or not the Full and the Lite App just like Android did.

Opera, Vivaldi and Brave already said that they will not follow this path and will keep to support this functionalities.

Remember that IE had 90% of browser market share and now holds 5%. History tend to repeat itself. Learn with Android success.

My final appeal is don't cripple extensions. Add more features, make them more powerful and open a Lite section on the Chrome WebStore. Let be the users to choose, like they did on Android, and don't force Devs and users to go back 10 year in time. 

If I (user) want the Lite App, I will Install it. But don't force me (user) to go that way if I don't wan't to.

Greetings.

[wOxxOm]

There's no confusion in the press.

The only "confusion" is the one that Devlin Cronin, and the rest of the ManifestV3 proponents, are inventing by implying everyone is concerned about total removing of the webRequest API. This is just total bull. Really, everyone who's concerned with the issue understands the problem is the removal of the blocking ability of webRequest, which will destroy lots of valid use cases, not just innovative ad-blockers.

Dodging the issue this way is really pathetic or worse, it's like the abovementioned Chromium developers think everyone's a moron. They continue to tell us about gains in security or performance, but so far they didn't provide any proof the change will be meaningful in real world use cases. They show pretty graphs and how it's good for performance to avoid inter-process serialization, but they don't provide any real measurements. The real world experience tells us they are talking about negligible gains like milliseconds, whereas the adblockers and privacy-enhancers provide gains like half a second or even multiple seconds. I wonder how separated one must be from the reality in order not to see the difference.

On Thursday, June 13, 2019 at 1:36:17 AM UTC+3, Simeon Vincent wrote:

[Dave Teare]

Thanks for sharing, Simeon. At the risk of getting flamed here I'll go out on a limb and say I enjoyed reading your post.

Change is never easy but as a user I appreciate that Chrome is tightening up on what extensions can do. It's causing more work for extension devs, myself included, but I'm liking a lot of what I'm seeing in manifest v3.

++dave;

On Wednesday, June 12, 2019 at 6:36:17 PM UTC-4, Simeon Vincent wrote:

[John Doe]

found the shill bot. meanwhile everyone else is not using Chrome.

[wOxxOm]

Looks like a typical clueless user who doesn't understand the issue and blindly trusts the official announcements. All of these announcements have been targeting such users so far (obviously an intentional move) by presenting buzzwords like security wrapped in an authoritatively sounding corporate fluff. There wasn't a single announcement with real data and measurements that confirm the official claims. On the contrary, there are contradictions and nonsensical exaggerations, pointed out in various comments.

[Dave Teare]

Tough crowd. 

[Chris Palmer]

Indeed. Let's please all keep in mind Chromium's Code Of Conduct. Name-calling and abuse are not constructive and are not going to help us move forward on this tricky problem which, necessarily, has multiple contesting equities.

On Sun, Jun 16, 2019 at 3:21 PM Dave Teare <da...@agilebits.com> wrote:

Tough crowd. 

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/fd29e938-f6f9-4f6d-9291-d0dc819ec80c%40chromium.org.

[wOxxOm]

Oh, please. The crowd here is just sane, not willing to applaud every official announcement blindly. The Chromium developers have been ignoring all criticism (or appeals to justify the official claims) towards the base of the ManifestV3 plan regardless of the form it's expressed and regardless of how well it's founded or worded, which is a huge "F. you" in the face of all extension developers who understand what's about to be broken with no alternative for no clear benefit other than vague promises.

On Wednesday, June 19, 2019 at 1:39:22 AM UTC+3, Chris Palmer wrote:

Indeed. Let's please all keep in mind Chromium's Code Of Conduct. Name-calling and abuse are not constructive and are not going to help us move forward on this tricky problem which, necessarily, has multiple contesting equities.

On Sun, Jun 16, 2019 at 3:21 PM Dave Teare <da...@agilebits.com> wrote:

Tough crowd. 

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.

[Chris Palmer]

The security claims are not vague; we have stated them clearly and repeatedly. Similarly, cross-process communication (see the diagram in the post) is typically much more expensive than a pure-compute operation in-process, even when N is large-ish. (As people may have seen, Chrome engineers are working to show this, with a realistic test ad blocker using Mv3 APIs.) The performance concern is real (more on this below).

The down-side is that those N rules must now be stored in a long-lived browser or network service process, so the cost in space of those N rules becomes a concern. (This is likely also why Safari has a limit — lower than the limit we now propose, thanks to people's feedback here. Yet nobody could reasonably say that Apple is trying to get rid of content blocking extensions.)

Finally, nobody is getting rid of content blockers. In fact, as one of the blog posts says, "These performance improvements will make extensions significantly more viable on resource-constrained platforms." Our goal is to make content blockers work faster, on more platforms, while at the same time requiring fewer powerful permissions — permissions that we know are being abused.

All these points have been made before. And we hear the concerns of extension developers that Manifest v3 might not suit all their goals perfectly. Thank you to everyone who is posting specific code examples and use cases, and doing so respectfully. That's what we all need. We all — the many Chromium embedders, extension developers, and web developers — are working together in a complex ecosystem to provide everyone in the world a great experience on the web. That's profoundly difficult, and there are legitimate differences as to what is best, but we all can make and are making good forward progress.

One last note: I sense there may be some legitimate confusion about the overloaded meaning of the term "blocking", and that confusion may have partially driven some of the concerns we've seen in these threads and in the press. "Blocking" sometimes means "stopping content from loading", but at other times means "synchronous". The "blocking" in "blocking WebRequest" is talking about synchronous I/O. It is a general goal of web platform developers to get rid of synchronous operations, because the caller has to wait for the callee to respond — that's slow, and we want things to be fast. When we say we want to get rid of the blocking WebRequest API, we mean getting rid of synchronous I/O, not getting rid of content blocking.

On Tue, Jun 18, 2019 at 9:33 PM wOxxOm <wox...@gmail.com> wrote:

Oh, please. The crowd here is just sane, not willing to applaud every official announcement blindly. The Chromium developers have been ignoring all criticism (or appeals to justify the official claims) towards the base of the ManifestV3 plan regardless of the form it's expressed and regardless of how well it's founded or worded, which is a huge "F. you" in the face of all extension developers who understand what's about to be broken with no alternative for no clear benefit other than vague promises.

On Wednesday, June 19, 2019 at 1:39:22 AM UTC+3, Chris Palmer wrote:

Indeed. Let's please all keep in mind Chromium's Code Of Conduct. Name-calling and abuse are not constructive and are not going to help us move forward on this tricky problem which, necessarily, has multiple contesting equities.

On Sun, Jun 16, 2019 at 3:21 PM Dave Teare <da...@agilebits.com> wrote:

Tough crowd. 

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/fd29e938-f6f9-4f6d-9291-d0dc819ec80c%40chromium.org.

--

You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/3bd5fa54-8c0c-48d2-b166-4b7b0ae0d8bb%40chromium.org.

[Luke Greib]

Once again, you've dodged the point that extensions are free to passively monitor web requests and page content without restraint, and that removing blocking webRequest does absolutely nothing to increase privacy in that regard. And again, more vague generalizations and platitudes regarding performance and how everything synchronous is automatically terrible and slow. How slow? My guess is that if you were up front about performance gains, no one would be willing to hand over their choice in how content can be blocked over to Google in exchange for those minimal gains. And why are we suddenly so concerned about performance and low resource devices? Planning to bring extensions and ad blocking to Chrome on Android?  

On Wednesday, June 19, 2019 at 2:45:13 PM UTC-4, Chris Palmer wrote:

The security claims are not vague; we have stated them clearly and repeatedly. Similarly, cross-process communication (see the diagram in the post) is typically much more expensive than a pure-compute operation in-process, even when N is large-ish. (As people may have seen, Chrome engineers are working to show this, with a realistic test ad blocker using Mv3 APIs.) The performance concern is real (more on this below).

The down-side is that those N rules must now be stored in a long-lived browser or network service process, so the cost in space of those N rules becomes a concern. (This is likely also why Safari has a limit — lower than the limit we now propose, thanks to people's feedback here. Yet nobody could reasonably say that Apple is trying to get rid of content blocking extensions.)

Finally, nobody is getting rid of content blockers. In fact, as one of the blog posts says, "These performance improvements will make extensions significantly more viable on resource-constrained platforms." Our goal is to make content blockers work faster, on more platforms, while at the same time requiring fewer powerful permissions — permissions that we know are being abused.

All these points have been made before. And we hear the concerns of extension developers that Manifest v3 might not suit all their goals perfectly. Thank you to everyone who is posting specific code examples and use cases, and doing so respectfully. That's what we all need. We all — the many Chromium embedders, extension developers, and web developers — are working together in a complex ecosystem to provide everyone in the world a great experience on the web. That's profoundly difficult, and there are legitimate differences as to what is best, but we all can make and are making good forward progress.

One last note: I sense there may be some legitimate confusion about the overloaded meaning of the term "blocking", and that confusion may have partially driven some of the concerns we've seen in these threads and in the press. "Blocking" sometimes means "stopping content from loading", but at other times means "synchronous". The "blocking" in "blocking WebRequest" is talking about synchronous I/O. It is a general goal of web platform developers to get rid of synchronous operations, because the caller has to wait for the callee to respond — that's slow, and we want things to be fast. When we say we want to get rid of the blocking WebRequest API, we mean getting rid of synchronous I/O, not getting rid of content blocking.

On Tue, Jun 18, 2019 at 9:33 PM wOxxOm <wox...@gmail.com> wrote:

Oh, please. The crowd here is just sane, not willing to applaud every official announcement blindly. The Chromium developers have been ignoring all criticism (or appeals to justify the official claims) towards the base of the ManifestV3 plan regardless of the form it's expressed and regardless of how well it's founded or worded, which is a huge "F. you" in the face of all extension developers who understand what's about to be broken with no alternative for no clear benefit other than vague promises.

On Wednesday, June 19, 2019 at 1:39:22 AM UTC+3, Chris Palmer wrote:

Indeed. Let's please all keep in mind Chromium's Code Of Conduct. Name-calling and abuse are not constructive and are not going to help us move forward on this tricky problem which, necessarily, has multiple contesting equities.

On Sun, Jun 16, 2019 at 3:21 PM Dave Teare <da...@agilebits.com> wrote:

Tough crowd. 

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/fd29e938-f6f9-4f6d-9291-d0dc819ec80c%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.

[Chris Palmer]

On Wed, Jun 19, 2019 at 12:15 PM Luke Greib <lgr...@gmail.com> wrote:

Once again, you've dodged the point that extensions are free to passively monitor web requests and page content without restraint, and that removing blocking webRequest does absolutely nothing to increase privacy in that regard.

The point is that someone could write an extension using Declarative Net Request to block content without necessarily getting the power to passively monitor requests and page content.

The extension might use additional APIs that require host permissions (i.e. that full power). But note that we're also changing how users grant that, to make it more explicit and to give users the ability to control the extension better, if they want. This has been described in the blog posts.

[wOxxOm]

At least it's certainly good to have a direct acknowledgment finally, this is important in a real dialog, and an attempted clarification also is a step in the right direction. Way to go. Here's hoping for a properly technical and in-depth announcement for extension developers that really proves why it's necessary to kill those parts of extensions architecture mentioned in the ManifestV3 draft.

On Wednesday, June 19, 2019 at 9:45:13 PM UTC+3, Chris Palmer wrote:

The security claims are not vague; we have stated them clearly and repeatedly. Similarly, cross-process communication (see the diagram in the post) is typically much more expensive than a pure-compute operation in-process, even when N is large-ish. (As people may have seen, Chrome engineers are working to show this, with a realistic test ad blocker using Mv3 APIs.) The performance concern is real (more on this below).

The down-side is that those N rules must now be stored in a long-lived browser or network service process, so the cost in space of those N rules becomes a concern. (This is likely also why Safari has a limit — lower than the limit we now propose, thanks to people's feedback here. Yet nobody could reasonably say that Apple is trying to get rid of content blocking extensions.)

Finally, nobody is getting rid of content blockers. In fact, as one of the blog posts says, "These performance improvements will make extensions significantly more viable on resource-constrained platforms." Our goal is to make content blockers work faster, on more platforms, while at the same time requiring fewer powerful permissions — permissions that we know are being abused.

All these points have been made before. And we hear the concerns of extension developers that Manifest v3 might not suit all their goals perfectly. Thank you to everyone who is posting specific code examples and use cases, and doing so respectfully. That's what we all need. We all — the many Chromium embedders, extension developers, and web developers — are working together in a complex ecosystem to provide everyone in the world a great experience on the web. That's profoundly difficult, and there are legitimate differences as to what is best, but we all can make and are making good forward progress.

One last note: I sense there may be some legitimate confusion about the overloaded meaning of the term "blocking", and that confusion may have partially driven some of the concerns we've seen in these threads and in the press. "Blocking" sometimes means "stopping content from loading", but at other times means "synchronous". The "blocking" in "blocking WebRequest" is talking about synchronous I/O. It is a general goal of web platform developers to get rid of synchronous operations, because the caller has to wait for the callee to respond — that's slow, and we want things to be fast. When we say we want to get rid of the blocking WebRequest API, we mean getting rid of synchronous I/O, not getting rid of content blocking.

On Tue, Jun 18, 2019 at 9:33 PM wOxxOm <wox...@gmail.com> wrote:

Oh, please. The crowd here is just sane, not willing to applaud every official announcement blindly. The Chromium developers have been ignoring all criticism (or appeals to justify the official claims) towards the base of the ManifestV3 plan regardless of the form it's expressed and regardless of how well it's founded or worded, which is a huge "F. you" in the face of all extension developers who understand what's about to be broken with no alternative for no clear benefit other than vague promises.

On Wednesday, June 19, 2019 at 1:39:22 AM UTC+3, Chris Palmer wrote:

Indeed. Let's please all keep in mind Chromium's Code Of Conduct. Name-calling and abuse are not constructive and are not going to help us move forward on this tricky problem which, necessarily, has multiple contesting equities.

On Sun, Jun 16, 2019 at 3:21 PM Dave Teare <da...@agilebits.com> wrote:

Tough crowd. 

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/fd29e938-f6f9-4f6d-9291-d0dc819ec80c%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.

[Jeremy Gillula]

Chris, could you provide some links to those security claims? The best I could find is "Since January 2018, 42% of malicious extensions use the Web Request API." But that number has absolutely no context. What percentage of all extensions use the Web Request API? What percentage of extensions that use it are bad? How many bad extensions are we talking about? How many users does this affect? How does this compare to the number of users who rely on privacy-protecting extensions like uBlock, NoScript, Privacy Badger, and PhishDetect?

Further, why is the change to host permissions insufficient to address the security concerns?

Right now, Chrome's extension platform lets developers innovate and protect users in ways that Chrome by itself doesn't--in ways the Chrome team never even imagined, and might even disagree with. Mv3 will flip that, so that extension developers can only protect users in ways the Chrome team has approved (and built out the capability for). So far, the Chrome team has failed to explain why such a fundamental change is necessary.

 - Jeremy

On Wednesday, June 19, 2019 at 11:45:13 AM UTC-7, Chris Palmer wrote:

The security claims are not vague; we have stated them clearly and repeatedly. Similarly, cross-process communication (see the diagram in the post) is typically much more expensive than a pure-compute operation in-process, even when N is large-ish. (As people may have seen, Chrome engineers are working to show this, with a realistic test ad blocker using Mv3 APIs.) The performance concern is real (more on this below).

The down-side is that those N rules must now be stored in a long-lived browser or network service process, so the cost in space of those N rules becomes a concern. (This is likely also why Safari has a limit — lower than the limit we now propose, thanks to people's feedback here. Yet nobody could reasonably say that Apple is trying to get rid of content blocking extensions.)

Finally, nobody is getting rid of content blockers. In fact, as one of the blog posts says, "These performance improvements will make extensions significantly more viable on resource-constrained platforms." Our goal is to make content blockers work faster, on more platforms, while at the same time requiring fewer powerful permissions — permissions that we know are being abused.

All these points have been made before. And we hear the concerns of extension developers that Manifest v3 might not suit all their goals perfectly. Thank you to everyone who is posting specific code examples and use cases, and doing so respectfully. That's what we all need. We all — the many Chromium embedders, extension developers, and web developers — are working together in a complex ecosystem to provide everyone in the world a great experience on the web. That's profoundly difficult, and there are legitimate differences as to what is best, but we all can make and are making good forward progress.

One last note: I sense there may be some legitimate confusion about the overloaded meaning of the term "blocking", and that confusion may have partially driven some of the concerns we've seen in these threads and in the press. "Blocking" sometimes means "stopping content from loading", but at other times means "synchronous". The "blocking" in "blocking WebRequest" is talking about synchronous I/O. It is a general goal of web platform developers to get rid of synchronous operations, because the caller has to wait for the callee to respond — that's slow, and we want things to be fast. When we say we want to get rid of the blocking WebRequest API, we mean getting rid of synchronous I/O, not getting rid of content blocking.

On Tue, Jun 18, 2019 at 9:33 PM wOxxOm <wox...@gmail.com> wrote:

Oh, please. The crowd here is just sane, not willing to applaud every official announcement blindly. The Chromium developers have been ignoring all criticism (or appeals to justify the official claims) towards the base of the ManifestV3 plan regardless of the form it's expressed and regardless of how well it's founded or worded, which is a huge "F. you" in the face of all extension developers who understand what's about to be broken with no alternative for no clear benefit other than vague promises.

On Wednesday, June 19, 2019 at 1:39:22 AM UTC+3, Chris Palmer wrote:

Indeed. Let's please all keep in mind Chromium's Code Of Conduct. Name-calling and abuse are not constructive and are not going to help us move forward on this tricky problem which, necessarily, has multiple contesting equities.

On Sun, Jun 16, 2019 at 3:21 PM Dave Teare <da...@agilebits.com> wrote:

Tough crowd. 

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/fd29e938-f6f9-4f6d-9291-d0dc819ec80c%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.

[Luke Greib]

If an extension doesn't require advanced request manipulation, blocking or feedback and can get away with using DNR, then great, they should have that option. The problem is putting up DNR as a replacement for blocking webRequest when it clearly isn't.

[Chris Palmer]

On Wed, Jun 19, 2019 at 1:05 PM Jeremy Gillula <jer...@eff.org> wrote:

Chris, could you provide some links to those security claims? The best I could find is "Since January 2018, 42% of malicious extensions use the Web Request API." But that number has absolutely no context. What percentage of all extensions use the Web Request API? What percentage of extensions that use it are bad? How many bad extensions are we talking about? How many users does this affect? How does this compare to the number of users who rely on privacy-protecting extensions like uBlock, NoScript, Privacy Badger, and PhishDetect?

Simeon is better positioned to answer that. (I am on the Chrome Platform Security team, not the Extensions team.)

Further, why is the change to host permissions insufficient to address the security concerns?

Right now, Chrome's extension platform lets developers innovate and protect users in ways that Chrome by itself doesn't--in ways the Chrome team never even imagined, and might even disagree with. Mv3 will flip that, so that extension developers can only protect users in ways the Chrome team has approved (and built out the capability for). So far, the Chrome team has failed to explain why such a fundamental change is necessary.

The principle of least privilege is sound and we want to increasingly adhere to it. (Both for extension APIs and Open Web Platform APIs.) DNR is much more in line with the principle, and so we think it's a huge step in the right direction.

Permission prompts and grants for all-powerful APIs are nice, but not as good as precluding the possibility of certain kinds of abuse. Making the permission grant UX more explicit and limit-able is good in its own way, but adhering to the principle of least privilege is typically much more effective and usable. Especially at scale. Hoping that developers will always be as well-meaning as the fine folks here, and that users will always understand the full implications of install-time privilege grants, has proven not to work all that well.

There will still be a lot of possibilities for experimentation and innovation. (And, what you can do with DNR is likely to expand, in response to these threads.) Some of that innovation is likely to require using additional APIs, possibly in concert with DNR, and those APIs might require the user to grant more privilege.

Designing a platform that enables the most good and the least harm is likely to require an on-going dialogue and further iteration.

[Andrey Meshkov]

Chris,

Should we be preparing to the complete removal of the content scripts or webRequest in manifest V4?

From what I am reading it seems that DNR is just a first step in limiting extensions capabilities, and you are not going to stop there.

[Chris Palmer]

On Thu, Jun 20, 2019 at 2:00 AM Andrey Meshkov <ay.me...@gmail.com> wrote:

Should we be preparing to the complete removal of the content scripts or webRequest in manifest V4?

From what I am reading it seems that DNR is just a first step in limiting extensions capabilities, and you are not going to stop there.

As a platform security person and not an extensions team member, I have no idea.

[William Budington]

HTTPS Everywhere developer here. I wanted to address the impact of mv3 on our security-focused extension. In the blog post, you mention:

> We are actively exploring other ways to expand this API, including adding methods to get feedback about matched rules, and support for richer redirects leveraging URL manipulation and regular expressions. Additionally, we are currently planning to change the rule limit from maximum of 30k rules per extension to a global maximum of 150k rules.

Currently, HTTPS Everywhere services 157,156 targets.  A target consists of a FQDN or wildcarded subdomain.  This list has been culled to remove entries that are already on the HSTS preload list, which last I checked contains about 50,000 target domains and subdomains.  A single ruleset, the Bitly branded short-domains, contains 34,917 HTTP->HTTPS redirection rules.  These rules are both bundled with the extension as well as delivered dynamically (via https://www.https-rulesets.org), and verified using the WebCrypto API with a signing key which is pinned to the extension.  Dynamic delivery is necessary to quickly fix problems and reduces deployment overhead to a constant-time operation rather than linear with the number of browsers we deploy to.

150,000 rules shared with whatever ad-blocking or tracker-blocking addon a user has installed would be absolutely insufficient for our requirements if a separate declaration is required for each FQDN/wildcarded domain.

The HSTS preload list is on its own insufficient to comprehensively protect against sslstrip-style attacks.  The webRequests API has allowed us to provide this greater degree of protection to, by the Chrome Web Store estimates, over 2 million Chrome users.  I seriously hope there will be a way for us to continue to provide these protections to your users under mv3.

[Simeon Vincent]

Well dang. I was in the process of drafting a response to questions posed here in chronological order, but things got a bit lively here over the past two days. I think the below answers provide some useful context, so I'll post them. I'll have to think more about whether this pattern makes sense given the number of posts we've gotten & the answers Chris has provided.

The privacy claim makes absolutely zero sense. As you've repeatedly stated, only the blocking webRequest API is being targeted for removal, yet the observational webRequest remains fully intact and extensions will still be able to read all your requests.

The Manifest V3 effort aims to improve the security and privacy guarantees of the extensions platform. Today, the only way for an extension to block content is with the webRequest API, the webRequestBlocking permissions, and host permissions for all domains you wish to block content on. These APIs and permissions expose a massive amount of data and capabilities to the extension; data and capabilities that the extension may neither want or need.

In rethinking how we approach the extensions platform, one of the goals we're trying to achieve is to enable extensions to do what they do today, but in ways that don't compromise the user's privacy. Broadly speaking, the declarativeNetRequest (DNR) API fits into this mold by giving extensions a way to modify requests without exposing the request object to the extension or (as much as possible) without requiring the extension to have host permissions to modify requests. In other words, a hypothetical content blocker implemented in Manifest V3 could deliver the same results as its Manifest V2 counterpart but with no risk of data exfiltration.

Now you may be asking "why keep webRequest (+ host permissions) if it's so dangerous?" Two reasons. First, because it gives extensions an important capabilities that can't be replicated with a more secure API. Second, the risks around webRequest can be mitigated by increasing the friction of getting (and keeping) host permissions.

This latter point is especially important as this is core to how we're approaching other aspects of the Manifest V3 plan. The host permissions changes described in the "User controls for host permissions" section of the Trustworthy Chrome Extensions, by default blog post and the associated Transition Guide are a key part of this. We're also looking for ways to make activeTab-style permissions more common. This particular change is a step towards improving security, not the end goal. 

And again, I see no numbers or empirical evidence to support the performance claims, in contrast to the defenders of blocking webRequest, who have demonstrated a very minimal performance impact

We originally planned to go into more detail on the performance angle but had to trim that for time reasons. I'm hoping we can dig more into this angle in another post soon.

Briefly, the primary performance aspect the community has discussed is the execution time of blocking webRequest callbacks, but this isn't what the extension team is concerned about. Rather, the team is looking at the entire cost of processing a request in Chrome. With the blocking version of webRequest, Chrome has to serialize a lot of data (the request itself and request metadata, etc.) and send it to each extension. This results in significant costs from type conversions, copies, interprocess communication, (de)serialization, etc. for each listening extension (which are usually in separate processes). 

Unfortunately this overhead applies to all blocking webRequest extensions. Even an extension with an entirely trivial listener that executes in negligible time can slow down Chrome because of these costs.  And, needless to say, the situation only gets worse with poorly crafted extensions.

All that said, I want to re-emphasize that performance is not the primary motivator for the webRequest API changes. Rather, this was motivated by a desire to improve the privacy and security of request modification and to reduce the attack surface for malicious extensions. The extensions team did not realize the potential performance improvements until after they started exploring this path.

Where can we test latest manifest v3 extension?. Is it available on chromium master branch?. 

Manifest V3 is not yet ready for experimentation and feedback. The extensions team is currently working towards releasing a Developer Preview in the Canary channel at the end of July or beginning of August. We'll be sharing additional details when that lands.

The truly brave among you can manually build the master branch of the Chromium repository. Please note that this path comes with no guarantees, guidance, or support. You'll likely need to read through source in order to figure out what's currently implemented and how to use it. You have been warned.

I might have missed an update, but can you clarify what this means? Is the maximum per extension being raised to 150k, or will each browser have a maximum of 150k rules distributed among all extensions? For example, if HTTPS Everywhere requires 100k rules, and AdBlock Plus requires another 60k rules, will a user have to choose between installing one or the other?

The latter; Chrome will have a global maximum of 150k rules shared among all extensions. Please note that this limit is likely not final. We are still analyzing what the real-world performance cost is at different ruleset sizes. As before, developers should only expect this limit to remain the same or increase.

The extensions team believes the global limit provides a better balance over per-extension limits, since we don't need to find a limit that works well for both users with one extension and users with N extensions. This gives developers and users more flexibility to decide how best to use the resource pool.

I should also note that this is very early and the team is still exploring the APIs required to support this pattern. We'll be sharing more details and soliciting feedback soon(ish).

Simeon - @dotproto

Extensions Developer Advocate

[Luke Greib]

The Manifest V3 effort aims to improve the security and privacy guarantees of the extensions platform. Today, the only way for an extension to block content is with the webRequest API, the webRequestBlocking permissions, and host permissions for all domains you wish to block content on. These APIs and permissions expose a massive amount of data and capabilities to the extension; data and capabilities that the extension may neither want or need.

In rethinking how we approach the extensions platform, one of the goals we're trying to achieve is to enable extensions to do what they do today, but in ways that don't compromise the user's privacy. Broadly speaking, the declarativeNetRequest (DNR) API fits into this mold by giving extensions a way to modify requests without exposing the request object to the extension or (as much as possible) without requiring the extension to have host permissions to modify requests. In other words, a hypothetical content blocker implemented in Manifest V3 could deliver the same results as its Manifest V2 counterpart but with no risk of data exfiltration.

Now you may be asking "why keep webRequest (+ host permissions) if it's so dangerous?" Two reasons. First, because it gives extensions an important capabilities that can't be replicated with a more secure API. Second, the risks around webRequest can be mitigated by increasing the friction of getting (and keeping) host permissions.

Bolded is exactly the issue - webRequest simply can't be replaced with DNR. No matter how many use cases you try to shoehorn into DNR (and turning it into a complex, bloated API in the process), it will always be inferior to the flexibility of blocking webRequest. Everyone agrees DNR should be an option. Almost no one agrees that it should be the only option.

[Simeon Vincent]

> Now you may be asking "why keep webRequest (+ host permissions) if it's so dangerous?" Two reasons. First, because it gives extensions an important capabilities that can't be replicated with a more secure API. Second, the risks around webRequest can be mitigated by increasing the friction of getting (and keeping) host permissions.

Bolded is exactly the issue - webRequest simply can't be replaced with DNR. No matter how many use cases you try to shoehorn into DNR (and turning it into a complex, bloated API in the process), it will always be inferior to the flexibility of blocking webRequest.

I fear you misinterpreted my statement. The capabilities of the observable version of webRequest can't be replicated with a more secure API (or at least I don't see a clear way to), but the blocking version of webRequest can be. The declarativeNetRequest API is that secure alternative.

As you implied, a declarative API can't realistically do everything an imperative one can. If I understand your point of view correctly, you see this as a loss of capabilities and adaptability. From the Chrome team's perspective this change and others combine to close off an entire class of exploits. DNR isn't a perfect tradeoff, but we strongly believe the privacy guarantees it brings outweigh the loss of capabilities. That said, we want to enable content blockers and request modifying extensions as much as we can without exposing too much data or empowering malicious actors.

This is a tough balance to strike. We need community help to identify use cases that DNR doesn't currently support so that we can iterate on DNR's capabilities and consider other ways to accomplish developers' goals.  

Everyone agrees DNR should be an option. Almost no one agrees that it should be the only option.

I mentioned this in my Chromium blog post, but we've seen a consistent pattern where developers strongly prefer capabilities-focused APIs even long after more secure or performant alternatives are introduced. But even in an ideal world where good actors voluntarily give up capabilities, malicious actors will use whatever tools they have at their disposal.

Simeon - @dotproto

Extensions Developer Advocate

[Mark James]

It's the old argument: "Does restricting our freedom mean that the criminals win?"

I think the first line of defence should be establishing trust between a user and a developer on install through an engaging automated dialog of question and short explanation, with the browser-maker participating as a guide so that the important points of security and privacy are raised and understood.

Trusted extensions should then be given powerful capabilities like mobile apps.

[Theodore Dubois]

uBlock origin does a number of things that don't work with the current API. The biggest one is having block rules with higher priority take precedence over allow rules with lower priority. uBlock origin allows users to interactively set per-origin and per-domain rules through the popup, and those rules take priority over rules in static lists. The control is tri-state: either allow regardless of static rules, block regardless of static rules, or defer to static rules. The API as currently written will allow a request if any rule at any priority allows it. (Note that uBlock origin would maintain all rules including "static" rules as Chrome dynamic rules so that they can be customized by the user, so it doesn't help that Chrome static rules are lower priority than Chrome dynamic rules.) The algorithm would have to work like this:

Find the highest priority rule that matches this request

Find all other rules that match this request with the same priority

If any rule found allows this request, return ALLOW

If any rule found block this request, return BLOCK

If any rule found redirects this request, return REDIRECT

If multiple extensions are installed, this would have to be run for each extension to determine whether that extension wants to block the request.

If priorities worked this way, uBlock origin could then create rules from the popup at a higher priority than "static" rules, and they would behave as expected.

uBlock origin also supports marking static rules as "important" which gives them precedence over all other static rules. If priorities worked this way, this could be implemented by giving these rules a higher priority than other static rules.

[Developer Developer Developer Developer]

You should be deeply ashamed for working at such an evil company.

On Thursday, 13 June 2019 06:36:17 UTC+8, Simeon Vincent wrote:

Hey extensions devs,

I wanted to give this distro list a heads up that we shared two blog posts this morning that attempt to address some of the recent confusion in the press around the webRequest API changes.

Improving Security and Privacy for Extensions Users

https://security.googleblog.com/2019/06/improving-security-and-privacy-for.html

Web Request and Declarative Net Request: Explaining the impact on Extensions in Manifest V3

https://blog.chromium.org/2019/06/web-request-and-declarative-net-request.html

Happy coding!

[Chris Palmer]

Let's keep this respectful and constructive. Thanks.

--

You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/a4194013-c796-4b53-bd46-b203fdc89273%40chromium.org.

[Vasco Gomes]

Just received this newsletter from Websummit 2019:

Spotify Lite for Android gets official launch in 36 countries

Tinder Lite Is Launching In Vietnam — Here’s What’s Different About It

Why doesn't Chrome go Lite? Create a Lite section in the Chrome WebStore.

Instead of cripple Extensions already in place in the WebStore and go back 10 years in time give both Developers and Users the option to choose. 

Lite or Full. And Impose the new restrictions on the Lite section.

Just dont cripple the most important and thousands of extensions int he Web Store. Otherwise you may be killing the WebStore.

Android went this way. Just follow Android steps.

Greetings.

[Kent Brewster]

On Thursday, June 20, 2019 at 7:11:22 PM UTC-7, Simeon Vincent wrote:

Where can we test latest manifest v3 extension?. Is it available on chromium master branch?. 

Manifest V3 is not yet ready for experimentation and feedback. The extensions team is currently working towards releasing a Developer Preview in the Canary channel at the end of July or beginning of August. We'll be sharing additional details when that lands.

Did this happen with v78? If not, can you please say more about where and how you will be sharing additional details when it does?

--Kent

[Simeon Vincent]

We talked a bit on Twitter, but I'm going to re-answer here for the sake of addressing a wider audience.

The Developer Preview hasn't launched yet. To my knowledge it will land in Canary in the v78 timeline. That said, I'll be syncing with the team later today to catch up on the current status of the dev preview work. Updates will be shared on this mailing list. 

Simeon - @dotproto

Extensions Developer Advocate