A few questions & answers regarding MV3

[Simeon Vincent]

Hey all,

A handful of questions related to MV3 recently landed in my lap. I figured the answers would be interesting to the broader community, so I figured I'd share more info here. If you have additional questions feel free to drop them in this thread and I'll do my best to answer.

Are you planning to implement alternatives to force the state to be persistent or to use storage APIs synchronously?

We are not planning to provide any other background execution environment or to make the current chrome.storage APIs synchronous. There is a feature request to create a new storage API that would provide an in-memory storage API for critical persistent data (crbug.com/930235) .

With regards to synchronicity, I want to highlight that all message passing between content scripts and the background context are asynchronous. While managing asynchronous data can cause a bit of a headache, we're planning to implement promise based APIs as part of MV3. Promises in combination with async functions make it possible to work with asynchronous logic in an imperative manner.

Will there be an option to keep using Manifest v2 after end-of-life? (e.q. running chrome with the flag)

A final decision has not been made, but speaking from a personal point of view I wouldn't be surprised if we had something like this during a transition period. I believe we did something similar during the Manifest V2 migration. This would primarily be exposed to aid in enterprise environments. Generally speaking end users should not be manually configuring their Chrome flags. Also note that CWS will require that extension developers upload MV3 versions of their extensions before Chrome stops loading them.

When can we expect to have a version that is stable to test it?

Manifest V3 has been available for testing in Chrome Canary since October 31st, 2019. Once we have enough of the Manifest V3 platform implemented and it's solid enough for public release, it will be made available in Chrome Stable. We do not have a firm date, but we expect this to happen in 2020.

How much time can we expect to have to migrate to v3 after the end-of-life will be announced?

I expect at least at least a 1 year transition period between when MV3 lands in Stable and when MV2 reaches EOL. 

"Like other listeners, alarm listeners should be registered in the top level of your script."

Are you planning to remove setTimeout completely?

No. Extension service workers are based on the open web's service workers, and those service workers can still access setTimeout. The quoted section of the migration guide is attempting to highlight that setTimeout may not work as expected in a service worker environment due to their ephemeral design. If the service worker is terminated before the timeout expires, the timer will be dropped and the callback will never be called.

Are you planning to add an alternative that would allow us to make heavy logic asynchronous in another way?

As of yet we haven't seen a demonstrated need for this capability and therefore have no plans on this front. If you have a use case where the current serviceWorker implementation is demonstrably insufficient, please open a feature request on crbug.com to discuss.

What will be the minimum alarm trigger time?

Yes, alarms require a minimum amount of time to pass. Per the alarms API documentation:

In order to reduce the load on the user's machine, Chrome limits alarms to at most once every 1 minute but may delay them an arbitrary amount more. That is, setting delayInMinutes or periodInMinutes to less than 1 will not be honored and will cause a warning. when can be set to less than 1 minute after "now" without warning but won't actually cause the alarm to fire for at least 1 minute.

Are you planning to completely remove blocking webRequest APIs?

The observational capabilities of webRequest will remain in Manifest V3, but the introduction of declarativeNetRequest and planned changes to host permissions will significantly reduce the number of extensions that require the webRequest permission.

We are planning to disallow blocking webRequest in consumer extensions. Extensions that have been force-installed in managed environments via policies such as ExtensionInstallForcelist will be able to access blocking webRequest. If your extensions may be used in both and you opt to continue using blocking webRequest in managed environments, you will either need to distribute different versions for each environment or add conditional logic to gracefully handle when this feature is not available.

Will there still be a way to modify headers or the response?

The declarativeNetRequest API will allow header modification via the modifyHeaders action. I'm not aware of any plans to support response body modification. If this is critical to your use case, please open an issue on crbug.com to discuss.

Cheers,

Simeon - @dotproto

Extensions Developer Advocate

[wOxxOm]

>  As of yet we haven't seen a demonstrated need for this capability and therefore have no plans on this front 

FWIW It was demonstrated multiple times both in these groups and on crbug. Not that I expect anyone in chromium to hear at this point seeing hundreds of bug reports on extension bugs and features being ignored for the last 5+ years due to a lack of programmers on this front (for comparison, the devtools component receives like, hundred times more meaningful development activity).

[PhistucK]

Is this not achievable using workers?

Or do you need to do this from within a service worker and because service workers cannot spawn more workers, this is not achievable?

If so, you should take it to the standards (WHATWG HTML).

☆PhistucK

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/167386c1-33b7-4f8c-a0e6-0672d9a609ddn%40chromium.org.

[Simon Bassu]

Are you working with Mozilla and Apple (new WebExtension member)?

Now, it's really convenient to not have to rewrite add-on for each browser.

And where is the documentation? This : https://developer.chrome.com/extensions/migrating_to_manifest_v3 is not a real documentation (to be polite).

Simon B.

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/CAFY0HLNMEGgUwkKnkg4_LzthKhPABZkEjv5MUYx7NXwH6WPJVA%40mail.gmail.com.

[Adi Bhaskar]

When will the non-blocking webRequest be available to test with service workers in MV3, or even event pages in MV2?

[wOxxOm]

> Is this not achievable using workers?

It was discussed already, also on crbug, at length, so I don't see the need to do it here as well. Also note that deferring to WHATWG essentially amounts to abandoning and betraying extension authors for years (quite probably a decade) doing hackish workarounds to implement something that was possible in MV2 by default.

[PhistucK]

Service workers are in active specification development, so it is a good opportunity to suggest use cases and issues, it might not take a long time at all (especially if you show demand).

You know the team is understaffed, so this is likely the faster option, unfortunately.

☆PhistucK

[Simeon Vincent]

>  As of yet we haven't seen a demonstrated need for this capability and therefore have no plans on this front

FWIW It was demonstrated multiple times both in these groups and on crbug. - wOxxOm 

While the desire for a persistent execution environment has been discussed several times, I don't believe that the need has been demonstrated. As I see it, there's still a gap between your concerns and the understanding of Chrome engineers. The issues that you've identified are currently theoretical and there may be other technical solutions that are better suited to address the underlying issues.

For example, one issue you've mentioned was that V8 doesn't cache regular expressions across executions, so re-initializing a service worker has additional overhead. A while back I reached out to a V8 team member to confirm. They agreed, but also indicated V8 might be able to directly address this.

That's why I used the word "demonstrated"; I believe the best way to bridge the gap is to have actual code (a working demonstration) that we can reference, discuss, and explorer. Having a concrete use case is incredibly helpful in figuring out what the actual problem is and how best we can address it.

Are you working with Mozilla and Apple (new WebExtension member)? - Simon

I've talked to a few different browser vendors about browser extensions over the past year, some more frequently than others, and am looking forward to continuing those conversations. Cross-browser compatibility is one of my major sources of frustration as an extension dev.

And where is the documentation? This https://developer.chrome.com/extensions/migrating_to_manifest_v3 is not a real documentation (to be polite). - Simon

Sorry, I don't follow the question. What would you like more documentation on? The more detailed you can be the better able I'll be able to answer or understand what needs to be created. 

When will the non-blocking webRequest be available to test with service workers in MV3, or even event pages in MV2?

You can already use webRequest in MV3 extensions on Canary. I just threw together & tested the following demo in Canary: https://gist.github.com/dotproto/1da2d4512685c8f6c7f8e2e83a2bbbae.

While it's not possible to use webRequest with an MV2 event page, you can use service workers and webRequest in an MV2 extension in Canary. 

Simeon - @dotproto

Extensions Developer Advocate

[wOxxOm]

>  That's why I used the word "demonstrated"; I believe the best way to bridge the gap is to have actual code (a working demonstration) 

I wasn't the only one who demonstrated the examples, just the more vocal one probably. Among many "theoretical" examples (that are used practically) I've named several popular extensions (Tampermonkey, Violentmonkey, Stylus of those I remember right now) that will be unable to work (or will be severely crippled) without a persistent background script environment or an ability to make it persistent on demand. Such an ability exists in MV3 though - an open messaging channel to a random content script - but it's almost a hack. The state these extensions need to maintain to ensure fast execution isn't limited to regular expressions so solving it in V8 won't help even if it happens in this decade. Anyway, like I said, at this point I don't see the point of these discussions anymore because no one in Chromium actually has time to do anything even if by a miracle they hear.

[PhistucK]

I thought that point was about 'Are you planning to add an alternative that would allow us to make heavy logic asynchronous in another way?", not about "Are you planning to implement alternatives to force the state to be persistent or to use storage APIs synchronously?"...

(Which is why I suggested workers)

A persistent execution environment could be an extension opening a tab (with a chrome-extension URL) and doing its things there, right? While it may seem a bit awkward to the user (a tab just standing there "doing nothing"), it is more transparent to the user.

The only thing that needs solving in this case is the slow background timers/execution/similar, but I am sure extensions can be special-cased (maybe with a certain permission/API call).

So I think there are solutions, but the user experience/perception might not be as nice as it was before (completely hidden).

☆PhistucK

[wOxxOm]

>  A persistent execution environment could be an extension opening a tab (with a chrome-extension URL) and doing its things there, right? While it may seem a bit awkward to the user (a tab just standing there "doing nothing"), it is more transparent to the user.  

It was also suggested and discussed. This is a very awkward idea indeed that will create a lot more problems than it solves, which were also discussed at length so I don't see the point of doing it here as well because like I said, nothing will change anyway.

[PhistucK]

You opened the discussion here by commenting about it, so you should expect people to respond.

You also did not link to an issue where I could figure that it was discussed, so repeating this without giving references is not very helpful ("no point").

And if "nothing" is going to "change anyway", why did you reply in the first place? This kind of comments are highly not constructive (or promote anything) and such negativity sometimes leads to the opposite response/action, just saying (they are still humans, so psychology works on them, too). If you just wanted to complain, I think you achieved your goal, though (not sure why others had to read it, though).

Sorry if I am being harsh.

☆PhistucK

[wOxxOm]

If you want to help you can find all those discussions and issues on crbug yourself, it's the easiest part of helping. I battled, contributed, helped, argued for improvements on extensions API for years on crbug and recently here. It's just after seeing such blatant attempts of neglecting all the previous concerns and user feedback, that I feel bitterness and express that. Someone should say out loud that the proverbial king is naked.

[PhistucK]

Again no references, why make people search? One of the easiest parts of advocating for a better state is to provide others with the right place to join the discussion.

(I will not join the discussion, but maybe others will)

You have expressed that bitterness multiple times a day in the past years, I think it is no longer achieving its goal apart from making you feel better and others looking at those as "here comes another rant" (I know I am).

☆PhistucK

[wOxxOm]

> Again no references, why make people search? One of the easiest parts of advocating for a better state is to provide others with the right place to join the discussion. (I will not join the discussion, but maybe others will) You have expressed that bitterness multiple times a day in the past years, I think it is no longer achieving its goal apart from making you feel better and others looking at those as "here comes another rant" (I know I am).

Out of thousands of posts and comments I made I think about 99% were referencing actual material and only 1% was expressing bitterness about being neglected. You seem to be bitter yourself now that I neglected your attempt to help which demonstrates your point that my behavior isn't constructive for which I'm sorry. My bad. I guess I should have some faith and hope for the better.

[PhistucK]

Just to clarify any misunderstanding - you are immensely helpful! You help a lot of people and you are very, very insightful. I see your comments on the tracker as well as on this group, your help is definitely noticed (at least by me, but who am I?) and - I am sure - also appreciated by the team.

At the same time, a lot of your helpful responses are accompanied by those bitter comments. Do not get me wrong, I understand your frustration and share it - but adding those bitter comments almost every time, in such a negative tone, turns them into unconstructive ranting that (in my opinion) deters from the goal.

☆PhistucK

[mur...@influesolutions.com]

Sorry for being a bit lost or naive on the subject, but isn't the current background script already an "persistent execution environment"? It's content is running for as long as the browser is open, and you can even use to localstorage to persist data for when the browser get closed and open later. What 'more' are people asking for? (this is a genuine question, i'm sure the problems my extensions have to solve are way simpler than what people are experiencing)

[PhistucK]

Manifest version 3 drops this feature. Extensions can only have a service worker in the background and it stops when idle.

☆PhistucK

[JohnnyCee]

Will MV3 support host messaging?

Will the host program be started and stopped in alignment with the service worker lifetime?

[PhistucK]

How does native messaging work today with event pages? I am guessing it would work the same way with service workers.

☆PhistucK

On Thu, Jul 16, 2020 at 5:35 PM JohnnyCee <jfcar...@gmail.com> wrote:

Will MV3 support host messaging?

Will the host program be started and stopped in alignment with the service worker lifetime?

--

You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/16a08f92-a18b-43a7-a70f-79b781468922o%40chromium.org.

[Giorgio Maone]

A simple use case, common to GreaseMonkey and its clones, but also ad-blockers which need to make up for functionality broken by blocking stuff on the page (AKA surrogate scripts), or otherwise modify the page's script execution environment before any active content runs; and of course to NoScript for almost anything it does, especially if blocking webRequest is gone (e.g. injecting user-dependent CSP policies via <meta> elements or injecting the aforementioned surrogates):

modifying the DOM and/or executing scripts depending on user-provided configuration which could vary per-site (do this on foo.com and that on bar.com) per tab (don't do anything on this tab), per frame-hierarchy (do the same as the parent/top frame) or a combination thereof, before any script on the page is executed and anything on the page is rendered.

I'm pretty sure this (already quite difficult in MV2) is utterly impossible in MV3, making at least these 3 classes of extensions non-viable, but I hope someone proves me wrong.

BTW, more than 10 months ago, after stating this case to Simeon in a video-call (he had graciously reached out to ask what are the critical pain points for NoScript), I proposed him

"a more generic, less intrusive and probably easier to implement, way to work around most if not all the problems we discussed also about your changes to WebRequest: adding a synchronous option to the messaging API (e.g. browser.runtime.sendSyncMessage()), working only from content scripts (much like the E10s sendSyncMessage() API which is restricted to the child process only), and possibly further restricted to just one message per page load and/or in the first content script execution per page, throwing an exception if this restriction is violated. In a multi-process browser the perceived and real performance impact would be negligible, since the main process / extension process would never be blocked and the content-side blocking, if any, would be limited to one spot at the very beginning of page loading.

This way any page / tab specific configuration could be easily retrieved from the background page (or service worker) at the right time (i.e. before page scripts start being executed) and action (e.g. DOM-based CSP injection or other countermeasures, even calling window.stop() to block page loading) could be taken by the content script, even without the need of a blocking WebRequest API."

He promised he would discuss this with the developers in charge, but nothing apparently happened in that direction since.

Another way to partially serve this use case (maybe a bit more complicated to use for extensions developers, but already specced and ready to implement) would be porting to Chromium Mozilla's WebExtensions contentScripts.register() API: it wouldn't help in the per-tab configuration case, though, but maybe it could be extended with a tabId filter option.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/CABc02_L9--%2Bd1ZHLL-yfyakhgvYFn%3DTvYJTapB7ofSpmhJ8duw%40mail.gmail.com.

[JohnnyCee]

The background script communicates with the native application. In my case, the background page opens a port, and I assume that the browser starts the native application when it loads the extension but I suppose it's possible that it loads it when the background application attempts to open the port. When a content script needs to communicate to the native application, it sends a message to the background script which then sends it along to the native application, and the path is reversed when the native application responds.

At present, the lifetime of the native application appears to be the same as the lifetime of the browser. Whether that changes or not is probably not critical to my extension, but it would be nice to know these sorts of details. A lot of what I know was gathered while observing what happens and not from reading the docs, which are pretty sparse (IMO) with regard to native messaging.

[Simon Bassu]

We have almost the same use case as JohnnyCee.

But, in our case, the native app could send back to the background much later, depending on the user choice. It won't be a good user experience if he has to redo all his work after x minutes because a ServiceWorker dies.

We might try to put a heartbeat to keep alive the port... this is not the way we want.

Simon B.

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/75c974eb-cdb9-4056-90ec-68eda138f5a8o%40chromium.org.

[Adi Bhaskar]

Another query: Will there be support for modules in service workers in MV3?

With service workers limited to a single file, not having module support may force either a lot of refactoring, or combining all background scripts into one file. 

> When will the non-blocking webRequest be available to test with service workers in MV3, or even event pages in MV2?

> You can already use webRequest in MV3 extensions on Canary. I just threw together & tested the following demo in Canary: https://gist.github.com/dotproto/1da2d4512685c8f6c7f8e2e83a2bbbae.

> While it's not possible to use webRequest with an MV2 event page, you can use service workers and webRequest in an MV2 extension in Canary. 

Thanks for this update, Simeon

Cheers!

Adi

[PhistucK]

Service workers in the web platform do not support modules yet (if I am not mistaken), so I do not expect them to be supported in extension service workers as well.

However, I believe their ugly cousin, importScript, is supported.

☆PhistucK

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/97ed5fca-3877-4622-b85a-1e489592f749n%40chromium.org.

[wOxxOm]

See https://crbug.com/824647 for ES Modules in service workers, the work will be "resumed soon". Since it's primarily beneficial to the web platform in general there's some chance it'll be implemented in time.

> While it's not possible to use webRequest with an MV2 event page 

Not true. The webRequest API can be successfully used with event pages in an MV2 extension by declaring it in optional_permissions and then preventing the event page from unloading (example) for the duration you need to use the API. While it may appear hackish, but technically it's just using the officially documented behavior to overcome the only problem with webRequest and event pages - webRequest events don't wake an unloaded event page, which is why the above workaround is needed.

[bradley cushing]

> Are you planning to add an alternative that would allow us to make heavy logic asynchronous in another way?
> As of yet we haven't seen a demonstrated need for this capability and therefore have no plans on this front. If you have a use case where the current serviceWorker implementation is demonstrably insufficient, please open a feature request on crbug.com to discuss.

I just wanted to clarify this statement as the documentation on migration seems to state otherwise. Based on your statement and the others below, it seems to not be possible to spawn other workers from a service worker and I am forced to have everything running in the same, single thread. (https://developer.chrome.com/extensions/migrating_to_manifest_v3) The page specifically is quoted as saying about service workers in MV3, "It takes a string specifying the relative path of the extension's root service worker. Additional scoped service workers can be registered from the root service worker, but they will not have access to Chrome APIs and cannot be registered for the root ('/') scope." Is this not the case and something that needs to be taken up with WHATWG specifically if so?

[Simeon Vincent]

FYI I started drafting this reply last week but was only just able to finish it. I'll be catching up on the thread soon and hopefully replying to other posts tomorrow.

I wasn't the only one who demonstrated the examples, just the more vocal one probably. Among many "theoretical" examples (that are used practically) I've named several popular extensions (Tampermonkey, Violentmonkey, Stylus of those I remember right now) that will be unable to work (or will be severely crippled) without a persistent background script environment or an ability to make it persistent on demand. - wOxxOm

FWIW I appreciate your vocality. I second PhistucK's comments that "You help a lot of people and you are very, very insightful. I see your comments on the tracker as well as on this group, your help is definitely noticed". That's why I keep shrugging off the barbs & continuing to engage. I'm hoping that by clarifying the Chrome team's position, assumptions, and concerns you and others will be able to more effectively achieve your goals.

To that end, I still feel like we're not on the same page, so I'll take another shot. We're consciously making foundational changes to the platform with MV3 and the cost of supporting long lived execution environments is significant. It feels like an uphill battle to maintain features of the previous model because, well, it is. We're relatively confident that nearly all extensions will be able to adapt and the end result will be a net better for the ecosystem. That said, I don't think anyone I work with is absolutist or unwilling to be swayed by evidence.

There are certainly ways to write extensions that won't work with service workers, but our position is that most if not all extensions can be adapted. As such, I think the most practical way to convince the engineering team that the extension platform must have a long lived execution context is to present a Manifest V3 extension that does not work and cannot be reasonably adapted to an ephemeral model with the current or planned APIs. Once we have something to dig into, I expect that we'll look at how the extension works and propose other ways to address the issues the extension runs into. And if it cannot be adapted we'll be in a much better position to assess the impact that missing capability has on the platform and the constraints we're working.

I'm not aware of any in-progress ports or MV3 versions of the extensions you mentioned, so at the moment we don't have hard evidence that these extensions will not be able to adapt due to the lack of a long lived execution environment.

I thought that point was about 'Are you planning to add an alternative that would allow us to make heavy logic asynchronous in another way?", not about "Are you planning to implement alternatives to force the state to be persistent or to use storage APIs synchronously?"... - PhistucK

IMO both readings are good questions. There was some additional context in the original contact that I excluded because it was too much background, but that content focused on state persistence and synchronous access patterns for storage APIs. That said, I'm happy to address whatever questions folks have about MV3.

He promised he would discuss this with the developers in charge, but nothing apparently happened in that direction since. - Giorgio

I expect that we'll go in a different direction than the one you've proposed as we're moving the platform away from synchronous & blocking APIs. We were originally hoping to have something to share by now, but we haven't started on this yet in no small part due to the pandemic's impact on the team.

We're considering an API that would allow for dynamic content scripts in a service worker compatible way. I can't get into specifics because we haven't started design work yet (we will soon), but based on previous discussions I'm cautiously optimistic that it will address your use case and hopefully support an appropriate level of dynamism.

I'm looking forward to your feedback once we have something more concrete to share.

Cheers,

Simeon - @dotproto
Extensions Developer Advocate

[cho ngao]

Vào 21:30, Th 6, 17 thg 7, 2020 JohnnyCee <jfcar...@gmail.com> đã viết:

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/75c974eb-cdb9-4056-90ec-68eda138f5a8o%40chromium.org.

[James Browning]

Will Manifest V3 bring extensions to mobile devices? It seems the more to service workers to improve performance and the increased privacy/security of V3 would solve a lot of the issues with extensions on mobile. So I was wondering if this was on the plans.

[wOxxOm]

Officially, bringing extensions to mobile platforms has never been planned in Chrome/Chromium at any point judging by https://crbug.com/113111. It wasn't mentioned in the MV3 draft either.

Personally, I've been using a Chromium fork that can run extensions on Android, Yandex browser, for several years and it was an excellent experience, I don't even remember any problems. I've also tried Kiwi browser when it just announced its support of extensions, it was also good. There were no additional problems posed by the presence of extensions on a budget phone so for me the performance wasn't even an issue so I'd extrapolate that generally it's not a deal-breaker, too.

The problem, judging by the git commit log analysis and my observations on the bug tracker is that even the current implementation of the extension component in Chrome/Chromium is in so-called maintenance mode for the last 5+ years with hundreds of reported&confirmed bugs or suggestions that no one has time to tackle.

Some potentially good news is that Kiwi's developer is fixing issues around mobile support of extensions in the source code in Chromium, see https://crbug.com/1074710, which will make it easier to splice in the support of extensions in the future. Right now, it's helpful only to Kiwi and Yandex (that I know of), but hopefully it would nudge Chromium developers (or whoever makes these kind of decisions there).

BTW, there should be no noteworthy performance difference in general compared to the MV2 event pages. Only extensions that were using webRequest a lot (like, on every network request) will be noticeably faster if they switch to the new declarativeNetRequest.

[Giorgio Maone]

Wed 22 Jul 2020 05:18 Simeon Vincent <sim...@chromium.org> wrote:

We're considering an API that would allow for dynamic content scripts in a service worker compatible way. I can't get into specifics because we haven't started design work yet (we will soon), but based on previous discussions I'm cautiously optimistic that it will address your use case and hopefully support an appropriate level of dynamism.

 

Just to be sure it's clear, we do NOT need any kind of eval()-like capability to dynamically register arbitrary strings as scripts to be evaluated at runtime (which I understand from other discussions about MV3 would be absolutely unacceptable), but we DO need a way to initialize the current static content scripts with dynamic data, readily available on instantiation and dependent both on the URL of the page they're loaded into and/or on the tab they're running.

As I wrote, it might something similar to contentScripts.register(), or even a partially compatible porting, with some important differences:

1. It would likely not support the js.code option, because it's the kind of eval()-like capability you want to avoid (even though is the only way the API as it is at this moment would serve our use case)
2. Instead of js.code, it must support a js.data option, which should be a purely JSONable object (no executable code allowed) which would be merged into the global object of the content script(s) matching the registration filters (which should optionally include one or more tab IDs) before they are executed (run_at for data would always be "document_start")
   
3. To be useful in a ServiceWorker, the registrations should survive until explicitly deregistered (or until the extension is disabled/uninstalled or the browser is closed), instead of being automatically deregistered when the registering script is unloaded, as currently specced. Therefore either serializable handles for later deregistration or at least a deregisterAll() method should be provided.
4. Bonus (nice to have, but not strictly required if both include/exlude filters are provided), merging/execution order reflects registration order.
   

For instance,

// SERVICE WORKER

var unrestrictedSites = ["https://a.org/*", "https://b.org/*"];

var unrestrictedTabs = [ 13, 18 ];

browser.contentScripts.register({

 js: { data: { policy: "restricted" } },

 matches: ["<all_urls>"],

 excludeMatches: unrestrictedSites,

 excludeTabIds: unrestrictedTabs,


 matchAboutBlank: true


});

browser.contentScripts.register({

  js: { data: { policy: "none" } },

  matches: unrestrictedSites,

  tabIds: unrestrictedTabs

}

// CONTENT SCRIPT

switch (window.policy) {

   case "restricted":

     // everywhere, except...

     break;

  case "none":

    // tabs 13 & 18 or a.com / b.com

    break;

  default:

    // maybe about:blank?

}

I'm looking forward to your feedback once we have something more concrete to share.

Needless to say, I'm extremely eager to have something concrete to start reasoning about.

-- G

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/CAFY0HLMxhT5or2AwMSwpcRAQgaOChxtApxpwKQAamuC0-RPWKA%40mail.gmail.com.

[Giorgio Maone]

Another important use case which should be covered by DNR before it can be considered as a viable option for NoScript would be per-tab enablement/disablement of rules, e.g. via excludeTabs / includeTabs int array properties (containing tab ids) in the RuleConditions type.

This would allow users to temporarily remove restrictions for the current tab, which is a quite common usage pattern not just in NoScript but in other content-blocking extensions such as uBlock.

[Karan Bhatia]

declarativeNetRequest supports the allowAllRequests rule action which can be used to disable all rules on a page. Does that suffice? If it doesn't, can you elaborate on the use case by giving some examples of user flow?

- Karan (On behalf of extensions team)

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.

[Giorgio Maone]

The typical use case and is:

• Initiating a trusted workflow potentially spanning mulltiple domains, which I  absolutely don't want to be interrupted by unforeseeable consequences of current restrictions, e.g. finalizing a purchase on a trusted e-commerce website, which may redirect to one or more 3rd party payment processor services
• I select "Remove all restrictions for this tab"
• Perform the workflow until completion
• Either "Restore restrictions for this tab" or close the tab

I'm sure it's a limit of mine, but I cannot see any way to apply your allowAllRequest or any other rule filtering by tabId and/or frameId.

BTW, another important use case (initially requested by the Tor Project, and probably useful to anyone wanting to reduce fingerprinting) is the option to apply the same rules for all the frame hierarchy, independently of the settings which would otherwise apply to the subframes.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/CAFY0HLMxhT5or2AwMSwpcRAQgaOChxtApxpwKQAamuC0-RPWKA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/5ce2922e-1460-4a87-ac85-e1adf32258b4o%40chromium.org.

[ilyaigpetrov]

What are the plans for chrome.proxy and PAC-scripts support in MV3?

[Simeon Vincent]

ilyaigpetrov, forgive me, but I don't follow the question. I'm not aware of any plans to change how chrome.proxy or PAC scripts behave in Manifest V3. Can you provide more context on your concerns here?

Cheers,

Simeon - @dotproto

Developer Advocate for Chrome Extensions

[ilyaigpetrov]

> ilyaigpetrov, forgive me, but I don't follow the question. I'm not aware of any plans to change how chrome.proxy or PAC scripts behave in Manifest V3. Can you provide more context on your concerns here?

My concerns are that PAC scripts may conflict with the proposed principles:

1) Remote PAC-scripts may be considered as remotely hosted code (though restricted in its abilities).

2) PAC-scripts in general may be considered as a persistent js-code.

[Karan Bhatia]

Thanks for the explanation. I checked out NoScript and it seems they do allow excluding a tab temporarily. Even something like this can be implemented with allowAllRequests rule but will probably be racy and somewhat complicated. (E.g. listen for main-frame navigations and add an allowAllRequests rule for the main-frame url). I'll think about this more.

Regarding content blockers, the ones I have seen (including ublock origin) just allow-list the top level frame url and not the tab. This should be possible to do with an allowAllRequests rule. 

BTW, another important use case (initially requested by the Tor Project, and probably useful to anyone wanting to reduce fingerprinting) is the option to apply the same rules for all the frame hierarchy, independently of the settings which would otherwise apply to the subframes.

Again, can you point to an existing extension which does something similar? Alternatively, you mentioned this was requested by the Tor project. Can you link the relevant thread?

Thanks

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/CAFY0HLMxhT5or2AwMSwpcRAQgaOChxtApxpwKQAamuC0-RPWKA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.

[Giorgio Maone]

2020-09-12 01:06 Karan Bhatia <karan...@chromium.org> wrote:

Thanks for the explanation.

Thank you for listening.

I checked out NoScript and it seems they do allow excluding a tab temporarily.

"They" is me, I'm the developer of NoScript ;)

Even something like this can be implemented with allowAllRequests rule but will probably be racy and somewhat complicated. (E.g. listen for main-frame navigations and add an allowAllRequests rule for the main-frame url). I'll think about this more.

I'm not sure that would work at all: each main frame load in the tab needs itself to be subject to the new allowAllRequest rule, otherwise DNR header modification would be used to inject a script-blocking CSP to take care of inline scripts.

Therefore the decision must be taken with reliable timing in respect of the main frame HTTP request (currently in webRequest.onHeadersReceived), a level of coordination for which the webNavigation API provides no guarantees AFAIK.

This is just an example of why the lack of flexibility, disproportionate to the supposed advantages of removing the blocking webRequest APIs (instead for instance of making it asynchronous like the Firefox version, which is great: the lack of an asyncrhonous blocking onBeforeRequest is the reason why I could not port NoScript's XSS filter on Chromium), makes DNR a poor replacement for webRequest.

Regarding content blockers, the ones I have seen (including ublock origin) just allow-list the top level frame url and not the tab. This should be possible to do with an allowAllRequests rule.

That's unfortunately inadequate to the level of control NoScript users need (and the level of workflow disruption the lack of it can cause, which is very different from a blacklist-based content blocker).

Can you point to an existing extension which does something similar?

NoScript is quite existing, isn't it? NoScript Options>General>Cascade top document's restrictions to subdocuments

You mentioned this was requested by the Tor project. Can you link the relevant thread?

I've been under contract with the Tor Project in 2014 to implement this and other anonymity-related features in NoScript "Classic" (before Mozilla switched to the Chrome-inspired WebExtensions API).

Here's a more recent Tor ticket about a bug in the WebExtensions porting of this feature (in NoScript "Qantum").

Now that I hope I've got your attention, can I point you to this proposal I detailed in this thread in back in July, which Simeon said was similar to something you were actually designing or implementing, and that would solve a lot of problems (even, with some gymnastic, the two just mentioned in this message (for instance allowing the CSP to be reliably and dynamically injected directly in the DOM by a content script) and would also serve other important use cases, such as all the Greasemonkey-like userscript extensions and the so called "Surrogate Scripts", i.e. configurable replacements/modifications of the JavaScript environment available to a page before any page scripts gets to run?

I apologize for the redundancy, but let me quote Simeon and myself again for clarity - that would be a really useful API, and already partially implemented by Firefox:

Thank you for reading.

-- G

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/CAFY0HLMxhT5or2AwMSwpcRAQgaOChxtApxpwKQAamuC0-RPWKA%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/5ce2922e-1460-4a87-ac85-e1adf32258b4o%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/9268b8cd-31a1-4b9d-9e17-8c7661996f84o%40chromium.org.

[Karan Bhatia]

This is just an example of why the lack of flexibility, disproportionate to the supposed advantages of removing the blocking webRequest APIs (instead for instance of making it asynchronous like the Firefox version, which is great: the lack of an asyncrhonous blocking onBeforeRequest is the reason why I could not port NoScript's XSS filter on Chromium), makes DNR a poor replacement for webRequest.

There is always going to be a tradeoff between the level of flexibility we provide to extensions and the level of privacy/performance guarantees that we can provide to the users. Even this case should be solvable by DNR with some additional support. 

NoScript is quite existing, isn't it? NoScript Options>General>Cascade top document's restrictions to subdocuments

Not debating that :) I am not familiar with NoScript's workflow, hence the request for clarification.

Now that I hope I've got your attention, can I point you to this proposal I detailed in this thread in back in July, which Simeon said was similar to something you were actually designing or implementing, and that would solve a lot of problems 

You didn't link to anything, but I see that you mentioned contentScripts.register elsewhere in this thread. If you were alluding to it, I'd like to reiterate what Simeon mentioned. This is already in the pipeline and something that'll be implemented. We'll share a proposal on this group when we have aligned on a design.

Back to the original question, regarding filtering by tabID for DNR rules. I don't actually know of other use cases, besides the one you mentioned for NoScript. Do you mind filing a bug for this on crbug.com with the necessary context?

[Giorgio Maone]

Issue 1128112: The declarativeNetRequest [API] needs a tabIds RuleCondition property for dynamic rules

And yes, contentScripts.register() was the API I mentioned as a model (to be further amended with tab filtering as well) and linked, too, in the reiterated quote I was apologizing for, which GMails unexpectedly hid but that you could retrieve by clicking the [...] ellipsis on the bottom of the previous message.

Thank you.

-- G

--

You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/6efb4376-6438-4db9-941b-a58139975592o%40chromium.org.

[Guilherme Dellagustin]

Today it is possible to play media from the background page. I use this in my extension so that users can continue listening to media even if they close the extension packaged page.

With the migration to Service Workers I understand this will not be possible.

Is there any mitigation plan that will enable this feature?

It will be a breaking change for my extension.

Em quarta-feira, 15 de julho de 2020 às 00:49:56 UTC+2, Simeon Vincent escreveu:

Hey all,

A handful of questions related to MV3 recently landed in my lap. I figured the answers would be interesting to the broader community, so I figured I'd share more info here. If you have additional questions feel free to drop them in this thread and I'll do my best to answer.

Are you planning to implement alternatives to force the state to be persistent or to use storage APIs synchronously?

We are not planning to provide any other background execution environment or to make the current chrome.storage APIs synchronous. There is a feature request to create a new storage API that would provide an in-memory storage API for critical persistent data (crbug.com/930235) .

With regards to synchronicity, I want to highlight that all message passing between content scripts and the background context are asynchronous. While managing asynchronous data can cause a bit of a headache, we're planning to implement promise based APIs as part of MV3. Promises in combination with async functions make it possible to work with asynchronous logic in an imperative manner.

Will there be an option to keep using Manifest v2 after end-of-life? (e.q. running chrome with the flag)

A final decision has not been made, but speaking from a personal point of view I wouldn't be surprised if we had something like this during a transition period. I believe we did something similar during the Manifest V2 migration. This would primarily be exposed to aid in enterprise environments. Generally speaking end users should not be manually configuring their Chrome flags. Also note that CWS will require that extension developers upload MV3 versions of their extensions before Chrome stops loading them.

When can we expect to have a version that is stable to test it?

Manifest V3 has been available for testing in Chrome Canary since October 31st, 2019. Once we have enough of the Manifest V3 platform implemented and it's solid enough for public release, it will be made available in Chrome Stable. We do not have a firm date, but we expect this to happen in 2020.

How much time can we expect to have to migrate to v3 after the end-of-life will be announced?

I expect at least at least a 1 year transition period between when MV3 lands in Stable and when MV2 reaches EOL. 

"Like other listeners, alarm listeners should be registered in the top level of your script."

Are you planning to remove setTimeout completely?

No. Extension service workers are based on the open web's service workers, and those service workers can still access setTimeout. The quoted section of the migration guide is attempting to highlight that setTimeout may not work as expected in a service worker environment due to their ephemeral design. If the service worker is terminated before the timeout expires, the timer will be dropped and the callback will never be called.

Are you planning to add an alternative that would allow us to make heavy logic asynchronous in another way?

As of yet we haven't seen a demonstrated need for this capability and therefore have no plans on this front. If you have a use case where the current serviceWorker implementation is demonstrably insufficient, please open a feature request on crbug.com to discuss.

What will be the minimum alarm trigger time?

Yes, alarms require a minimum amount of time to pass. Per the alarms API documentation:

In order to reduce the load on the user's machine, Chrome limits alarms to at most once every 1 minute but may delay them an arbitrary amount more. That is, setting delayInMinutes or periodInMinutes to less than 1 will not be honored and will cause a warning. when can be set to less than 1 minute after "now" without warning but won't actually cause the alarm to fire for at least 1 minute.

Are you planning to completely remove blocking webRequest APIs?

The observational capabilities of webRequest will remain in Manifest V3, but the introduction of declarativeNetRequest and planned changes to host permissions will significantly reduce the number of extensions that require the webRequest permission.

We are planning to disallow blocking webRequest in consumer extensions. Extensions that have been force-installed in managed environments via policies such as ExtensionInstallForcelist will be able to access blocking webRequest. If your extensions may be used in both and you opt to continue using blocking webRequest in managed environments, you will either need to distribute different versions for each environment or add conditional logic to gracefully handle when this feature is not available.

Will there still be a way to modify headers or the response?

The declarativeNetRequest API will allow header modification via the modifyHeaders action. I'm not aware of any plans to support response body modification. If this is critical to your use case, please open an issue on crbug.com to discuss.

[wOxxOm]

>  play media from the background page

The fact that it'll be impossible was mentioned when ManifestV3 was just announced in the main tracking issue but no one followed up on that. The author probably thought the chromium team would take note (as it usually happens in software projects) but chromium team decided to ignore all that feedback under pretext it's all non-constructive.

Open a new bug report on https://crbug.com just in case. Personally I don't think they can do anything about that but I'm notorious for my pessimism so keep the hope alive.

[Matt Gaspar]

Is there any update on when V3 will be released to Stable? We're quickly approaching the last few months of 2020.

When can we expect to have a version that is stable to test it?

Manifest V3 has been available for testing in Chrome Canary since October 31st, 2019. Once we have enough of the Manifest V3 platform implemented and it's solid enough for public release, it will be made available in Chrome Stable. We do not have a firm date, but we expect this to happen in 2020.  

[Simeon Vincent]

Is there any update on when V3 will be released to Stable? We're quickly approaching the last few months of 2020. - Matt

No updates to share. We're still aiming to get MV3 into Stable in 2020.

Today it is possible to play media from the background page. I use this in my extension so that users can continue listening to media even if they close the extension packaged page. … Is there any mitigation plan that will enable this feature? - Guilherme

This use case will not be supported In the initial version of Manifest V3 when it lands in Chrome Stable. It's on our radar but we don't have a concrete plan at the moment. Also, I just opened crbug.com/1131236 to track audio playback specifically. It would be helpful if you could describe how you use audio in your extension & the impact the loss of this feature would have on your extension

As for workarounds, the Service Worker migration doc has some guidance that boils down to "spawn or reuse a page to play audio." If this is a critical feature for your use case, it may be reasonable to hold off on migrating to MV3 until a more robust solution is available. 

Cheers,

Simeon - @dotproto

Developer Advocate for Chrome Extensions

[wOxxOm]

Per https://crbug.com/1133034 the next Chrome i.e. 86 will have "service worker-based extensions in the Stable channel" which probably means the entire ManifestV3 because the former wouldn't make sense alone.

[glr...@gmail.com]

Thank you @simeon. This was very informative.

[Simeon Vincent]

Folks may not have noticed, but service workers have been available in MV2 extensions in Canary for several months now.

The reason we're exposing service workers in MV2 is to help developers migrate to MV3 without having to drop development of their production extensions. I expect that a number of devs will take a piecemeal migration where they flip the manifest_version field to 3, see what breaks, switch back to 2, and work on addressing that issue. 

Cheers,

Simeon - @dotproto

Developer Advocate for Chrome Extensions

[Kamt Schatka]

Since you seem to be so convinced that it is possible for developers to just rewrite their applications, here is what we are doing:

We have an extension that loads a page, tracks every single webrequest, waits for all of them to finish(and provides credentials if required). 
So with Manifest v3 we would be in constant fear of losing the complete state in our serviceworker, so we would need to listen to a new request being started, read the current state from the asynchronous storage or create a new object, store our information in that object and then write it asynchronously into storage. Then when the next phase of the request starts, we read the previously stored data asynchronously from the storage, modify the object, write it back into storage and keep doing that stuff until all requests are finished. And how will we know that all requests are finished? Right now we have a setTimeout call that comes back every 500ms so we can check that nothing happened in the last 500 ms and then we are finished and can continue processing the data we gathered.
I would be very much interested in how these kinds of use cases are supposed to be solved?

Am I completely missing something here?

[wOxxOm]

>  Am I completely missing something here? 

If I understand correctly, you're worried that the service worker will be terminated in those 500 ms of setTimeout, but a service worker is kept alive for 30 seconds in Chrome after each API event (like webRequest) that wakes it, and each such API event triggered while service worker is alive will reset this internal termination timer, so there should be no problems in your case.

A problem would arise if your background script makes a network request itself (via fetch/XHR) that doesn't complete in those 30 seconds or if a setTimeout is 30+ seconds and there were no API events triggered in between frequently enough to keep it alive. In that case there's a workaround: open one port to any tab's content script and keep it open (indefinitely or until you finished processing something). This will prevent unloading of the service worker, which is a documented feature. You can use the port's onDisconnect event to repeat the process with a different random tab when the first one was closed. When there are no connectable tabs (i.e. you can't connect to an empty new tab as it doesn't run content scripts in modern Chrome) you'll either have to let the worker die or open a new dummy tab just to have a port connection, which is why this workaround is kinda lame.

[Tomislav Zorčec]

How can we add a dynamic header value for all the requests, is it already possible, or will be implemented? We need to specify a dynamic server-timing property (an ID) for every request, that will be used to correlate the same requests between the website & background.

[Kamt Schatka]

There seems to be still a big whole when it comes to modifying or even adding headers. How to set Server-Time Headers for each request, what about the upcoming Trace-Context Headers? We are also setting "access-control-allow-origin" headers on requests to make sure we can capture resource timings in the page.

There is also the use case to remove headers only if they are in an old format we used to have a long time ago, but is still used by some customers, which messes up quite a lot.

Adding a static header value is possible, but everything that requires a simple if is basically impossible from what I have seen?

[wOxxOm]

Yes, your extension is yet another victim of Chromium team's decision to nuke the blocking webRequest in ManifestV3. The only thing you can do now is to describe your use case on https://crbug.com and hope they would come up with an alternative solution.

[馮韋元]

Hello,

I'm not sure if this solution would work for your use case without some bigger changes on your current logic, but you could consider the following:

1) Use declarativeNetRequest to filter requests by using some regular expression filter

2) Redirect to a local page redirect.html which is part of the extension and will execute redirect.js script

3) Redirect.js script can execute any sort of script, and finally redirect again to the URL of your choice.

Basically you're doing a 2 step redirection. Not sure if redirect.js would be able to read your original headers though. So it may require a different flow, but this is a solution to have a logic layer with declarativeNetRequest.

// Francois

[wOxxOm]

On Monday, November 16, 2020 at 10:35:47 AM UTC+3 馮韋元 wrote:

1) Use declarativeNetRequest to filter requests by using some regular expression filter

2) Redirect to a local page redirect.html which is part of the extension and will execute redirect.js script

3) Redirect.js script can execute any sort of script, and finally redirect again to the URL of your choice.

Sounds complicated and terribly fragile as well as confusing for the users as they will see the URL flicker. And it sounds like the navigation history will be a total mess. I really hope this is not what Chromium engineers envisioned for us when they promoted their arbitrary idea that a blocking webRequest is a bad thing and suggested switching to declarativeNetRequest. Use cases like this exist, but they're completely ignored by the Chromium team (as seen from the outside and in the source code of Chromium) even if they deeply and truly care about these cases in their heart of hearts.

[Kamt Schatka]

Boy I really hope that answer was a joke and you are not really suggesting to redirect every request to a page with a javascript file that performs stuff and then redirects further... As wOxxOm said, this is complicated, terribly fragile, confusing and just plain ugly and TBH a solution I would expect from a junior software developer and not from someone working at google trying to create a solution for the most used browser(i know how rough that sounds, but come on, really?)
How would you even change response/request headers in that way?
We have found a way with some rules to work around some of the issues, but we are still not sure how we would add dynamic headers to a request/response based on some information we only calculate in the extension.

[Tomislav Zorčec]

Like ssch mentioned, the current API evolved really nicely and many problems we faced were solved, but it is still not possible to add dynamic header values what is holding us back.
We need to have some kind of identifier of the request that will be visible in the background and in the resource timings (so we can combine collected information together). I created a feature request, and you can check out details if you are interested;  https://bugs.chromium.org/p/chromium/issues/detail?id=1147340

[guest271314]

I do not really use basic extensions. I use extensions for Natie Messaging. While working on a solution or workaround  for cancelling a download that is actually a Response from ServiceWorker I noticed that under certain conditions BroadcastChannel can remain alive even after unregistering the ServiceWorker https://bugzilla.mozilla.org/show_bug.cgi?id=1676043. So one option would be to establish a MessageChannel or BroadcastChannel communication channel and post messages back to main thread intermittenly to try to keep the ServiceWorker "alive" (https://plnkr.co/edit/4AWjlY2wFbgiqbZQ?preview) then close() the message channel. Even while working on the download cancellation solution the most difficult part to debug - without using an extension - is the ServiceWorker part, registering and unregistering. 

I am not sure how viable it would be to eliminate usage of extensions altogether and just use PerformanceObserver with Response and dynmaic ServiceWorker(s) that are spawned just to maintain some form of persistence.

Since Chromium source code is ostensibly FOSS, we can build Chromium without "MV3" extension implementation altogether and continue developing that branch of Chromium, to avoid having to ask for features, rollbacks, or to just leave things alone that are already working.

[馮韋元]

Well, we have extensions with thousands of users, and the new declarativeNetRequest do not offer the features that we have been using previously.

What are the alternatives here? Complain on this discussion group and hope for a new feature? 

That has not proven to be very useful in the past, specially if your use-case is very unique.

Do you have a better alternative for my use case?

- Intercept a query as it is being triggered on the ["main_frame"], block it

- Extract a URL parameter and do an asynchronous real-time operation on it

- Redirect to the final URL with the updated URL parameter

Even with webRequest, that was tricky, because webRequest could only redirect synchronously. Now it's even worse.

I'm still looking for a better solution, hence my browsing this discussion group, but it does not seem that it's a common use case and I have only little hope for such use case to be taken into account.

[guest271314]

The use case appears to be possible using ServiceWorker, onfetch event handler, Response, AbortController, and, or postMessage() without using an extension at all.

A draft template workaround for the use case of AbortController.signal.onabort handler somehow being fired in ServiceWorker onfetch event. https://plnkr.co/edit/oO2MOkVQb2GleSH4?preview. Adjust, test, break as you see fit.

[guest271314]

See https://bugs.chromium.org/p/chromium/issues/detail?id=823697#c21, https://bugs.chromium.org/p/chromium/issues/detail?id=823697#c22. Basically, just use fetch, onfetch, Response, AbortController, and some form of messaging (BroadcastChannel, MessageChannel, client.postMessage()), include PerformanceObserver in the procedure if you want.

On Tue, Nov 17, 2020 at 1:21 AM 馮韋元 <fran...@gmail.com> wrote:

[guest271314]

Again, since Chromium is ostensibly FOSS, and the interest is evident, we can build Chromium from scratch to do whatever we want and deploy the binary https://github.com/Eloston/ungoogled-chromium. Some features might require some careful consideration https://bugs.chromium.org/p/chromium/issues/detail?id=601636#c29.

I am not in to begging, so I create workarounds in lieu of some other individual or institutions' action or non-action.

On Tue, Nov 17, 2020 at 1:21 AM 馮韋元 <fran...@gmail.com> wrote:

[馮韋元Francois Devatine]

Thanks for the pointers but I'm not sure it can be applied here...

My use case is about intercepting requests made from the omnibox. There are no other html pages or scripts running, just the extension's background script...

[Tomislav Zorčec]

It is a long shot, but, if you really need to intercept the requests in a blocking way; you could proxy the traffic to the webasembly proxy where you would have full control over the requests. But, I haven't tried that yet, and to be honest, I would really hate to do something like that.
(I also wonder what would wOxxOm & Simeon say about that idea)

[guest271314]

In that case there is not a need for "intercept", rather to create a server installable locally or an external server to be set as the default search engine. That is one of the first actions I take when running Chromium or Chrome for the first time. In the past I have used that to serve speech synthesis output based on what is typed into the "omnibox", locally. Yet the same principle applies if you have a remote server.

[guest271314]

This is what I referred to 

where in this case localhost:8000 is the local server, but can also be a remote server. You can ship the local server with your extension.

On Tue, Nov 17, 2020 at 7:25 AM 馮韋元Francois Devatine <fran...@gmail.com> wrote:

[guest271314]

To start and stop the local server you can use Native Messaging. This allows bypassing ServiceWorker entirely. For example, where Chromium authors has decided for an unknown reason to not fix the issue, but refuse to capture PulseAudio monitor devices altogether at getUsermedia(), to capture system audio output I start a php server and stop the php server when am done capturing system audio output, see in pertinent part at https://github.com/WebAudio/web-audio-api-v2/issues/97#issue-688754941

captureSystemAudio() {
  if pgrep -f php > /dev/null; then
    killall -9 php & sendMessage  
  else
    php -S localhost:8000 -t /path/to/host/ & sendMessage
  fi
}
captureSystemAudio

On Tue, Nov 17, 2020 at 7:25 AM 馮韋元Francois Devatine <fran...@gmail.com> wrote:

[馮韋元Francois Devatine]

The whole point of what we are doing is privacy. We can not send the request to another server.

We also can't ask the thousands of users to install a local server on their machine.

They are users. They need to install the extension, and it must work out of the box

So the solution must come from within the extension, and so far my ugly javascript solution seems to be the only one working :\

[guest271314]

Well, there is really no such thing as "security" relevant to any signals, see A Good American https://agoodamerican.org/

You ship all of the server code with the extension, which runs locally, use Native Messaging to start and stop the server. Does not have to be php, C, Rust, Jelly, etc. or you can use netcat if you want. 

If your solution works, then use that.

[guest271314]

An example Native Messaging application https://github.com/guest271314/native-messaging-espeak-ng/blob/master/host/install_host.sh that downloads and builds espeak-ng from GitHub and opus-tools from downloads.xiph.org, meaning that it is possible for the script itself to download server code to function locally. There is a curve to using Native Messaging, yet once you get the pattern you can essentially do whatever you want at the OS level that communicates to the browser, including setting externally_connectable dynamically for the ability to run the code at any origin. So, you create an extension (Native Messaging) for each platform that you are expecting to use the extension, for *nix, *indows, *ac, etc.

On Tue, Nov 17, 2020 at 2:45 PM 馮韋元Francois Devatine <fran...@gmail.com> wrote:

[Matt Gaspar]

Simeon - checking in on updated schedule for MV3 going into Stable. Any news to share?

Is there any update on when V3 will be released to Stable? We're quickly approaching the last few months of 2020. - Matt

> No updates to share. We're still aiming to get MV3 into Stable in 2020.

[Simeon Vincent]

How can we add a dynamic header value for all the requests, is it already possible, or will be implemented? - Tomislav

Adding a static header value is possible, but everything that requires a simple if is basically impossible from what I have seen? - Kamt

This use case is not currently supported. Thanks for filing crbug.com/1147340, Tomislav. I'm glad to see there's already been some discussion on that issue.

It is a long shot, but, if you really need to intercept the requests in a blocking way; you could proxy the traffic to the webasembly proxy where you would have full control over the requests. But, I haven't tried that yet, and to be honest, I would really hate to do something like that. - Tomislav

I'm not sure what you mean by "webassembly proxy". You could certainly proxy or redirect requests through a local server. This appears to be the root of guest271314's approach (apologies if I got the wrong impression), but they aren't distributing the extension through CWS and their examples appear to require that end users are comfortable using CLIs.

Simeon - checking in on updated schedule for MV3 going into Stable. Any news to share?

Unfortunately not yet, but I think we'll have an update in the next few weeks.

Cheers,

Simeon - @dotproto

Chrome Extensions DevRel

[Tomislav Zorčec]

Simeon; with the "webassembly proxy" I meant to have a wasm implementation of the proxy server shipped with the extension. So we can still have everything inside the extension. But I'm still not sure if that is possible to implement. Also, it would be a very complex solution we would like to avoid.
At the moment it seems that we are missing some filter conditions in the declarativeNetRequests (will open a ticket for it), and https://crbug.com/1147340, so I'm still optimistic that we might get a Chromium solution for our use cases.

[Ceof1]

Follow up to Simeon Bassu and JohnnyCee's posts in July 2020:

If you are using a Service Worker instead of a Backgroup page as mandated by MV3, if your extension is using the Native Messaging API to communicated with a native app,

is there a need for the hearbeat sent from the native app back to the service worker alive? Or is there another strategy recommended for this scenario?

Thanks in advance.

On Friday, July 17, 2020 at 3:57:10 PM UTC+1 Simeon Bassu wrote:

We have almost the same use case as JohnnyCee.

But, in our case, the native app could send back to the background much later, depending on the user choice. It won't be a good user experience if he has to redo all his work after x minutes because a ServiceWorker dies.

We might try to put a heartbeat to keep alive the port... this is not the way we want.

Simon B.

Le 17 juill. 2020 à 10:30, JohnnyCee <jfcar...@gmail.com> a écrit :

The background script communicates with the native application. In my case, the background page opens a port, and I assume that the browser starts the native application when it loads the extension but I suppose it's possible that it loads it when the background application attempts to open the port. When a content script needs to communicate to the native application, it sends a message to the background script which then sends it along to the native application, and the path is reversed when the native application responds.

At present, the lifetime of the native application appears to be the same as the lifetime of the browser. Whether that changes or not is probably not critical to my extension, but it would be nice to know these sorts of details. A lot of what I know was gathered while observing what happens and not from reading the docs, which are pretty sparse (IMO) with regard to native messaging.

On Thursday, July 16, 2020 at 5:34:43 PM UTC-4, PhistucK wrote:

How does native messaging work today with event pages? I am guessing it would work the same way with service workers.

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/75c974eb-cdb9-4056-90ec-68eda138f5a8o%40chromium.org.

[guest271314]

> is there a need for the hearbeat sent from the native app back to the service worker alive?

should not be necessary. If that procedure is necessary I wuld consider that to be a bug.

Can you provide code to test?