Patching Worker scopes (esp. ServiceWorkerGlobalScope)

[Giorgio Maone]

I've recently commented on the MV3 dynamic scripting API bug to advocate for a friendlier API for this use case in Mainfest V3, but as far as I can tell on Chromium it's very difficult for Worker and SharedWorker (because you generally need to patch CSP in order to allow blob: or data: URIs in constructors) and impossible for ServiceWorker, which mandates a remote JavaScript file from the same domain.

As I wrote in my comment, on Firefox we can hack-around this using webRequest.filterResponseData() API, which unfortunately was created only because I was working for Mozilla in the WebExtensions API development team: there's nothing like that in Chromium, in facts.

Just in case I've missed something, can anybody here point me to any way to accomplish this (executing a script choosen by the extension in a ServiceWorker scope before the service worker's original code runs) on Chromium, either MV2 or MV3?

Notice also (I'm kind of a broken record about this) that even patching a "normal" window scope is currently very problematic because you cannot reliably configure a content script based on the page's URL before any page script gets to run.

Any suggestion is welcome.

[PhistucK]

> Notice also (I'm kind of a broken record about this) that even patching a "normal" window scope is currently very problematic because you cannot reliably configure a content script based on the page's URL before any page script gets to run.

"run_at": "document_start" should do it, in which case does it not work?

If you mean using chrome.tab.executeScript, yeah, makes sense.

☆PhistucK

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/2a695743-b4a0-45fc-b719-7842fdfbb1dan%40chromium.org.

[wOxxOm]

The only theoretically possible way to patch a Service Worker script in Chrome, AFAICT, is to use `chrome.debugger` API with a CDP command Fetch.fulfillRequest.

Patching web workers, which are instantiated from JS, should be possible by overriding `window.Worker` in page context. To patch a script from a remote URL you'll have to either use a synchronous XHR or devise a nontrivial Proxy wrapper that will download the code asynchronously and instantiate the real worker later meanwhile accumulating all calls to postMessage in order to send them when the real worker is created.

To configure a content script that runs at document_start you can set a cookie via chrome.webRequest and read it using a synchronous XHR + blob, it's the only reliable method in Chrome. Currently it's used by several extensions I know e.g. Tampermonkey's and Stylus's instant inject mode. The initial idea is here.

Interestingly, executeScript with document_start used inside webNavigation.onCommitted occasionally may run before the content script with document_start declared in manifest so you can combine the two approaches (example) to inject a config object via `code`.

[guest271314]

> Just in case I've missed something, can anybody here point me to any way to accomplish this (executing a script choosen by the extension in a ServiceWorker scope before the service worker's original code runs) on Chromium, either MV2 or MV3?

If I understand the requirement correctly, and you have write permissions to the local directory or remote server directory, you can dynamically create the script that the ServiceWorker will run, include any code that you want, then create the ServiceWorker.

[wOxxOm]

>  If I understand the requirement correctly, and you have write permissions to the local directory or remote server directory 

The requirement is similar to what content scripts do in the web page, but here the goal is to inject such a script inside the worker. Extensions don't operate on server or local directories though so maybe you could elaborate...

[guest271314]

I have no experience with Chrome Web Store extensions. I do have experience with local extensions. If the requirement is to dynamically insert code into a Worker, SharedWorker, or ServiceWorker script before the script is loaded the entire script can be written dynamically, to insert code at a specific position in an existing file, or write the file from scratch using File System Access API or WebTransport, then load the ServiceWorker.

[guest271314]

Alternatively, since Worker, SharedWorker, ServiceWorker each implement messaging, the code to be inserted can be done so with postMessage(), await completion of the code posted or transferred to the Worker, then proceed.

Again, this is just extension I am posting about, not Chrome Web Store that has restrictions and monitoring, etc.

[guest271314]

Chromium, Chrome implements Transferable Streams, which technically provides a means to insert code dynamically in parallel to other code being executed in the Worker scope, without waiting for completion of inserted code. The requirement appears to be to wait for inserted code to be completed before proceeding. Either option is available. I am not sure what code is expected to be executed, what the expected result is. I have streamed arbitrary audio from a WebTransport connection created in a ServiceWorker back to main thread which is output at HTMLMediaElement.

On Sunday, March 7, 2021 at 8:20:15 AM UTC-8 wOxxOm wrote:

[wOxxOm]

I don't see how File System Access API or WebTransport could help to patch an arbitrary service worker script, say, on twitter.com or mail.google.com. These service workers are downloaded from their remote URLs which the extension doesn't control. So it doesn't matter if the extension is local or from the web store.

[guest271314]

> These service workers are downloaded from their remote URLs

The files which create the ServiceWorker can be "downloaded" before actually creating the ServiceWorker and modified, then loaded.

Alternatively, a ServiceWorker can be created just to use onfetch, modify the ServiceWorker code then load the modified script.

[guest271314]

It would be helpful if the actual use case was posted.

[guest271314]

DevTools Local Modifications is also an option to run local code instead of code served by the site.

[wOxxOm]

> The files which create the ServiceWorker can be "downloaded" before actually creating the ServiceWorker and modified, then loaded.

Intercepting the js file that registers the service worker isn't a problem, it can be done without downloading the code in page context as described in my comment at the top of this thread, but it won't help here because the problem is what happens when navigator.serviceWorker.register is called.

> modify the ServiceWorker code then load the modified script 

How would you register the modified service worker? navigator.serviceWorker.register needs a remote URL that belongs to the site's origin for the service worker, it can't be a Blob or a data URI or a literal code string. In Chrome we can patch that remote file's contents only via chrome.debugger API (see my first comment in this thread).

> It would be helpful if the actual use case was posted.

The use case is simple: make an extension that injects arbitrary code in the context of the service worker for twitter.com.

> DevTools Local Modifications is also an option

It's not. The problem we discuss here is how an extension can do it.

[guest271314]

Is the requirement: On an arbitrary site that loads a ServiceWorker, 1) stop the ServiceWorker code served by the site from being registered at all; 2) modify the ServiceWorker code served by the site, then register the modified ServiceWorker (as if served by the site)?

Is something like this https://developers.google.com/web/tools/chrome-devtools/workspaces trying to be achieved?

[guest271314]

> How would you register the modified service worker?

Local Modifications maps to local resources. The locally modified code will be run, not the code served from Twitter.

[wOxxOm]

> Is the requirement: On an arbitrary site that loads a ServiceWorker, 1) stop the ServiceWorker code served by the site from being registered at all; 2) modify the ServiceWorker code served by the site, then register the modified ServiceWorker (as if served by the site)?

Technically, yes. Stopping isn't the required part though but it's unavoidable because there's no other way to inject arbitrary code except modifying the entire service worker. The problem is that it should be done by an extension, not by the user in devtools, so workspaces is not helpful. So far I don't see anything extensions can do in addition to the methods cited by the OP and in my comment at the top (chrome.debugger API).

> Local Modifications maps to local resources. The locally modified code will be run, not the code served from Twitter.

I see, so it doesn't seem helpful because the problem we discuss here is how an extension can do it.

[guest271314]

Actually,  Local Overrides https://developers.google.com/web/updates/2018/01/devtools#overrides.

For example, GitHub (github.com) does not allow loading of blob: or data: URL's (CSP) which results in AudioWorklet (effectively implemented as an Ecmascript module) not being capable of being loaded on github.com (https://github.com/WebAudio/web-audio-api-v2/issues/109), where I am using WebTransport to stream audio from espeak-ng executed in a shell script (https://github.com/guest271314/webtransport/blob/main/webTransportEspeakNg.js; https://github.com/guest271314/webtransport/blob/main/tts.sh).

I used Local Overrides to workaround that issue (https://github.com/mourner/bullshit.js/issues/72#issuecomment-757050803; https://github.com/guest271314/AudioWorkletStream/issues/1#issuecomment-766869141), meaning that the coe run is from the local directory.

> the problem we discuss here is how an extension can do it.

I have not tried that approach in an extension, certainly not from a Chrome Web Store extension becuase I do not use Chrome Web Store, nonetheless the same principle applies to extension code. Tests will produce a conclusive answer.

[wOxxOm]

>  I have not tried that approach in an extension nonetheless the same principle applies to extension code.   

The principle is patching the code of the service worker and forcing it to reregister, but it's the implementation part that is the problem, for which I see only one solution while staying entirely within the extensions API: chrome.debugger. As for WebTransport I don't see how it can be used to register the modified code of the service worker because service workers must belong to the site's origin so the URL can't point to another site like localhost, and this approach seems to require users to install a companion utility which is not something many users would be comfortable with.

[Alexei Miagkov]

>you cannot reliably configure a content script based on the page's URL before any page script gets to run

https://bugs.chromium.org/p/chromium/issues/detail?id=478183

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/CABc02_LprimRinq9wF0%2BykEMi8oasDkZX%2BRe%2BVGcMQ8z5QF8fg%40mail.gmail.com.

[guest271314]

> The principle is patching the code of the service worker and forcing it to reregister, but it's the implementation part that is the problem, for which I see only one solution while staying entirely within the extensions API: chrome.debugger.

Have you tested the Local Overrides approach in, or for an extension?

That is, navigate to Twitter, follow instructions at the linked article, modify the ServiceWorker script.

> As for WebTransport I don't see how it can be used to register the modified code of the service worker because service workers must belong to the site's origin so the URL can't point to another site like localhost

Native File System API and WebTransport were mentioned here and used to modify the local files programmatically.

> because service workers must belong to the site's origin so the URL can't point to another site like localhost

Local Overrides maps the local directory to the site being loaded. That is how I am able to run AudioWorklet on GitHub, where the CSP explicitly forbids loading blob: and data: protocols.

> and this approach seems to require users to install a companion utility which is not something many users would be comfortable with.

It is not necesary to install a quic-transport server. That is what I was using at the time to stream arbitrary data to arbitrary origins. 

> seems to require users to install a companion utility which is not something many users would be comfortable with.

Well, if users are installing extensions from Chrome Web Store they are already doing that.

File System Access API is shipped by default now with Chrome and Chromium, whih provides a means to write and read files, specifically to insert content in exact positions in an existing local file, or delete or overwrite the file.

I suggest testing the approach before condemning based on a user being uncomfortable. The approach is not ideal. However, I can confirm that is one way I am able to run AudioWorklet on a site where I use my own files on a site where loading external files from other origins is by design not permitted.

[wOxxOm]

> https://bugs.chromium.org/p/chromium/issues/detail?id=478183

Doesn't seem related because the linked issue is about registering the content script, which isn't a problem because the script is simply declared in manifest.json. The problem is configuring the script i.e. providing a data object that will be used by the content script at document_start to decide what to do before the very first page script runs, even the inline scripts in <head>.

> Have you tested the Local Overrides approach in, or for an extension?

Local Overrides feature isn't a part of the extensions API, what is there to test?

 

>That is how I am able to run AudioWorklet on GitHub, where the CSP explicitly forbids loading blob: and data: protocols. I suggest testing the approach before condemning based on a user being uncomfortable. The approach is not ideal. However, I can confirm that is one way I am able to run AudioWorklet on a site where I use my own files on a site where loading external files from other origins is by design not permitted.

The problem with service worker registration is not about CSP, it's about inherent restrictions of the service worker specification on the URL of the script.

[guest271314]

Note, quic-transport protocol is not blocked by CSP directives. If I wanted I could execute any arbitrary shell script I wanted by code I write in the browser using File System Access API, or use WebTransport itself to (over)write the file first, then execute.

File System Access API is not ideal for reasons other than the read/write capability: Google appears to not "respect" "Google Safe Browsing" being turned off at Settings, thus evidently "scans" the written files for "viruses", resulting in the potential for (temporary) files to be re-written several times before actually being written to local filesystem when close() is called https://github.com/fivedots/storage-foundation-api-explainer/issues/4#issuecomment-774548620. For this use-case that should not be an issue.

[guest271314]

> The problem is configuring the script i.e. providing a data object that will be used by the content script at document_start to decide what to do before the very first page script runs, even the inline scripts in <head>.

You can write whatever you want to to the script. 

> Local Overrides feature isn't a part of the extensions API, what is there to test?

The above approach. To avoid conjecture.

Is there a restriction on achieving the requirement using means other than Chrome's extension API implementation?

> The problem with service worker registration is not about CSP, it's about inherent restrictions of the service worker specification on the URL of the script.

As I stated above, the URL remains the same as on the original site downloaded locally (https://site.xyz/path/to/script.js), the actual file is fetched from the local filesystem directory.

[guest271314]

> The problem is configuring the script i.e. providing a data object that will be used by the content script at document_start to decide what to do before the very first page script runs, even the inline scripts in <head>.

How is that use case related to modifying a ServiceWorker script loaded by Twitter? 

What actual output is expected?

[wOxxOm]

> You can write whatever you want to to the script. 

Each tab needs different data so modifying the script is not an option. And even if we consider the case of a self-modifying extension that's not from the web store, it won't work anyway because the entire extension needs to be reloaded for the modifications in the content script to be seen by Chrome.

> Is there a restriction on achieving the requirement using means other than Chrome's extension API implementation?

Yes, this entire topic is about a way for an extension to patch a web site's service worker script, which is a very special case. So local overrides or file system API won't help.

> How is that use case related to modifying a ServiceWorker script loaded by Twitter? 

It's not, see the last paragraph of the original post.

[guest271314]

> it won't work anyway because the entire extension needs to be reloaded for the modifications in the content script to be seen by Chrome.

Then reload the extension. I do that after dynamically setting externally_connectable https://github.com/guest271314/captureSystemAudio/blob/master/native_messaging/file_stream/app/set_externally_connectable.js. 

> Yes, this entire topic is about a way for an extension to patch a web site's service worker script, which is a very special case. So local overrides or file system API won't help.

I am not sure how you can substantiate that conclusion without testing to verify?

>> How is that use case related to modifying a ServiceWorker script loaded by Twitter? 

>It's not, see the last paragraph of the original post.

Then why did you propose that use case as being related?

> Any suggestion is welcome.

I am highly skeptical of Chrome extensions API resolving users in the field feature requests and bugs, given Re: 💡[Idea] Support publishing both MV2 and MV3 version for the same extension at the same time

> Yes, every decision or approach has its cost. The cost of MV3 design is a lot of incompatible changes and creating a new version of extension. The cost of a smooth upgrade is to maintain two versions(MV2 and MV3) in a period of time, and need the support of Chrome and CWS. If the cost is too high to adopt, it can be solved by time (wait until most users upgrade to Chrome version X).

The structure is top-down hierarchy, not democracy. Thus, I posted here because of last paragraph at OP.

Testing, preferably by individuals other than those making the claim, is required by the scientific method "So local overrides or file system API won't help." needs to be verified, else that conclusion is premature and conjecture https://gist.githubusercontent.com/guest271314/1fcb5d90799c48fdf34c68dd7f74a304/raw/c06235ae8ebb1ae20f0edea5e8bdc119f0897d9a/UseItBreakItFileBugsRequestFeaturesTestTestTestTestToThePointItBreaks.txt.

I can appreciate the necessity of placing specific requests on the record. That does not preclude experimentation with alternative solutions in lieue of a formally specified or implemented API. If I were to wait (I do not wait on any other individual or institution for anything) on Chromium implementing capture of monitor devices for Linux https://github.com/guest271314/captureSystemAudio, or capture of Web Speech API speechSynthesis.speak() https://lists.w3.org/Archives/Public/public-speech-api/2017Jun/0000.html, or SSML parsing https://github.com/guest271314/SSMLParser, I would now be on at least several years of waiting (and counting).

Good luck!

[wOxxOm]

>  Then reload the extension 

It won't help anyway because each tab needs its own config.

> I am not sure how you can substantiate that conclusion without testing to verify?

I don't say it won't work with local overrides, but local overrides can't be created by extensions so there's nothing to test.

> Then why did you propose that use case as being related?

The OP mentioned this task and I've described a known workaround in my first comment. Then someone mentioned a loosely related issue on crbug and I commented on that, but it was never related to service workers.

[Giorgio Maone]

Thank you everybody for your attempts, and especially to wOxxOm, good willing and spot on as always.

Just to clarify: both use cases, i.e. (service)workers patching and early dynamic script configuration are requirement of two existing extensions (one, NoScript, with millions of users on Firefox and more than 100,000 on Chromium), so manual approaches like Local Overrides and, as far as I can tell, even the debugging API (unless its usage can be made frictionless and transparent to users who are not interested in dev tools) are not an option.

The cookie hack to synchronously pass arguments to the content script is something I did use year ago, but like several others I've tried (including my last resort, i.e. emulating synchronous messages through XHR+webRequest) had its issues (e.g. this).

Anyway, these use cases are indeed related, because in order to gain reliable control on a potentially hostile JavaScript/DOM environment, including the worker constructors / factory method, you need to run very early, before any other page script. And you need to know what to actually do, which may vary from domain to domain and from tab to tab depending on users' preference, hence the need for further configuration past any embedded in the static content script loaded at "document_start".

[guest271314]

> in order to gain reliable control on a potentially hostile JavaScript/DOM environment

If users in the wild are already using user scripts, extensions, and are attempting to control the above, they must be aware and capable of following the instructions on a Google developer site. Local Overrides only needs to be implemented once. 

I would suggest using Lynx, or disabling JavaScript altogether for the requirement, else by the time this is implemented formally (with or without a governing specification agreed upon by implementers), other technologies (Moore's Law) might have already been developed to, if not deliberately, effectively, thwart whatever is actually specified and implemented as to the above. That does not include workarounds sites, and developers themselves experiment with and deploy to thwart the implementation or specification.

My use case would be to stop the New Tab page from loading any JavaScript, or expect searches, and to permanently get rid of "tiles" amd "cards".

Policies can be rather effective, though I assess that users want only some JavaScript disabled or modified, not all, which might be difficult to achieve without leaving room to exploit that window.

If you can use any field testing just ping.

Again, good luck.

Cheers,

/guest271314/

[wOxxOm]

>  The cookie hack [...] had its issues (e.g. this).

Since the about:blank frames can access their `window.parent` directly they can read the config from there (it'll be the isolated world, too), see this example.

[wOxxOm]

> Local Overrides

AFAICT, this feature requires that devtools is kept open on the tab so it won't work for normal browsing.

> debugging API

It doesn't require anything from devtools or from the user, but when it's used Chrome displays a persistent warning above all tabs which won't autohide in modern Chrome. The only way to hide it is to run Chrome with a command line switch --silent-debugger-extension-api. Not something I would advise and definitely not something a typical user would be comfortable doing.

There no other methods to patch service workers in Chrome, AFAIK, so the only other approach would be to write an external program that acts like a proxy and communicates with the extension via nativeMessaging API to control its behavior.

[Giorgio Maone]

> Cookies

Unfortunately there were other issues, esp. regarding timing when multiple frames from the same domain are loaded at once and reliably avoiding to leak information

 to a hostile page, but I'm willing to re-examine once more time this option in the light of the existing examples you've pointed out.

> debugging API

Put this way, maybe it's something NoScript / JS Shield users might swallow up if they have to deal with pages requiring service workers but which they want to otherwise restrict against selected exploitable features.

> Proxy

Not really an option, because our use cases mandates modification of HTTPS response content.

Thanks again.

[Alexei Miagkov]

Sort of related: Have you looked into disabling Service Workers entirely (by injecting csp=worker-src 'none' I think)? (How much of the Web does that break? Has anybody written this up anywhere?)

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/CAF%2BkS4D%2BL3MNmnSBZZHRduGQpg_VPG%3D3JRq874Whyvx8DSfL6Q%40mail.gmail.com.

[Philip valdez]

Tag: ttcpw

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/CAOcx_p_X-7C-DzLLVbNSkvT26Gh5cUVtgXtZUje6b-0CzNyiyA%40mail.gmail.com.

[wOxxOm]

> Proxy - Not really an option, because our use cases mandates modification of HTTPS response content.

It should be possible if the user installs a certificate. Sounds scary but that's how Fiddler works and probably Charles as well. Also Proxomitron, which I've used when extensions were much less capable.