webRequest. How to set up a redirection logic with dynamic rules

[Bogdan Nazaruk]

Hi there! 

I looked at how it used to work in v2 and that seemed to be very straightforward. The v3 reimplementation of the webRequestBlocking called declarativeNetRequest seems super awkward to me.

I can understand the use of it when the redirection rules are sort of static, however, I'm not sure what would be the best way to implement dynamic redirections. 

Basically, I want to let the users set the redirection rules in the extension's popup, then I save those in the sync storage, then I want to pick them up from the storage in my background script on every onBeforeRequest (which sounds expensive, but I'm not sure what would be better to do in the background given how it's not reloaded on every page reload.) and in the callback there decide whether I want to redirect based on my storage or not.

That ruleset can change on the fly after the user changes it in the popup and they would typically reload the page to apply the changes.

Thanks!

[Jackie Han]

How dynamic? You can update dNR rules dynamically. For example, when a user adds a rule in your extension, you add a new rule in declarativeNetRequest.

If you want to programmatically determine at runtime, dNR can't do it. One possible approach is to redirect some urls to your extension page first, then do further processing on the page (Not sure if this way can solve your problem).

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/7eb42e5b-3329-48b3-b8fd-0c87d9bcbae5n%40chromium.org.

[wOxxOm]

Generally, there's no way to modify request behavior per request individually and dynamically in ManifestV3. Chrome team decided that such use cases are negligible and they periodically confirm they still believe so despite the multiple reports to the contrary and no evidence to support their claims. Once ManifestV2 is killed, your only general-purpose workaround will be to use webRequestBlocking in a force-installed extension via policy or in a patched binary. In some cases you might be able to update DNR rules inside a non-blocking webRequest listener, but it'll only apply to requests made after the update call completes asynchronously, i.e. AFAIK it won't apply to responses for the ongoing requests.

[Bogdan Nazaruk]

Okay. That's a biggie. Wow. Didn't expect it. 

First, yes, Google is very wrong about this. Not even sure what was wrong with this functionality in v2. Is this what people talk about when they say v3 is sneakily making it impossible for the adblockers to continue their existence? I'm not making an adblocker tho. My tool is a debugger and in my industry we routinely use extensions to redirect production libraries loaded on pages to lower env libraries that we've just built to test our work.

Okay, I really-really don't wanna reimplement this extension in v2 now and this piece of functionality is too good to not implement. Plus I spent like ten hours on the not-fun part of it (UI) and now I only have the fun part left. Can I update the DNR rules when the user adds the redirection rules in the extension popup? It's just I'm not sure how flexible that thing is. How we can programmatically update those rules? Let it be not inside of the listener. I don't like the idea of delaying the listener on every request to get the settings from the storage anyhow. If I need to reload the page after setting the redirection, it's fine. I think almost all existing extensions do that.

And I can't do it via the tab's window either... Gosh, what a pickle. Well, I guess I'll have to go and check how existing extensions do that, although most of them are v2 indeed.

[Bogdan Nazaruk]

You know what I could do? I could overwrite window.fetch... Feeding all my redirection rules to it and making it go through them all before fetching anything. Overwriting it on every pageview. Dirty? But what other options do I have?

[wOxxOm]

> overwrite window.fetch

It will work (you'll probably need to overwrite stuff in Response.prototype too), but there's a caveat: it won't work for the requests made by the workers of the page. For those you'll need to use `chrome.debugger` API to attach to a worker and then inject the override code.

> Can I update the DNR rules when the user adds the redirection rules in the extension popup? 

Yes, chrome.declarativeNetRequest.updateDynamicRules or updateSessionRules.

> It's just I'm not sure how flexible that thing is.

It's not flexible at all. The rules are declarative, there's no way to use JS code inside the rule, but depending on what your rules are actually doing it may be possible to solve some trivial cases for which DNR was created.

[Bogdan Nazaruk]

You know, I think my particular use case is trivial. I don't need regular expressions, wildcards or code in my redirections, it's always one endpoint to another per rule. I think what you gave me is exactly what I was looking for. Will try to work with the DNR. Overriding native JS feels fishy.

Thanks a lot!

[Jackie Han]

My tool is a debugger and in my industry we routinely use extensions to redirect production libraries loaded on pages to lower env libraries that we've just built to test our work.

 For your use case, MV3 declarativeNetRequest should work for you.

[Bogdan Nazaruk]

Ha. Seems like transform doesn't allow redirects to js files. And that's exactly what I need to do. I redirect one js lib to another. Here:

https://developer.chrome.com/docs/extensions/reference/api/declarativeNetRequest

"The redirect url. Redirects to JavaScript urls are not allowed."

Do I understand it correctly that DNR won't work for me? I'll have to go ahead with overriding the fetch?

Also, looks like neither fetch nor DNR can affect the workers' fetches. 

[Bogdan Nazaruk]

If they do it for security reasons, why don't they just add a permission to redirect to js files? Maybe there's one but I'm just missing it? I tried to look around and all they seem to be stating is that they just don't allow redirecting to js "URLs", which I'm not sure what it means. I can respond with JS in many ways, good luck catching them all. Probably either headers check for js or url check for .js. 

They already make it possible to load arbitrary js into extensions via a permission. Seems to me like a poor MVP nobody cares to complete. No wonder v2 is still not sunset. Sunsetting it at this point would be quite disastrous.

[Jackie Han]

To redirect to a file in your extension. You must declare these files in web_accessible_resources in your manifest.json.

The document already says this. More details see https://developer.chrome.com/docs/extensions/reference/manifest/web-accessible-resources

Redirects to JavaScript urls are not allowed. 

Here "JavaScript urls" means a url that is "javascript:" protocol, e.g. javascript:alert('Hello').There are various url protocol, like http: data: blob: javascript:

[Bogdan Nazaruk]

Ok, I don't want to redirect to files in my extension, I'm trying to allow users to redirect from an arbitrary .js to an arbitrary .js.

web_accessible_resources in my manifest allow all urls, so it shouldn't be a problem.

Never in my life would I have interpreted "JavaScript urls" as javascript: I even forgot you could execute JS through the address line. I think it was to make it easier to bookmark simple snippets or something. Totally forgot. The doc should mention what it means. Other people understand it as blocking .js redirects too.

talking about protocols. I'm actually trying to limit my extension from doing anything on chrome:// and other urls by only allowing http and https like so, but I still very rarely get the error pop up that extensions aren't allowed in chrome://

  "content_scripts": [{

      "matches": ["<all_urls>", "http://*/*", "https://*/*"],

      "js": ["js/content-script.js"]

    }],

  "web_accessible_resources": [{

      "matches": ["<all_urls>", "http://*/*", "https://*/*"],

      "resources": [ "js/page-script.js" ]

    }],

  "host_permissions": ["<all_urls>", "http://*/*", "https://*/*"],

  "externally_connectable": {

    "ids": ["*"],

    "matches": ["<all_urls>", "http://*/*", "https://*/*"],

    "accepts_tls_channel_id": false

  },

Anything I could do here to prevent it from doing anything on chrome:// ? Maybe get rid of <all_urls>?

Thanks for clarifying what they meant by JS urls. Will go back and try working with it.

[Jackie Han]

 I don't want to redirect to files in my extension, I'm trying to allow users to redirect from an arbitrary .js to an arbitrary .js.

If so, you don't need web_accessible_resources.

 I'm actually trying to limit my extension from doing anything on chrome:// and other urls by only allowing http and https like so

If the target url is the user's input, you should validate user input only starting with http(s).

[Bogdan Nazaruk]

I think I need the web accessible resources for something else. 

My extension does a lot of things. Setting up redirections is just a small feature that I'm trying to add now. The extension listens to web requests, selects some of them, parses them, then console.logs them nicely in the console, adding certain info from the tab's window scope via a bunch of chrome.scripting.executeScript calls.

My chtome:// errors relate to me trying to deploy listeners on the chrome:// tab. And I can't get the href of a chrome:// page to prevent deploying listeners there cuz that tab is protected. I discussed it before here and was told to limit it via the manifest, and it helped, but not completely. I still get the error. Very rarely tho.

[Bogdan Nazaruk]

Oh I just realized why this error happens.

my background script communicates with my content script by sending a message to an active tab. I get this error whenever the message is sent to the chrome:// tab. Nice. 

Is there a comfy way to get the related tab id from the callbacks of 

chrome.webRequest.onBeforeRequest

chrome.webRequest.onHeadersReceived

and

chrome.webRequest.onErrorOccurred

I would much rather speak to the same tab the requests were made from rather than the active one. The active one was just an MVP. heh. 

[wOxxOm]

tabId is provided in the listener's parameter, see the documentation: https://developer.chrome.com/docs/extensions/reference/webRequest/