webRequest.onHeadersReceived - how to remove the X-Frame-Options header

[kazag89]

Hey,

we're looking into an option to use offscreen documents to perform some dom scraping of a website loaded in an iframe.

As far as I understand, we can use the following code to open the document:

chrome.offscreen.createDocument({
    url: 'offscreen.html',
    reasons: [chrome.offscreen.Reason.DOM_SCRAPING],
    justification: 'To do smth',
});

And this works fine.

Then we create an iframe with the URL of a website we want to scrap. The website however uses X-Frame-Options: Sameorigin so it doesn't load:

"Refused to display 'https://www.website.com/' in a frame because it set 'X-Frame-Options' to 'sameorigin'."

So we need to get rid of that header. We have MV3 extension, but we haven't used blocking web requests so far. AFAIU we would need a new permission to remove that header from the response: declarativeNetRequest (and its blocking version). The problem is that the permission will trigger a warning and users really don't like that.

I tried to use he following code:

chrome.webRequest.onHeadersReceived.addListener(
  data => {
    if (data.url.includes('website.com')) {
      const headers = data.responseHeaders ?? [];
      const withNoXFrameOptions = headers.filter(h => h.name !== 'x-frame-options');
      return { responseHeaders: withNoXFrameOptions };
    }
  },
   options,
   ['extraHeaders', 'responseHeaders']
);

const options: RequestFilter = {
  urls: ['<all_urls>'],
  types: ['xmlhttprequest', 'main_frame', 'sub_frame'],
};

It doesn't work. The listener is triggered but it doesn't override the headers. I believe it's because we would need the "blocking" option specified, which requires the blocking webRequest permission, and this permission is not available in MV3.

Is there any other option to override that header or load a website in an iframe in offscreen doc, without causing warnings (new permissions)? Currently we have "["storage", "alarms", "webRequest", "scripting", "offscreen"]" and broad host permission because of the content script settings.

[Oliver Dunk]

Hi,

Have you looked at the declarativeNetRequestWithHostAccess permission? This is the same as `declarativeNetRequest`, but only allows you to perform actions on hosts you have permissions for. As a result, it does not generate a permission warning.

We did have some documentation for this but it looks like it might've been lost in the migration to our new site. I'll take a look at that.

Oliver Dunk | DevRel, Chrome Extensions | https://developer.chrome.com/ | London, GB

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/55b1f8ab-3550-4b14-93cc-e6fb153aa0e9n%40chromium.org.

[wOxxOm]

Here's a DNR example with a caveat about service workers: https://stackoverflow.com/a/69177790

[kazag89]

It works! Thank you!

I saw declarativeNetRequestWithHostAccess but for some reason I assumed it would trigger warnings as well.

w0xx0m, not sure if I follow the SW warning though. Does it apply to iframes loaded inside offscreen documents? I didn't notice any issues. How can I reproduce a problem to know if I have to apply the fix you proposed?

[wOxxOm]

It applies to sites that use a service worker e.g. X/twitter.

[Oliver Dunk]

Quick update here, we've just updated the site with some documentation on the permissions again: https://developer.chrome.com/docs/extensions/reference/api/declarativeNetRequest#declarativenetrequestwithhostaccess

Thanks for helping us spot that!

Oliver Dunk | DevRel, Chrome Extensions | https://developer.chrome.com/ | London, GB