Working with files in the background

[Vladimir Yankovich]

Colleagues, I need your advice. I am making an extension that displays media content in a new tab. Sometimes it can be large files, photos up to 7 megabytes, videos up to 100 megabytes. We use MV3. 

Now the mechanism is as follows. The user does not have to wait for the download; we download the files in the background. Next, we put them in the extension storage. We are waiting for the user to open a new tab; at this moment, we create thumbnails of our files through the canvas and move the files themselves to the user's file system. And we save the records about the files in the indexDB.

Everything works, but we are not satisfied that to complete the preparation of the file completely, we need to wait for the user to open the tab. We need a windows object to transfer files to the file system. But we want to perform all operations in the background to already wait for the user with ready content.

We tried to work with other storages, but they are essential for us the speed of displaying files, and we did not find anything comparable in speed to the file system. Are we missing something?

Or maybe someone will tell us how to call some windows from the background in which we can finish preparing the content: creating thumbnails in the canvas and moving to the file system?

[Jackie Han]

Background Fetch API:

• https://developers.google.com/web/updates/2018/12/background-fetch
• https://github.com/WICG/background-fetch

This is an API for a special scenario like your case. But I have never really used it, and don't know if it works in extensions. You can give it a try.

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/df353ab9-f07d-473a-98bc-945a670c4782n%40chromium.org.

[Jackie Han]

Maybe I didn't fully understand your question. Background Fetch API is used to download files in the background, it is not related to the file system.

[Vladimir Yankovich]

Jackie Hahn, thanks for your answer. I will study the documentation, maybe this is what we need.

To be on the safe side, I briefly cite my question again:

1. Is it possible to make Chrome Storage work with large files as fast as the File System (https://web.dev/file-system-access/) works with them?

2. If not, is it possible, after the file has been downloaded in background; to call from background some invisible window in which we a) create a thumbnail of our media file using the canvas b) transfer the file to the file system?

[Jackie Han]

1. Do you mean comparing the performance of IndexedDB and File System Access API for large files?

I don't know the real situation. Maybe you can share some actual test data, or report an issue to the Browser.

2. a) create a thumbnail of our media file using the canvas in the background or invisible window.

Solution 1: use OffscreenCanvas in service worker?

Solution 2: create thumbnails in your server side?

[wOxxOm]

• IndexedDB for images
• OffscreenCanvas for thumbnails
• Use virtual URLs like chrome-extension://extensionId/fs/thumb/123.png in your visible page
• Use a `fetch` event listener in the service worker so it will process these virtual URLs and use the standard service worker process of maintaining `caches` and generating data as necessary as a transferable stream.
• Use BroadcastChannel to transfer ArrayBuffer and other structured-clone compatible types directly between the service worker and the open page.

> make Chrome Storage work with large files as fast as the File System 

No, chrome.storage is quite inefficient for large stuff because it was implemented in a very simplified manner to accommodate for rather small amounts of data, it was never optimized since and I'd say it's unlikely to get optimized in the foreseeable future as there are still hundreds of bugs waiting to be fixed for years.

> some invisible window

No, currently ManifestV3 doesn't have anything like that so your extension will have to open a visible inactive window or a tab, which is irredeemably terrible but there's no alternative. Maybe it'll be implemented in the future, see https://crbug.com/1128240.

[wOxxOm]

To clarify, by "standard service worker process" I referred strictly to the `fetch` event and handling of `caches`. You don't need to register the worker as the browser does it automatically when the extension is installed.

[Vladimir Yankovich]

Colleagues, thanks for your answers. It will take me a little time to figure it out, talk to the team and try to implement these ideas. I think OffscreenCanvas and virtual URLs are helpful to us. And of course it's a pity that MV3 deprives us of background windows :(

[get.yo...@gmail.com]

On Monday, April 26, 2021 at 10:40:03 AM UTC-4 wOxxOm wrote:

> some invisible window

No, currently ManifestV3 doesn't have anything like that so your extension will have to open a visible inactive window or a tab, which is irredeemably terrible but there's no alternative. Maybe it'll be implemented in the future, see https://crbug.com/1128240.

I wasn't aware of this headless window proposal for MV3.

This is quite interesting. It reminds me of the following comment posted on this thread on August 2020:

On Monday, August 10, 2020 at 12:08:50 AM UTC-4 get.yo...@gmail.com wrote:

A background page is just a hidden browser window. Actually, it's not even a window, it's just the rendered process.

So, implementing a background page is as easy as this:

1. Create a browser window and load the extension page in it

2. Hide the window

3. Exclude the window/tab from the chrome.windows.getAll and chrome.tabs.query APIs.

That's it. I've just implemented the background page functionality for you Chrome team. It's really that simple because the background page is no different from any other extension page.

If this headless window proposal is actually implemented, then we'll have the MV2 background page back. And, just like that, MV3 will start looking much better.

I have my fingers crossed.

[Jackie Han]

(Not related to the original question)

Using fetch event in service worker as "virtual URL or Files" is an interesting idea. All files in the extension package are already offline (local files), so handling this event is usually unnecessary. But as virtual files, they can be dynamically generated contents, although I've never used them that way before.

[Jackie Han]

I did a little experiment for "virtual Files". It works well on the extension pages, but can't work like below:

• chrome.scripting.executeScript({ files: ['virtual.js'] }); , (Error: Could not load file)
  
• fetch(chrome.runtime.getURL("virtual.js")); , in content_script from web page to fetch virtual file as "web_accessible_resources" (return net::ERR_FILE_NOT_FOUND)