How can I make sure I'm accessing the most up-to-date storage state within other event handlers?

[Robbi]

Please see the following code:

//manifest.json
{
    "manifest_version": 3,
    "name": "foobar",
    "short_name": "fb",
    "version": "0.0.0.1",
"background": {
"service_worker": "sw.js"
},
"permissions": ["alarms", "storage"]
}
//-------------------------------------------------------------------------------
//SW
chrome.runtime.onInstalled.addListener(d =>
chrome.storage.session.set({ 'foo': 1, 'bar': 2 })
);

function sleep(s) {
return new Promise(ok => {
setTimeout(_ => {
x = 2 + Math.floor(Math.random() * 100);

chrome.storage.session.set({ 'foo': 3,    'bar': 4 });

ok(s)
},    s * 1000)
})
};

var x = 0;

chrome.alarms.onAlarm.addListener(async a => {
console.log("Alarm triggered");

x = 1;

await sleep(10); /* Think this line as a very slow storage write operation. */
/*  all in all chrome.storage.*.set is an asynchronous designed method which has its own running times */
/* If I do use "await" in the previous line I will see a random number in the console after 10 secs
  If I don't use "wait" in the above line I will immediately see X = 1 in the console.
       */
// Now, a message could arrive at every moment

console.log('X can be 1 or a random number. Now its value is', x)
});

chrome.runtime.onMessage.addListener((a, b, c) => {
/* By querying the value of X via message I will get different values depending on
    whether my asynchronous command "sleep(10)" has come to an end or not. */

console.log('With a message I check the current value of X. X now is ', x);

/* With the same message I check the values of the storage */
chrome.storage.session.get(['foo','bar'], itm => console.log(`foo: ${itm.foo}, bar: ${itm.bar}`) )

})

//---------------------------------------------------------

Let's say the SW responds to an alarm and that a storage variable is set in the chrome.alarms.onAlarm handler.
Now, during the saving proccess another event occurs, for example a message (send from an extension page or from the console).
How can I be sure the writing operation reachs the end before the onMessage handler starts to read the same storage?

To make some tests we can open the manifest file (chrome-extensio://<ext-id>/manifest.json) and from that console
we can create a new alarm with something like:

chrome.alarm.create('dummy', {'when': Date.now() + 3000})

After that alarm is triggered  we can test the value of X (and the storage) sending several messages at different times  from the same console like:

chrome.runtime.sendMessage('hi')

and getting different results.

A write operation into the storage can be considered almost instantaneous in many cases, but when we are facing with very large and/or nested objects, the saving times can grow significantly.
So how can I be sure that the reading phase from the storage through another event returns always the most updated values?
What could be the best technique we can use to prevent/avoid this kind of issue (if it actually exists)?

TIA

[Stefan Van Damme]

Hi Robbi,

If you want to make sure that the storage write operation is completed before the onMessage handler starts reading from it, you can use asynchronous programming techniques such as promises or async/await. And the use of setTimeout is not recommended for background server workers, as these APIs can fail in service workers. For more information, see this Chrome developer documentation:

https://developer.chrome.com/docs/extensions/migrating/to-service-workers/#convert-timers

To learn more about the Promise, see this web documentation:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise

Then you have something such as this:

function setSessionStorage(key, value) {

return new Promise((resolve, reject) => {

chrome.storage.session.set({ [key]: value }, () => {

if (chrome.runtime.lastError) {

reject(chrome.runtime.lastError);

} else {

resolve();

}

});

});

}

await setSessionStorage('foo', 3);

Thanks,

Stefan vd

[Robbi]

Hi Stefan,

The "setTimeout" I placed was intended to "exaggerate" the concept of a promise (and storage.*.set method return a promise).

What I wanted to point out is that the word "await" (as you also put in front of "setSessionStorage"), DOES NOT guarantee that any other event that arrives in the meantime is actually waiting for that promise to read the storage.
I mean, "await" word guarantee that next line will be execute after the relative promise will be fulfilled, but the execution flow may jump everywhere to serve an incoming event or to continue a routine previosly paused by another promise.
Am I wrong?

A while ago I developed a benchmark-extension to compare the performance between indexedDB and asynchronous storages.
If you want to take a look you can find it at this address: G-Drive Link

By doing some tests with my PC (quite obsolete) I found that writing a quite big and complex object in the storage can take up to a few seconds. (i.e 25k items takes almost 3 secs).
My question returns to being the same: "how do I know if the storage read, (requested by an event that occurred during the storage write), occurs only when the storage is committed?"

A possible solution could be to wait for the writing to complete before reading again (exactly the opposite of the solution you proposed).
Something like this:

/*--------------------------------------------------------------------*/

chrome.runtime.onInstalled.addListener(dtls => {
chrome.storage.local.set({

'foo': 1,
'bar': 2
})
});

var savingPromise;

function writeOnStorage(obj2Save, storage) {
savingPromise = new Promise(ok => {
chrome.storage[storage].set(obj2Save, _ => {
setTimeout(_ => {
savingPromise.isPending = false;
console.log('Saved in storage.');
ok()
}, 25000)
})
});
savingPromise.isPending = true
}

chrome.alarms.onAlarm.addListener(a => {
var o2s = {
'foo': 3,
'bar': 4
};
writeOnStorage(o2s, 'local')
});
chrome.alarms.create('dummy', {'when': Date.now() + 5000}); //after 5 secs I'll save new values.

// after 5" and before 30" make some tries sending one (or more) message from an extension page or from manifest.json page.
//i.e. chrome.runtime.sendMessage('ciao')

chrome.runtime.onMessage.addListener(async (a, b, c) => {
console.log("I'm inside runtime.onMessage handler")
if (savingPromise instanceof Promise && typeof savingPromise.then === 'function' && savingPromise.isPending) {
console.log("I have to wait a little 'cause there's a live saving op!")
await savingPromise;
console.log("Ok, now I can proceed")
} else
console.log("I'm lucky, there is promise to wait");

/* now, if from now my code will be syncronous, I will be be pretty sure that
I will be not suspended by any storage writing ops */

console.log('END')
})

/*--------------------------------------------------------------------*/

I opened this thread to pay attention to this kind of problem.

A write operation in the storage always has execution times although, in standard cases, they can be considered negligible.

So in my opinion there may be an edge case in which the storage is read during a write operation and therefore I will have to expect an unwanted value (unless I set logics that avoid it)

[Paul Marks]

You can solve this problem by reading from storage only once:

Service worker top level:

let storageMap = {};  // An example global state variable

const storageReady = (async () => {

  const items = await chrome.storage.session.get();
  // Use 'items' to populate storageMap.

  // This is also a good place to clean up any inconsistencies

  // in the data due to lost writes, etc.

})();

Reading state:

  await storageReady;

  let bar = storageMap[foo];

Writing state:

  await storageReady;

  storageMap[foo] = bar;

  await chrome.storage.session.set({[foo]: bar})

So your state is primarily maintained in global variables, and any changes get backed up to storage.session as soon as possible.  When the service worker (re)starts, all handlers wait for the backup to be restored before doing anything.

[Paul Marks]

When using this pattern, it's also good to prevent multiple simultaneous writes to the same storage key, so that old data can never overwrite new data.  Search https://github.com/pmarks-net/ipvfoo/blob/master/src/background.js for this.#dirty to see an example.

[Jackie Han]

I don't fully understand the original question.

If your read and write operations are executed only in one code instance, Oliver/Stefan/Paul's answers should all work. You can control it with guarantee.

But if your read and write operations are executed simultaneously in multiple code instances, for example they run in 

(a) the service worker instance 

(b) Tab-page-1-instance-1 

(c) Tab-page-1-instance-2 

(d) Tab-page-2-instance-1

the above methods cannot guarantee the consistency of reading and writing.

In this case, I recommend:

Solutions 1: Move read and write operations in one place, especially the write.

Solutions 2: Use Web Lock API. It supports mutex lock across different tabs and workers.

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/d9957990-b2ea-4644-aa53-722e0a489085n%40chromium.org.

[Robbi]

The problem exposed in this thread (maybe exposed not in the best way) is that, sometimes there can be extremely close events whose execution order is not always predictable and therefore the risk of reading a storage variable too soon could affect the logic of the extension.

For instance:

Let say we have a extension that modifies the new tab

Now we open a couple of "newtab"

We change the browser settings so that it "starts where it left off" (i.e. restarts with the last tabs open)

I restart my browser.

Now I have to understand who starts-finishes first, the service worker or the pages of the last session?

Let's also include an alarm that was supposed to go off while the browser was closed and that now, after a restart, wants to go off.

We also put in a couple of tabs events (onActivated onUpdated ...)

With this system I've been able to handle it quite well so far, I'm afraid, however, that sooner or later, based on unforeseen combinations of random events, I will fall into some trap.

Now allow\forgive me a meta-philosophical dissertation:

"It seems to me that all this asynchronous programming is causing us to jump through hoops to write code that tries so hard to be synchronous.

Will we have taken the right path?"

@Jackie Han

> Solutions 1: Move read and write operations in one place, especially the write.<

It will be very difficult to apply this approach because many storage items are read\written also based on the DOM events of the extension page.

If I wanted to delegate all reading\writing ops to the SW, I would have to more than double the message exchange from for the service worker.

I do not exclude this solution a priori, but it seems to me a really extreme solution.

Solutions 2: Use Web Lock API. It supports mutex lock across different tabs and workers.

I promise I will read the documentation you suggested tomorrow with a fresh mind :-)

[Robbi]

Reading the Web Lock API documentation suggested brought me  Web Locks API Explained  first

and then here: Event-Loop.

On this last page there is a sentence a few lines below this anchor which says: "All microtasks are completed before any other event handling or rendering or any other macrotask takes place."

Based on this article I assumed that writing to storage (chrome.storage.*.set) is a microtask as it returns a promise and therefore this promise "is completed before any other event handling..." such as runtime.onMessage?

If so, then all my prEmises go to hell (consequently also all the proposed solutions) because I will always be that the storage writing operation will be completed before any event is triggered.

Did I get it right?

[wOxxOm]

>  Based on this article I assumed that writing to storage (chrome.storage.*.set) is a microtask

No. Having a Promise result doesn't imply it runs as a microtask, there's no connection whatsoever. In this case it runs as a task because it entails IPC (inter-process communication), which is required for all asynchronous `chrome` API.

A microtask for a Promise will be used when you `await` or `then` it.

[wOxxOm]

[continued] ... on top of the task for the actual work.

[Robbi]

Hi wOxxOm, I don't think I got it right.

Could you please give an (some) example or give me a reliable link on the subject: "event-loop macro\microtask"

I understand that the article I've reported is (perhaps) inaccurate or too simplistic.

As Socrate said: "I know I don't know" or "I understand that I still have to understand some other stuff".

[wOxxOm]

Example of what? It's described in the article you've linked about the event loop, but of course there may be other/better articles, so don't stop looking. BTW, regarding the initial problem, you can read the storage directly just once at the start of the script into a variable, then you'd only write the changes (example) and optionally update the variable in chrome.storage.onChanged event in case you can also change this data in an extension page, but I'd probably suggest not doing it and instead send a message to the background script with the new data so that it writes the storage and updates the data variable directly, which should be faster than using onChanged.

[Robbi]

I do something very close to what is shown in your S.O. example, but maybe I do even more.
I basically read the storages not only in the main thread, but also in EVERY chrome event handler (especially the runtime.onMessage event).
This is because events can be triggered not only when SW is asleep but also when it is already awake.
But that's not enough for me; If I see the promise relating to the storage reading is pending, I wait for it to be completed and I don't take its output, but I read the storage again.

> This is because I'm not sure that during the pending read, in any part of the code (main thread or event handler) the storage hasn't changed in the meantime. <

( @Stefan in his answer states that it would suffice to create a new promise, wrap the storage writing method inside it and "await" that promise to be sure that the writing is completed before any other chrome API event can occur.

PLEASE, someone convince me of this, because I have STRONG DOUBTS. )

In any case, I wait for the previous promise (if any) to fulfill otherwise the new instance of my read-and-write-in-cache procedure would risk destroying the already created cache object.

I'm still in the process of migrating (and testing at the same time) so only time will tell me right or wrong.
I'm trying to find a compromise between writing as little code as possible (but easily readable code) and writing pitfall-proof code.
Exchanging messages to the SW is a technique I'm already doing when I change items of crucial importance for the extension. I send a message and on the other side I read the storage again.
However, if I modify a single storage item (item(s) of secondary importance), I would tend not to exchange a message, also because it would all become a "chitchat", which seems impractical and also inelegant to me.

But maybe I should focus more on building a system that LOCKS STORAGE AT THE WRITE STAGE and then releases it when it's done.
I could do this with a globally accessible promise that I create when I go to write to the storages and that I check during the subsequent read.
So when the event fires and I go to read the storage I ask myself: "Is there a write in progress? If so, wait for its conclusion and then read, otherwise read immediately."
Alternatively, if I don't want to use a promise, I could approach using this Web lock API which looks promising to me.
I could perhaps place an exclusive lock on the storage resource so that this resource is not read when it is written.

[wOxxOm]

>  not only in the main thread, but also in EVERY chrome event handler (especially the runtime.onMessage event).

The entire code runs in one thread inside one context (service worker), there are no multiple threads. You may be mistaking promises for threads, which some people do even those who understand this is wrong.