Chrome.storage.local is it 5mb or 10, I'm at 115% ??

[Tom Lahey]

I've read a lot of posts about the 5mb limit and and stiff confused. On this page https://groups.google.com/a/chromium.org/g/chromium-extensions/c/6_SoqKPkZpI/m/RyUCiZCyAwAJ  There was this text "HTML5 localStorage is always limited to 5MB (computed as a combined string length which actually means 10MB physical bytes on disk because strings are 2-byte unicode in JS)."

I have a storage viewer built into my extensions (hidden by default) It reports; 

Total Storage: 5.79MB f 5MB (5928.11 KB) - 115.78% used

So, can I go more, or am I already in trouble? It looks like unlimited storage in manifest will not help. Any help or clarification is appreciated.

 // Populate dropdown with storage keys and their sizes

        chrome.storage.local.get(null, function(items) {
            let totalSize = 0;
           
            // Add total size as first option
            const totalOption = document.createElement('option');
            totalOption.value = '';

            Object.keys(items).forEach(key => {
                const option = document.createElement('option');
                // Calculate size of this item
                const size = new TextEncoder().encode(JSON.stringify(items[key])).length;
                totalSize += size;
                const sizeInKB = (size / 1024).toFixed(2);
               
                option.value = key;
                option.text = `${key} (${sizeInKB} KB)`;
                select.appendChild(option);
            });

            // Update total size option
            const totalInMB = (totalSize / (1024 * 1024)).toFixed(2);
            const totalInKB = (totalSize / 1024).toFixed(2);
            const percentUsed = ((totalSize / (5 * 1024 * 1024)) * 100).toFixed(2);
            totalOption.text = `Total Storage: ${totalInMB}MB / 5MB (${totalInKB}KB) - ${percentUsed}% used`;
            select.insertBefore(totalOption, select.firstChild);

[Oliver Dunk]

Hi Tom,

While the `unlimitedStorage` permission actually will help with web storage APIs, localStorage is unfortunately an exception that is hardcoded to always be limited in space.

Have you considered using the extension storage APIs or something like IndexedDB?

Oliver Dunk | DevRel, Chrome Extensions | https://developer.chrome.com/ | London, GB

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/0288b391-2d90-49c7-b53f-c6f15ec58642n%40chromium.org.

[Tom Lahey]

Thanks, but per the comment I read, is it actually 10mb not 5mb?

[woxxom]

10 million bytes, but 5 million unicode characters and this is what javascript string length reports.

[woxxom]

Scratch that. I looked at the code and you're NOT using HTML5 localStorage, you're using chrome.storage.local, which is misleadingly named as local storage in the documentation, which has been always confusing lots of people.

There's no 5MB or 10MB limit on chrome.storage.local, at least in Chromium. Other browsers may impose a limit.

[Mythical 5th]

I've never checked to see if the limit is implemented, but this is what the docs say

https://developer.chrome.com/docs/extensions/reference/api/storage#properties_2

QUOTA_BYTES
10485760
The maximum amount (in bytes) of data that can be stored in local storage, as measured by the JSON stringification of every value plus every key's length.

[woxxom]

Indeed, I was wrong about chrome.storage.local. I incorrectly assumed it was the storage used by extensions I had without unlimitedStorage permission, but turns out those extensions were using IndexedDB, which doesn't have a limit in either case.

[woxxom]

BTW, there's no need for TextEncoder and UTF8 when calculating the size of the value in chrome.storage.local, because Chromium uses standard unicode strings with a simple formula JSON.stringify(keyName).length +  JSON.stringify(value).length, https://crsrc.org/extensions/browser/api/storage/settings_storage_quota_enforcer.cc;l=40;drc=0417caf4

[woxxom]

Correction, the key doesn't use JSON, i.e. the linked formula is keyName.length +  JSON.stringify(value).length

[Mythical 5th]

Is anyone from the docs team reading this? There are two ways to interpret the documentation.

> measured by the JSON stringification of every value plus every key's length

Correct interpretation:

((the JSON stringification of every value) plus every key)'s length

Alternative interpretation:

(the JSON stringification of (every value plus every key))'s length

Given that we commit data to storage as objects, the wrong interpretation makes intuitive sense.

[woxxom]

I've linked the actual formula in my post: https://crsrc.org/extensions/browser/api/storage/settings_storage_quota_enforcer.cc;l=40;drc=0417caf4

It's keyName.length + JSON.stringify(value).length

[Mythical 5th]

> It's keyName.length + JSON.stringify(value).length

It is, but I was pointing out that the documentation is ambiguous.

[Oliver Dunk]

Thanks for pointing this out!

This is actually coming from the Chromium source, so if anyone would like to suggest alternative wording you should be able to submit a patch with the change: https://source.chromium.org/chromium/chromium/src/+/main:extensions/common/api/storage.json;l=244;drc=753b4707b454b3b99f40509aa8a7608feb8ed5c0

Oliver Dunk | DevRel, Chrome Extensions | https://developer.chrome.com/ | London, GB