Intent to Ship: Storage Usage and Quota Estimation API

80 views
Skip to first unread message

Joshua Bell

unread,
May 17, 2017, 6:29:54 PM5/17/17
to blink-dev

Contact emails

Spec



Summary

Introduce navigator.storage.estimate(). Returns a Promise resolving to {usage, quota} values in bytes, only callable in secure contexts.

Link to “Intent to Implement” blink-dev discussion


Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Debuggability

Nothing special - the API is usable from the console in secure contexts. Exposing quota details is on the list of desired dev tools improvements for the Application panel.

Interoperability and Compatibility Risk

Chrome has some history here, having shipped 2 previous web-facing quota management APIs (and experimented with another one), which were not adopted by other browsers. More details in the I2I.

We have reasonable confidence that this one will stick: W3C TAG reviewed it, Firefox has implemented, and it's a strict subset of what Chrome had previously. (You can trivially polyfill it on the previous API.)

Edge: No signals
Firefox: In development (currently behind a flag) - target ship is FF57 (per https://wiki.mozilla.org/TPE_DOM/Storage#Roadmap)
Safari: No signals
Web developers: Positive - frequent request, "legacy" APIs have reasonably high usage (0.001% of page loads)

Is this feature fully tested by web-platform-tests?

Yes, with submissions from both Chrome and Firefox: https://github.com/w3c/web-platform-tests/tree/master/storage

OWP launch tracking bug

https://crbug.com/603992

Entry on the feature dashboard

But what about that "Intent to Remove webkitStorageInfo" or something?

Oh, you mean this one ? tl;dr: that was about removing the oldest of the Chrome-only APIs (which has had a deprecation warning pointing at the other Chrome-only API for years). We're still waiting on a HTTPArchive run with use counters in stable to qualify the usage and if it's really safe to remove. Once we have that we can decide on (a) remove or (b) just update the deprecation message to point at this new API. 

Chris Harrelson

unread,
May 18, 2017, 12:18:06 PM5/18/17
to Joshua Bell, blink-dev
LGTM1

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAD649j7iH%2ByPhw3diPWwsNVsonXc%3D-cNPLB4yPB7qXYx6cw-_A%40mail.gmail.com.

Philip Jägenstedt

unread,
May 22, 2017, 8:22:52 AM5/22/17
to Chris Harrelson, Joshua Bell, Mike West, blink-dev
LGTM2, but I found https://github.com/whatwg/storage/issues/31 + https://github.com/whatwg/storage/issues/35 and am not sure if there's a problem here? Has there been a security review of this?

Misc. trivial feedback:

I eyeballed the IDL and found "Should have [SecureContext] on interface" on StorageManager, is that possible to fix now or blocked on something?

Of the open spec issues, would any of these have an impact on navigator.storage.estimate()?
On the wpt side, great to see the shared test suite, and it looks like we're using that exclusively for navigator.storage.estimate(), yay! I found https://github.com/w3c/web-platform-tests/issues/5340, I take it that isn't tested or doesn't make sense in our implementation?

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

Joshua Bell

unread,
May 22, 2017, 1:14:10 PM5/22/17
to Philip Jägenstedt, Chris Harrelson, Mike West, blink-dev
On Mon, May 22, 2017 at 5:22 AM, Philip Jägenstedt <foo...@chromium.org> wrote:
LGTM2, but I found https://github.com/whatwg/storage/issues/31 + https://github.com/whatwg/storage/issues/35 and am not sure if there's a problem here? Has there been a security review of this?

Re: #31 - there is work going on to obfuscate the sizes of cross origin resources. Note that the same data is exposed by the current Chrome-only quota APIs, so as far as Chrome is concerned this API is not revealing any new data. 
 
Re: #35 - not explicitly; again, same data being revealed by existing APIs.


Misc. trivial feedback:

I eyeballed the IDL and found "Should have [SecureContext] on interface" on StorageManager, is that possible to fix now or blocked on something?

Not a blocker; this was a limitation in the bindings code generator present from when persist()/persisted() were shipped. IIRC, the existence of the StorageManager interface currently 'leaks' into non-secure contexts but the attribute/methods do not. I saw a CL go by recently that may add this support.
 

Of the open spec issues, would any of these have an impact on navigator.storage.estimate()?
Cookies and DOMStorage don't count towards Quota in Blink. Both have separate limits and LRU logic. That's something we'd like to revisit over time; the API doesn't preclude accounting changes. Cookies get particularly tricky as they are not origin-scoped.
The "boxes" concept in the spec is currently just spec fiction; an origin has exactly one "box" that everything storage-related goes into. There has been the thought that an origin could create multiple boxes for ease of resource management. This API (navigator.storage.estimate()) would apply to the entire origin; there would presumably be a way to ask a box how big it is, and we'd need a way to refer to usage of the "default box" distinct from total usage.
 
On the wpt side, great to see the shared test suite, and it looks like we're using that exclusively for navigator.storage.estimate(), yay! I found https://github.com/w3c/web-platform-tests/issues/5340, I take it that isn't tested or doesn't make sense in our implementation?

Isn't tested on the Blink side. :(
 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAD649j7iH%2ByPhw3diPWwsNVsonXc%3D-cNPLB4yPB7qXYx6cw-_A%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

Philip Jägenstedt

unread,
May 23, 2017, 7:16:59 AM5/23/17
to Joshua Bell, Chris Harrelson, Mike West, blink-dev
On Mon, May 22, 2017 at 7:14 PM Joshua Bell <jsb...@chromium.org> wrote:
On Mon, May 22, 2017 at 5:22 AM, Philip Jägenstedt <foo...@chromium.org> wrote:
LGTM2, but I found https://github.com/whatwg/storage/issues/31 + https://github.com/whatwg/storage/issues/35 and am not sure if there's a problem here? Has there been a security review of this?

Re: #31 - there is work going on to obfuscate the sizes of cross origin resources. Note that the same data is exposed by the current Chrome-only quota APIs, so as far as Chrome is concerned this API is not revealing any new data. 
 
Re: #35 - not explicitly; again, same data being revealed by existing APIs.

Is there a tracking bug for that obfuscation, that wouldn't be vulnerable to adding the same resource many times with '?'+Math.random() appended? As long as it's very unlikely that we won't want to fix it by removing navigator.storage.estimate() again, then it sounds fine.
 
Misc. trivial feedback:

I eyeballed the IDL and found "Should have [SecureContext] on interface" on StorageManager, is that possible to fix now or blocked on something?

Not a blocker; this was a limitation in the bindings code generator present from when persist()/persisted() were shipped. IIRC, the existence of the StorageManager interface currently 'leaks' into non-secure contexts but the attribute/methods do not. I saw a CL go by recently that may add this support.

Judging just by the IDL files, it looks like there'll be no way to get an instance of StorageManager in non-secure contexts, but StorageManager.prototype.estimate and friends will be visible, although useless.

Credential.idl has SecureContext on an interface, so perhaps it now works. Or not.
 
Of the open spec issues, would any of these have an impact on navigator.storage.estimate()?
Cookies and DOMStorage don't count towards Quota in Blink. Both have separate limits and LRU logic. That's something we'd like to revisit over time; the API doesn't preclude accounting changes. Cookies get particularly tricky as they are not origin-scoped.

I see, so it sounds like neither spec nor Blink counts cookies in the quota, and having an open spec issue for that is fine.
The "boxes" concept in the spec is currently just spec fiction; an origin has exactly one "box" that everything storage-related goes into. There has been the thought that an origin could create multiple boxes for ease of resource management. This API (navigator.storage.estimate()) would apply to the entire origin; there would presumably be a way to ask a box how big it is, and we'd need a way to refer to usage of the "default box" distinct from total usage.

I see, since estimate() doesn't peek in individual boxes, this is currently a non-issue then.
 
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAD649j7iH%2ByPhw3diPWwsNVsonXc%3D-cNPLB4yPB7qXYx6cw-_A%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

Joshua Bell

unread,
May 23, 2017, 1:08:47 PM5/23/17
to Philip Jägenstedt, Chris Harrelson, Mike West, blink-dev
On Tue, May 23, 2017 at 4:16 AM, Philip Jägenstedt <foo...@chromium.org> wrote:
On Mon, May 22, 2017 at 7:14 PM Joshua Bell <jsb...@chromium.org> wrote:
On Mon, May 22, 2017 at 5:22 AM, Philip Jägenstedt <foo...@chromium.org> wrote:
LGTM2, but I found https://github.com/whatwg/storage/issues/31 + https://github.com/whatwg/storage/issues/35 and am not sure if there's a problem here? Has there been a security review of this?

Re: #31 - there is work going on to obfuscate the sizes of cross origin resources. Note that the same data is exposed by the current Chrome-only quota APIs, so as far as Chrome is concerned this API is not revealing any new data. 
 
Re: #35 - not explicitly; again, same data being revealed by existing APIs.

Is there a tracking bug for that obfuscation, that wouldn't be vulnerable to adding the same resource many times with '?'+Math.random() appended? As long as it's very unlikely that we won't want to fix it by removing navigator.storage.estimate() again, then it sounds fine.

Yes. I've CC'd you on the issue.
 
 
Misc. trivial feedback:

I eyeballed the IDL and found "Should have [SecureContext] on interface" on StorageManager, is that possible to fix now or blocked on something?

Not a blocker; this was a limitation in the bindings code generator present from when persist()/persisted() were shipped. IIRC, the existence of the StorageManager interface currently 'leaks' into non-secure contexts but the attribute/methods do not. I saw a CL go by recently that may add this support.

Judging just by the IDL files, it looks like there'll be no way to get an instance of StorageManager in non-secure contexts, but StorageManager.prototype.estimate and friends will be visible, although useless.

Correct.
 

Credential.idl has SecureContext on an interface, so perhaps it now works. Or not.

https://chromium-review.googlesource.com/c/508328/ seems to be the CL I was thinking of. Still in review.
 
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAD649j7iH%2ByPhw3diPWwsNVsonXc%3D-cNPLB4yPB7qXYx6cw-_A%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

Rick Byers

unread,
May 24, 2017, 11:47:30 PM5/24/17
to Joshua Bell, Philip Jägenstedt, Chris Harrelson, Mike West, blink-dev
LGTM3

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAD649j7jgthZDq03Mf359Aik1Ye6V-aAsW0Cp3hxG7rTUM939w%40mail.gmail.com.

Reply all
Reply to author
Forward
0 new messages