Intent to Ship: IndexedDB getAllRecords() and direction option for getAll()/getAllKeys()

[Chromestatus]

Contact emails

ste...@microsoft.com, rah...@microsoft.com, evan...@microsoft.com

Explainer

https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/IndexedDbGetAllEntries/explainer.md

Specification

https://github.com/w3c/IndexedDB/pull/461

Summary

This feature adds the getAllRecords() API to IndexedDB's IDBObjectStore and IDBIndex. It also adds a direction parameter to getAll() and getAllKeys(). This functionality enables certain read patterns to be significantly faster when compared to the existing alternative of iteration with cursors. One key workload from a Microsoft property showed a 350ms improvement. getAllRecords() effectively combines getAllKeys() and getAll() by enumerating both primary keys and values at the same time. For an IDBIndex, getAllRecords() also provides the record's index key in addition to the primary key and value. getAllRecords() can optionally return records in either ascending or descending order. This option is also back-ported to the existing operations getAll() and getAllKeys().

Blink component

Blink>Storage>IndexedDB

Search tags

IndexedDB, getAllRecords, IDBObjectStore, IDBIndex

TAG review

https://github.com/w3ctag/design-reviews/issues/1117

TAG review status

Pending

Risks

Interoperability and Compatibility

Overloading getAll() and getAllKeys() to accept the IDBGetAllOptions dictionary introduces compatibility risk. Prior to this proposal, when passed a dictionary argument, both getAll() and getAllKeys() throw an exception after failing to convert the dictionary to a key range. After the overload, getAllKeys() and getAll() will no longer throw for dictionary input. When the IDBGetAllOptions dictionary initializes with its default values, it creates a query that retrieves all of the keys or values from the entire database.

Gecko: Positive (https://github.com/mozilla/standards-positions/issues/1261)

WebKit: No signal (https://github.com/WebKit/standards-positions/issues/521)

Web developers: Positive (https://nolanlawson.com/2021/08/22/speeding-up-indexeddb-reads-and-writes) Developers have reported the limitations addressed by getAllRecords(), including: "You cannot build a paginated cursor in descending order." The following includes another example where getAll() could help but needs to retrieve the index key and primary key: https://stackoverflow.com/questions/44349168/speeding-up-indexeddb-search-with-multiple-workers

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

None

Debuggability

None

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)?

Yes

Is this feature fully tested by web-platform-tests?

Yes

https://wpt.fyi/results/IndexedDB?label=master&label=experimental&aligned&q=getallrecords

Flag name on about://flags

Finch feature name

IndexedDbGetAllRecords

Rollout plan

Will ship enabled for all users

Requires code in //chrome?

False

Tracking bug

https://issues.chromium.org/issues/40746016

Sample links

https://patrickbrosset.com/articles/2024-11-19-even-faster-indexeddb-reads-with-getallrecords
https://microsoftedge.github.io/Demos/idb-getallrecords

Estimated milestones

Shipping on desktop	141
DevTrial on desktop	133
Shipping on Android	141
Shipping on WebView	141

Anticipated spec changes

Open questions about a feature may be a source of future web compat or interop issues. Please list open issues (e.g. links to known github issues in the project for the feature specification) whose resolution may introduce web compat/interop risk (e.g., changing to naming or structure of the API in a non-backward-compatible way).

None

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5124331450138624?gate=5186127942909952

Links to previous Intent discussions

Intent to Prototype: https://groups.google.com/a/chromium.org/g/blink-dev/c/zLHOayUfTcw/m/Z13e2qGhBQAJ

This intent message was generated by Chrome Platform Status.

[Alex Russell]

Per today's API OWNERS, there's some question about if and when the spec change might land.

Evan, do you have insight into that process?

Best,

Alex

[Evan Stade]

Thanks for the ping. I'm happy with the status of the PR and I think it's ready to merge.

[Steve Becker]

I merged the W3C IndexedDB getAllRecords() spec PR after reviews from Evan Stade (Microsoft Edge) and Steffen Larssen (Ladybird).  Please file an issue in the GitHub if you have any additional feedback and I'll investigate.  A few months ago, I volunteered to be an editor of the W3C IndexedDB spec.

The W3C Tag Review also completed after I sent this intent to ship.  

Thank you for reviewing!

- Steve

[Chris Harrelson]

Great!

LGTM1

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/8e5d76bd-4818-420c-a1eb-1bcd465bd1edn%40chromium.org.

[Mike Taylor]

LGTM2

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw8ppcado8_EdOYQF70fxy9mMZ%2B2VYLmgkCHrGnxTZefmg%40mail.gmail.com.

[Daniel Bratell]

LGTM3

/Daniel

To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/96411e69-73f6-42bf-93ba-f8de39a9f27c%40chromium.org.