Web-Facing Change PSA: Improved error reporting in IndexedDB for large value read failures

[Chromestatus]

Contact emails

abhishek.s...@microsoft.com, est...@chromium.org

Specification

https://www.w3.org/TR/IndexedDB/#dom-idbrequest-error

Summary

Change to reporting for certain error cases that were previously reported with a DOMException and the message "Failed to read large IndexedDB value". Chromium will now raise a DOMException with the name "NotFoundError" when the file containing the data being read by an IDBRequest is missing from the disk so that sites can take the appropriate corrective action when an unrecoverable failure occurs. Corrective actions could include deleting the entry from the DB, notifying the user, re-fetching the data from servers, etc. More details: a large value (above a specific size threshold) written to IndexedDB is not stored directly in the underlying LevelDB database but instead stored as a separate file. An IndexedDB read request for this value looks up the blob reference from LevelDB, reads the blob, then unwraps and returns the stored value. If a failure occurs in this process, the browser fires an "error" event and sets the error property on the IDBRequest to a DOMException with the message "Failed to read large IndexedDB value". The failure could be recoverable (caused by low memory) or unrecoverable (the blob file is missing from the disk). This feature updates the name to "NotFoundError" to enable distinguishing recoverable and unrecoverable cases.

Blink component

Blink>Storage>IndexedDB

TAG review

None

TAG review status

Not applicable

Risks

Interoperability and Compatibility

None

Gecko: No signal

WebKit: No signal

Web developers: No signals

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

None

Debuggability

None

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)?

Yes

This change is in the blink layer of the IndexedDB API implementation and hence applies to all Blink platforms.

Is this feature fully tested by web-platform-tests?

No

Flag name on chrome://flags

None

Finch feature name

None

Non-finch justification

This is a simple change to the error type when a failure occurs due to a specific cause. Web developers can choose to update their sites to respond differently to this specific error, but existing general error handling keyed to the "Failed to read large IndexedDB value" message will continue to work. Hence, this is unlikely to break sites and is therefore not behind a feature flag.

Requires code in //chrome?

False

Tracking bug

https://issues.chromium.org/issues/362123231

Estimated milestones

Shipping on desktop	130
Shipping on Android	130
Shipping on WebView	130

Anticipated spec changes

Open questions about a feature may be a source of future web compat or interop issues. Please list open issues (e.g. links to known github issues in the project for the feature specification) whose resolution may introduce web compat/interop risk (e.g., changing to naming or structure of the API in a non-backward-compatible way).

None

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5140210640486400?gate=5120313600507904

This intent message was generated by Chrome Platform Status.

[Mike Taylor]

Hi! A couple of questions:

What was the DOMException.name before this change? 

Do we know what do other browsers do in this situation? Or is LevelDB only used by Chromium?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/66e98e07.2b0a0220.195547.0126.GAE%40google.com.

[Abhishek Shanthkumar]

Hi, Mike!

The DOMException.name was "DataError" for all cases before this change. It remains the same for the "recoverable" cases even after this change. I'll add this to the Summary, thanks!

This error is unique to Chromium because, yes, LevelDB is not used by other browsers.

Thanks,

Abhishek

 

---

From: Mike Taylor <mike...@chromium.org>
Sent: Tuesday, September 17, 2024 7:46 PM
To: Chromestatus <ad...@cr-status.appspotmail.com>; blin...@chromium.org <blin...@chromium.org>
Cc: Abhishek Shanthkumar <Abhishek.S...@microsoft.com>; ay...@chromium.org <ay...@chromium.org>; est...@chromium.org <est...@chromium.org>; Steve Becker <ste...@microsoft.com>
Subject: [EXTERNAL] Re: [blink-dev] Web-Facing Change PSA: Improved error reporting in IndexedDB for large value read failures

 

You don't often get email from mike...@chromium.org. Learn why this is important

[Mike Taylor]

Sounds good - thx!

[Abhishek Shanthkumar]

A correction about the behavior of other browsers - it looks like Firefox too stores blobs and large values in files separate from the SQLite database.

Deleting the files and then attempting to read the value raises a DOMException on the IDBRequest with these parameters:

name: "UnknownError"
message: "The operation failed for reasons unrelated to the database itself and not covered by any other error code."

It might be helpful to converge on a common error across browsers for this scenario. This does not strictly fit in the IndexedDB spec though, since storing certain data in separate files is an implementation detail.

What is the usual guidance for such cases?

Thanks,

Abhishek

 

---

From: Mike Taylor <mike...@chromium.org>
Sent: Tuesday, September 17, 2024 8:26 PM
To: Abhishek Shanthkumar <Abhishek.S...@microsoft.com>; Chromestatus <ad...@cr-status.appspotmail.com>; blin...@chromium.org <blin...@chromium.org>
Cc: ay...@chromium.org <ay...@chromium.org>; est...@chromium.org <est...@chromium.org>; Steve Becker <ste...@microsoft.com>
Subject: Re: [EXTERNAL] Re: [blink-dev] Web-Facing Change PSA: Improved error reporting in IndexedDB for large value read failures

 

[Domenic Denicola]

Thanks for your attentiveness to interop here, and to the spec issues! This is above-and-beyond.

I think we should amend the spec to cover this scenario. Even though the implementation detail of using separate files is browser-specific, the general idea that "sometimes getting a value fails for reasons related to the underlying storage" is something that we can cover. From what I can tell, the spec already covers some such cases, e.g. https://w3c.github.io/IndexedDB/#dom-idbfactory-databases step 4.1.

Would you be willing to work on a small spec PR to modify the appropriate methods to include a sentence like that? /cc @Joshua Bell as spec editor.

As for the specific type of error, my read is that although the section at https://w3c.github.io/IndexedDB/#exceptions implies "NotFoundError" is a good choice, looking at actual usage in the spec, "NotFoundError" seems to be more about deterministic cases where the web developer is requesting something from the database which they have not previously written to it, whereas "UnknownError" is used in cases where something goes wrong with underlying storage. So maybe aligning with Firefox is the best path forward here, although I don't feel strongly.

I still think it's reasonable to do all this as a web-facing PSA instead of a full Intent to Ship, by the way.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CY8PR00MB15074249A0CDEF0E254CAAE790632%40CY8PR00MB1507.namprd00.prod.outlook.com.

[Abhishek Shanthkumar]

Thank you for the inputs, Domenic! Yes, I'll be happy to take this forward so that we can converge on a common error for such scenarios. I'll go ahead and file a spec issue as the first step.

Though the exact error can be discussed - "InvalidStateError" too may be a good choice since it says "... a request is made on a source object that has been deleted or removed" - I think the most important point to discuss from the spec perspective is whether to distinguish between the recoverable and unrecoverable cases. I'm not yet sure if Firefox throws the same "UnknownError" if there is a transient failure in reading the value.

Thanks,

Abhishek

 

---

From: Domenic Denicola <dom...@chromium.org>
Sent: Friday, September 20, 2024 7:46 AM
To: Abhishek Shanthkumar <Abhishek.S...@microsoft.com>; Joshua Bell <jsb...@chromium.org>
Cc: Mike Taylor <mike...@chromium.org>; Chromestatus <ad...@cr-status.appspotmail.com>; blin...@chromium.org <blin...@chromium.org>; ay...@chromium.org <ay...@chromium.org>; est...@chromium.org <est...@chromium.org>; Steve Becker <ste...@microsoft.com>

Subject: Re: [EXTERNAL] Re: [blink-dev] Web-Facing Change PSA: Improved error reporting in IndexedDB for large value read failures

 

You don't often get email from dom...@chromium.org. Learn why this is important

[Abhishek Shanthkumar]

I have created a spec issue at Specify the exception for read failures arising due to partial data loss on the file system · Issue #423 · w3c/IndexedDB (github.com). I'll then create a spec PR based on how the discussion evolves.

Thanks,

Abhishek

 

---

From: Abhishek Shanthkumar <Abhishek.S...@microsoft.com>
Sent: Friday, September 20, 2024 7:50 PM
To: Domenic Denicola <dom...@chromium.org>; Joshua Bell <jsb...@chromium.org>

[Abhishek Shanthkumar]

Update: A spec PR has been merged to update the exception type to be "NotReadableError" for these failures. Thank you for your inputs and guidance, Domenic, Joshua and Evan!

I'll make these follow-up changes in Chromium, still tracked by crbug.com/362123231 :

• Update the exception type from "NotFoundError" to "NotReadableError" for unrecoverable failures arising from data loss.
• Update the exception type from "DataError" to "UnknownError" for transient failures.
• Update the exception message to be more effective in conveying the nature of the error to developers.

The above changes will go out in M132 though (since M131 branched out this week), while the original change that started this thread (changing the error type to "NotFoundError") will be reaching M130 Stable this week.

So interested developers would need to update their error handlers again for M132, which isn't ideal...

Should we aim to maintain some sort of compatibility for a couple of releases, or is this an acceptable tradeoff for converging on a common error and a detailed message?

Thanks,

Abhishek

 

---

From: Abhishek Shanthkumar <Abhishek.S...@microsoft.com>
Sent: Monday, September 23, 2024 5:55 PM

[Domenic Denicola]

On Wed, Oct 16, 2024 at 5:22 PM Abhishek Shanthkumar <Abhishek.S...@microsoft.com> wrote:

Update: A spec PR has been merged to update the exception type to be "NotReadableError" for these failures. Thank you for your inputs and guidance, Domenic, Joshua and Evan!

I'll make these follow-up changes in Chromium, still tracked by crbug.com/362123231 :

• Update the exception type from "NotFoundError" to "NotReadableError" for unrecoverable failures arising from data loss.
• Update the exception type from "DataError" to "UnknownError" for transient failures.
• Update the exception message to be more effective in conveying the nature of the error to developers.

The above changes will go out in M132 though (since M131 branched out this week), while the original change that started this thread (changing the error type to "NotFoundError") will be reaching M130 Stable this week.

So interested developers would need to update their error handlers again for M132, which isn't ideal...

Should we aim to maintain some sort of compatibility for a couple of releases, or is this an acceptable tradeoff for converging on a common error and a detailed message?

In my opinion, it's an acceptable tradeoff. Changing only error names, for unpredictable errors, should not be a significant compatibility risk. And the sooner we reach the desired end state, the better.

[Abhishek Shanthkumar]

Update: With the merge of this CL, the DOMException thrown for the two cases of failure is updated as below from Chromium 132 onwards:

Type of failure	DOMException name	DOMException message
Unrecoverable (missing file)	NotReadableError	"Data lost due to missing file. Affected record should be considered irrecoverable"
Recoverable/transient	UnknownError

"Failed to read large IndexedDB value"

I have also updated the ChromeStatus entry with this information.

Thank you for all the inputs and guidance! Please feel free to reach out for further feedback/suggestions.

Thanks,

Abhishek

 

---

From: Domenic Denicola <dom...@chromium.org>
Sent: Thursday, October 17, 2024 8:09 AM
To: Abhishek Shanthkumar <Abhishek.S...@microsoft.com>
Cc: Domenic Denicola <dom...@chromium.org>; Joshua Bell <jsb...@chromium.org>; Mike Taylor <mike...@chromium.org>; Chromestatus <ad...@cr-status.appspotmail.com>; blin...@chromium.org <blin...@chromium.org>; ay...@chromium.org <ay...@chromium.org>; est...@chromium.org <est...@chromium.org>; Steve Becker <ste...@microsoft.com>