Intent to Ship: Clipboard: Preserve PNG metadata

[Austin Sullivan]

Contact emails

asu...@chromium.org, m...@chromium.org

Explainer

https://docs.google.com/document/d/1qZonN0xhfkuAOV58WOAMcPXjOB7uCfms3tf_ZDXNDks/edit?usp=sharing

Specification

Summary

Read unsanitized PNGs from the system clipboard. This will apply to both DataTransfer and the Async Clipboard API (navigator.clipboard.read()).

Motivation

Currently, reading PNGs from the system clipboard involves sanitizing the image by stripping its metadata. There is a strong argument that images from the system clipboard should not be sanitized on read, and this behavior is inconsistent with other major browser vendors and other forms of importing images, such as <input type="file">. See https://crbug.com/1177229 for further explanation.

Additionally, this significantly reduces the cost of pasting images from the clipboard in the vast majority of use cases (14x speed-up reading very large PNGs in limited testing).

Blink component

Blink>DataTransfer

TAG review

TAG review status

Not applicable

Risks

Interoperability and Compatibility

This change will put us in line with other browser vendors.

Gecko: Shipped/Shipping

WebKit: Shipped/Shipping

Web developers: Strongly positive (https://crbug.com/698793)

Security

This change is a net win for security on Android, since we will no longer be using an unsafe bitmap decoder.

Debuggability

N/A

Is this feature fully tested by web-platform-tests?

No

Flag name

ClipboardReadPng

Requires code in //chrome?

False

Tracking bug

https://crbug.com/1201018

Estimated milestones

M94

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5629962485760000

This intent message was generated by Chrome Platform Status.

[Alex Russell]

LGTM1 with caveats:

• this sanitization behavior was previously discussed with the TAG, and not updating them on it is a mistake. Please file a non-blocking FYI with them
• the explainer was less clear than the bug, requiring the OWNERs to read all the linked threads in detail. Ideally, an Explainer should clarify what is changing, why, and who it helps.
• Please post explainers as GH markdown files rather than google docs

Thanks!

On Friday, August 6, 2021 at 10:47:04 AM UTC-7 Austin Sullivan wrote:

Contact emails

asu...@chromium.org, mek@chromium.org

[Anupam Snigdha]

I don't see a spec linked in the I2S. Are we planning to make changes in the Clipboard API at some point in the future? I think it would be nice to document what unsanitized content means and the security/privacy concerns related to it.

-Anupam

On Thu, Aug 12, 2021 at 12:55 PM Alex Russell <sligh...@chromium.org> wrote:

LGTM1 with caveats:

• this sanitization behavior was previously discussed with the TAG, and not updating them on it is a mistake. Please file a non-blocking FYI with them
• the explainer was less clear than the bug, requiring the OWNERs to read all the linked threads in detail. Ideally, an Explainer should clarify what is changing, why, and who it helps.
• Please post explainers as GH markdown files rather than google docs

Thanks!

On Friday, August 6, 2021 at 10:47:04 AM UTC-7 Austin Sullivan wrote:

Contact emails

asu...@chromium.org, m...@chromium.org

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/244c6375-b991-4015-89ba-954295062d68n%40chromium.org.

[Marijn Kruisselbrink]

While I agree that it would be nice for the clipboard API to better specify what sanitization is and isn't done, do note that this change merely brings us in line with other existing implementations. I.e. both Safari and Firefox already ship the behavior we're proposing to ship with here as well. https://w3c.github.io/clipboard-apis/#image-transcode would probably be the most relevant section in the spec as it stands today, although it only addresses writing images to the clipboard, not reading images from the clipboard.

So I don't think changes to the spec are needed for this intent, as the spec arguably already matches our proposed new behavior. Nowhere (except for some subset of pasting HTML) does it mention any sanitization on reading from the clipboard, while it does explicitly mention sanitizing data as it is being written to the clipboard. I think what we're proposing here is fully in line with that behavior (and our previous/current behavior is not spec compliant).

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2Bm%3DdJpYJLoY7zsuHP_Rg5oX-_mK%2BpwvQLjdqEXbffQXDwMBWQ%40mail.gmail.com.

[Yoav Weiss]

Changing the spec to explicitly say that no sanitization is to be done on input would be good. I agree that aligning behavior with other implementers is the right thing to do, but we need to document that behavior. Can y'all please file a spec PR to that effect?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BOSsVaZ4nUQDqy_gZ_7HTUSLVgt5qUtoAFDj8P6C%2BJQO08f%3DQ%40mail.gmail.com.

[Mike West]

Friendly ping to Yoav's question. I would also like to see some sort of hook in the spec that made this mechanism explicit. I think this is a reasonable change to ship, I just want to ensure we lock in reasonable behavior at the spec level. Is that something y'all can take care of?

Thanks!

-mike

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAL5BFfUDSNk2yGOwzNyjuW%3D60BhSfUkHNTXETdCKEawZ9eHcYg%40mail.gmail.com.

[Anupam Snigdha]

I can probably add something to this discussion. In this thread, we are discussing about making the sanitization process more explicit in async clipboard APIs spec during read & write. We have proposed the HTML sanitization behavior here. I have limited knowledge on the sanitization process in image/png, so it'd be nice if someone can chime in that thread and describe the change that is being shipped here so we can standardize this behavior. I can make the spec changes once we all agree on the specifics.

-Anupam

[Austin Sullivan]

I just chimed in on the GitHub thread you linked. It's not entirely related, since this change is simply putting us in line with other browsers and we want to reflect this newly consistent behavior in the spec, while the HTML sanitization changes seem like they'll require a bit more discussion and perhaps include more substantial changes. Regardless, if you'll be updating the spec for this change that info should be useful. Feel free to reach out if you have other questions.

[Anupam Snigdha]

Thanks Austin! I think you are right that HTML changes would require more discussion so the spec updates for that can be done separately. SVG will probably depend on the outcome of that discussion as it uses the same sanitization procedure as HTML. I can update the spec to reflect the current behavior of the read/write operation for all the standard formats, but I'd like to get some feedback from other browser vendors on the proposals as that might change the algorithms substantially.

-Anupam

[Marijn Kruisselbrink]

I clarified the currently spec'ed behavior around image sanitization in https://github.com/w3c/clipboard-apis/pull/156. That should make explicit the current behavior of Safari and Firefox, and be in match the proposed changes for Chrome in this Intent.

[Yoav Weiss]

Thank you!!

LGTM2

[Mike West]

LGTM3. Thanks for following up on this.

-mike