Intent to Ship: Clipboard: Preserve PNG metadata

159 views
Skip to first unread message

Austin Sullivan

unread,
Aug 6, 2021, 1:47:04 PMAug 6
to blink-dev
Contact emails
asu...@chromium.orgm...@chromium.org

Explainer


Specification



Summary

Read unsanitized PNGs from the system clipboard. This will apply to both DataTransfer and the Async Clipboard API (navigator.clipboard.read()).


Motivation

Currently, reading PNGs from the system clipboard involves sanitizing the image by stripping its metadata. There is a strong argument that images from the system clipboard should not be sanitized on read, and this behavior is inconsistent with other major browser vendors and other forms of importing images, such as <input type="file">. See https://crbug.com/1177229 for further explanation.

Additionally, this significantly reduces the cost of pasting images from the clipboard in the vast majority of use cases (14x speed-up reading very large PNGs in limited testing).

Blink component

Blink>DataTransfer

TAG review



TAG review status

Not applicable

Risks



Interoperability and Compatibility

This change will put us in line with other browser vendors.



Gecko: Shipped/Shipping

WebKit: Shipped/Shipping

Web developers: Strongly positive (https://crbug.com/698793)

Security

This change is a net win for security on Android, since we will no longer be using an unsafe bitmap decoder.



Debuggability

N/A



Is this feature fully tested by web-platform-tests?

No

Flag name

ClipboardReadPng

Requires code in //chrome?

False

Tracking bug

https://crbug.com/1201018

Estimated milestones

M94

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5629962485760000

This intent message was generated by Chrome Platform Status.

Alex Russell

unread,
Aug 12, 2021, 3:55:00 PMAug 12
to blink-dev, Austin Sullivan
LGTM1 with caveats:
  • this sanitization behavior was previously discussed with the TAG, and not updating them on it is a mistake. Please file a non-blocking FYI with them
  • the explainer was less clear than the bug, requiring the OWNERs to read all the linked threads in detail. Ideally, an Explainer should clarify what is changing, why, and who it helps.
  • Please post explainers as GH markdown files rather than google docs
Thanks!

On Friday, August 6, 2021 at 10:47:04 AM UTC-7 Austin Sullivan wrote:

Anupam Snigdha

unread,
Aug 13, 2021, 12:27:56 PMAug 13
to Alex Russell, blink-dev, Austin Sullivan
I don't see a spec linked in the I2S. Are we planning to make changes in the Clipboard API at some point in the future? I think it would be nice to document what unsanitized content means and the security/privacy concerns related to it.

-Anupam

On Thu, Aug 12, 2021 at 12:55 PM Alex Russell <sligh...@chromium.org> wrote:
LGTM1 with caveats:
  • this sanitization behavior was previously discussed with the TAG, and not updating them on it is a mistake. Please file a non-blocking FYI with them
  • the explainer was less clear than the bug, requiring the OWNERs to read all the linked threads in detail. Ideally, an Explainer should clarify what is changing, why, and who it helps.
  • Please post explainers as GH markdown files rather than google docs
Thanks!

On Friday, August 6, 2021 at 10:47:04 AM UTC-7 Austin Sullivan wrote:

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/244c6375-b991-4015-89ba-954295062d68n%40chromium.org.

Marijn Kruisselbrink

unread,
Aug 18, 2021, 4:52:18 PMAug 18
to Anupam Snigdha, Alex Russell, blink-dev, Austin Sullivan
While I agree that it would be nice for the clipboard API to better specify what sanitization is and isn't done, do note that this change merely brings us in line with other existing implementations. I.e. both Safari and Firefox already ship the behavior we're proposing to ship with here as well. https://w3c.github.io/clipboard-apis/#image-transcode would probably be the most relevant section in the spec as it stands today, although it only addresses writing images to the clipboard, not reading images from the clipboard.

So I don't think changes to the spec are needed for this intent, as the spec arguably already matches our proposed new behavior. Nowhere (except for some subset of pasting HTML) does it mention any sanitization on reading from the clipboard, while it does explicitly mention sanitizing data as it is being written to the clipboard. I think what we're proposing here is fully in line with that behavior (and our previous/current behavior is not spec compliant).

Yoav Weiss

unread,
Aug 18, 2021, 5:19:40 PMAug 18
to Marijn Kruisselbrink, Anupam Snigdha, Alex Russell, blink-dev, Austin Sullivan
Changing the spec to explicitly say that no sanitization is to be done on input would be good. I agree that aligning behavior with other implementers is the right thing to do, but we need to document that behavior. Can y'all please file a spec PR to that effect?

Mike West

unread,
Aug 26, 2021, 3:07:29 PMAug 26
to Yoav Weiss, Marijn Kruisselbrink, Anupam Snigdha, Alex Russell, blink-dev, Austin Sullivan
Friendly ping to Yoav's question. I would also like to see some sort of hook in the spec that made this mechanism explicit. I think this is a reasonable change to ship, I just want to ensure we lock in reasonable behavior at the spec level. Is that something y'all can take care of?

Thanks!

-mike


Anupam Snigdha

unread,
Aug 31, 2021, 1:03:09 PMAug 31
to Mike West, Yoav Weiss, Marijn Kruisselbrink, Alex Russell, blink-dev, Austin Sullivan
I can probably add something to this discussion. In this thread, we are discussing about making the sanitization process more explicit in async clipboard APIs spec during read & write. We have proposed the HTML sanitization behavior here. I have limited knowledge on the sanitization process in image/png, so it'd be nice if someone can chime in that thread and describe the change that is being shipped here so we can standardize this behavior. I can make the spec changes once we all agree on the specifics.

-Anupam

Austin Sullivan

unread,
Sep 1, 2021, 1:10:41 PMSep 1
to Anupam Snigdha, Mike West, Yoav Weiss, Marijn Kruisselbrink, Alex Russell, blink-dev
I just chimed in on the GitHub thread you linked. It's not entirely related, since this change is simply putting us in line with other browsers and we want to reflect this newly consistent behavior in the spec, while the HTML sanitization changes seem like they'll require a bit more discussion and perhaps include more substantial changes. Regardless, if you'll be updating the spec for this change that info should be useful. Feel free to reach out if you have other questions.

Anupam Snigdha

unread,
Sep 1, 2021, 1:47:46 PMSep 1
to Austin Sullivan, Mike West, Yoav Weiss, Marijn Kruisselbrink, Alex Russell, blink-dev
Thanks Austin! I think you are right that HTML changes would require more discussion so the spec updates for that can be done separately. SVG will probably depend on the outcome of that discussion as it uses the same sanitization procedure as HTML. I can update the spec to reflect the current behavior of the read/write operation for all the standard formats, but I'd like to get some feedback from other browser vendors on the proposals as that might change the algorithms substantially.

-Anupam

Marijn Kruisselbrink

unread,
Sep 7, 2021, 5:31:08 PMSep 7
to Anupam Snigdha, Austin Sullivan, Mike West, Yoav Weiss, Alex Russell, blink-dev
I clarified the currently spec'ed behavior around image sanitization in https://github.com/w3c/clipboard-apis/pull/156. That should make explicit the current behavior of Safari and Firefox, and be in match the proposed changes for Chrome in this Intent.

Yoav Weiss

unread,
Sep 8, 2021, 3:19:42 AMSep 8
to Marijn Kruisselbrink, Anupam Snigdha, Austin Sullivan, Mike West, Alex Russell, blink-dev
Thank you!!

LGTM2

Mike West

unread,
Sep 8, 2021, 3:48:03 AMSep 8
to Yoav Weiss, Marijn Kruisselbrink, Anupam Snigdha, Austin Sullivan, Alex Russell, blink-dev
LGTM3. Thanks for following up on this.

-mike

Reply all
Reply to author
Forward
0 new messages