Intent to Ship: Async Clipboard read/write with image support

209 views
Skip to first unread message

huang...@chromium.org

unread,
Apr 15, 2019, 3:59:15 PM4/15/19
to blink-dev

Contact emails

huang...@chromium.org, gar...@chromium.org, pwn...@chromium.org


Explainer

Read/Write Image Explainer (Background explainer for original Async Clipboard API)


Spec

w3c.github.io/clipboard-apis/#async-clipboard-api


Intent to Implement

Link


Tag review

Pending. (See previous tag review for text-only)


Summary

Add programmatic copy/paste support of images to/from the Async Clipboard API. This would allow scripts to write images (ex. copy) to the OS clipboard, or read (ex. paste) images already on the OS clipboard.


To avoid security implications, originating from image decoders with remote code execution (RCE) security vulnerabilities  (ex. Firefox and Webkit (iOS/Safari/etc)) on other native apps (especially non-updated native apps), Chrome will re-encode images and drop all attached metadata, just as the existing clipboard does when copy/pasting images. The same re-encoding settings will be used for parity.


As image reencoding is expensive and slow, it would cause unacceptable jank if done on the main thread, especially for large images. Therefore, images will be re-encoded on a background thread.


The Async Clipboard API’s read() and write() functions will also be reimplemented and extended to include image support, as it previously only supported plain text, and standards have moved since then to no longer approve of the use of DataTransfer, and to opt for Blobs instead.


Motivation

As the most starred bug for Chromium (crbug.com/150835), addition of programmatic image clipboard support has high demand. Without it, web developers often use loopholes, like writing text/plain and then parsing the text/plain payload as an image (crbug.com/150835#c67), or creating selections, sometimes of invisible HTML image elements, and running document.execcommand(‘copy’) (owaisafaq.com/blog/demos/copier/). These methods are complicated, synchronous (may introduce jank), and ultimately a longstanding source of complaints that have led to the popularity of the aforementioned bug.


Risks

Interoperability and Compatibility

This feature is part of the Async Clipboard API discussion, which has received positive feedback from Firefox, Edge, and Safari, as well as web developers. For this specific feature, discussion is generally positive, and browser vendors are actively engaged in spec discussions (www.w3.org/2018/10/23-WebPlat-minutes.html#item02)


Edge: Actively engaged in spec discussions

Firefox:  Actively engaged in spec discussions

Safari:  Actively engaged in spec discussions

Web / Framework developers: Very positive support from websites with image support needs, such as Figma, as evidenced by the bug having more than >1800 stars crbug.com/150835.


Ergonomics

This feature will frequently be used in tandem with the Blob API, FileReader API, and Fetch API. As all these APIs are fairly asynchronous and performant, and this API runs on a background thread and returns asynchronously, this should not negatively impact performance.


Activation

This library should be fairly easy to use, especially when compared to existing methods using document.execcommand(‘copy’);


Debuggability

Dedicated debugging support on DevTools is not required for this change.


Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.


Tests:

w3c-test.org/clipboard-apis/


Link to entry on the feature dashboard

https://www.chromestatus.com/features/5074658793619456


Daniel Bratell

unread,
Apr 25, 2019, 11:02:33 AM4/25/19
to blink-dev, huang...@chromium.org
Looks like this is set to be discussed on a TAG meeting in 2 weeks. The open question being how it relates to equivalent changes for drag/drop. 

I've been holding off waiting for that discussion reaching some kid of conclusion (or decision that it's not relevant for the shipping decision).

/Daniel
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/f5a77a9c-9c06-4afd-99e9-298d134f82ac%40chromium.org.



--
/* Opera Software, Linköping, Sweden: CET (UTC+1) */

huang...@chromium.org

unread,
Apr 25, 2019, 4:23:42 PM4/25/19
to blink-dev, huang...@chromium.org
Thank you for the feedback. Yes, I'm curious what TAG feedback will be as well.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Philip Jägenstedt

unread,
May 28, 2019, 6:03:44 AM5/28/19
to huang...@chromium.org, blink-dev
Hi Darwin,

Do you have insight into the current state of the TAG review? It looks like it's been delayed once, but not done yet?

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.



--
/* Opera Software, Linköping, Sweden: CET (UTC+1) */

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/78965615-0b22-4874-aede-58ab00a0b275%40chromium.org.

Victor Costan

unread,
May 29, 2019, 8:09:39 PM5/29/19
to Philip Jägenstedt, Darwin Huang, blink-dev
Replying on behalf of Darwin,

We don't have any insight into what the TAG is thinking past what is on the public thread. The TAG discussion seems focused on transcoding, which is important, but is an implementation detail that we can change later without breaking the API. We did not get any feedback around the Web-exposed API shape. Given all this, and the high developer interest, we think the risk is low enough and would like to ship this in M76.

Thank you,
    Victor

Chris Harrelson

unread,
May 29, 2019, 8:17:35 PM5/29/19
to Victor Costan, Philip Jägenstedt, Darwin Huang, blink-dev
Makes sense to me, thanks Victor.

LGTM1

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAP_mGKpwZmkPwzg1n2eSP8CcaXQ1z6CV4P_y7uDYAmxL%3DPuFnQ%40mail.gmail.com.

Alex Russell

unread,
May 30, 2019, 3:08:14 PM5/30/19
to blink-dev, pwn...@chromium.org, foo...@chromium.org, huang...@chromium.org
LGTM2, however I think this feature remains relatively incomplete for professional app use-cases. Leaving feedback in the TAG issue to that extent as I view it as separable.
To unsubscribe from this group and stop receiving emails from it, send an email to blin...@chromium.org.



--
/* Opera Software, Linköping, Sweden: CET (UTC+1) */

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/78965615-0b22-4874-aede-58ab00a0b275%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blin...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blin...@chromium.org.

oj...@google.com

unread,
May 30, 2019, 3:14:13 PM5/30/19
to blink-dev, pwn...@chromium.org, foo...@chromium.org, huang...@chromium.org
LGTM3
Reply all
Reply to author
Forward
0 new messages