Intent to Implement: Async Clipboard read/write with image support

169 views
Skip to first unread message

huang...@chromium.org

unread,
Feb 28, 2019, 6:37:38 PM2/28/19
to blink-dev

Contact emails

huang...@chromium.org, gar...@chromium.org, pwn...@chromium.org


Explainer

Background explainer (for original Async Clipboard API) Explainer link

Read/Write Image Explainer) (earlier discussion/background)


Spec

w3c.github.io/clipboard-apis/#async-clipboard-api


Tag review

None, but the tag review will be completed before our Intent to Ship. (previous tag review for text-only)


Summary

Add programmatic copy/paste support of images to/from the Async Clipboard API. This would allow scripts to write images (ex. copy) to the OS clipboard, or read (ex. paste) images already on the OS clipboard.


To avoid security implications, originating from image decoders with remote code execution (RCE) security vulnerabilities  (ex. Firefox and Webkit (iOS/Safari/etc)) on other native apps (especially non-updated native apps), Chrome will re-encode images and drop all attached metadata, just as the existing clipboard does when copy/pasting images. The same re-encoding settings will be used for parity.


As image reencoding is expensive and slow, it would cause unacceptable jank if done on the main thread, especially for large images. Therefore, images will be re-encoded on a background thread.


The Async Clipboard API’s read() and write() functions will also be reimplemented and extended to include image support, as it previously only supported plain text, and standards have moved since then to no longer approve of the use of DataTransfer, and to opt for Blobs instead.


Motivation

As the most starred bug for Chromium (crbug.com/150835), addition of programmatic image clipboard support has high demand. Without it, web developers often use loopholes, like writing text/plain and then parsing the text/plain payload as an image (crbug.com/150835#c67), or creating selections, sometimes of invisible HTML image elements, and running document.execcommand(‘copy’) (owaisafaq.com/blog/demos/copier/). These methods are complicated, synchronous (may introduce jank), and ultimately a longstanding source of complaints that have led to the popularity of the aforementioned bug.


Risks

Interoperability and Compatibility

This feature is part of the Async Clipboard API discussion, which has received positive feedback from Firefox, Edge, and Safari, as well as web developers. For this specific feature, discussion is generally positive, and browser vendors are actively engaged in spec discussions (www.w3.org/2018/10/23-WebPlat-minutes.html#item02)


Edge: Actively engaged in spec discussions

Firefox:  Actively engaged in spec discussions

Safari:  Actively engaged in spec discussions

Web / Framework developers: Very positive support from websites with image support needs, such as Figma, as evidenced by the bug having more than >1800 stars crbug.com/150835.


Ergonomics

This feature will frequently be used in tandem with the Blob API, FileReader API, and Fetch API. As all these APIs are fairly asynchronous and performant, and this API runs on a background thread and returns asynchronously, this should not negatively impact performance.


Activation

This library should be fairly easy to use, especially when compared to existing methods using document.execcommand(‘copy’);


Debuggability

Dedicated debugging support on DevTools is not required for this change.


Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.


Tests:

w3c-test.org/clipboard-apis/


Link to entry on the feature dashboard

https://www.chromestatus.com/features/5074658793619456


Requesting approval to ship?

No


Daniel Bratell

unread,
Mar 1, 2019, 5:35:52 AM3/1/19
to blink-dev, huang...@chromium.org
Not having looked at any of the details, I'm really happy at you taking a serious shot at this most requested feature according to crbug stars! 

Will the support include vector formats (SVG) as well?

/Daniel
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/d6e2d8af-45ef-4761-aa2e-42ee4e117b2b%40chromium.org.



--
/* Opera Software, Linköping, Sweden: CET (UTC+1) */

huang...@chromium.org

unread,
Mar 3, 2019, 2:09:38 AM3/3/19
to blink-dev, huang...@chromium.org
Yayy, thank you for the support! 

While SVG is in the long-term road-map, this intent to implement doesn't cover it yet, so that we can reduce its scope, and due to concerns regarding SVG encoder/decoder XSS vulnerabilities.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

gl...@microsoft.com

unread,
Mar 6, 2019, 7:43:41 PM3/6/19
to blink-dev
To avoid security implications, originating from image decoders with remote code execution (RCE) security vulnerabilities  (ex. Firefox and Webkit (iOS/Safari/etc)) on other native apps (especially non-updated native apps), Chrome will re-encode images and drop all attached metadata, just as the existing clipboard does when copy/pasting images. The same re-encoding settings will be used for parity.'

By re-encoding the images that can't be sanitized, we are breaking at least one use case for Figma.

Given that a site is getting permissions through permissions API. Can't UA make an assumption about the "validity" of an image?

While true that's how it was implemented for legacy clipboard api, i feel like we have an extra layer of protection through permissions api.
 
thoughts?

Darwin Huang

unread,
Mar 7, 2019, 3:21:51 PM3/7/19
to blink-dev
This permission is only for the user allowing for programmatic read/write interaction with the clipboard, with the assumption that what will be placed on the clipboard is just as "safe" and sanitized as with the legacy clipboard API.

It's on the roadmap to implement a raw clipboard feature in the future, which will allow for arbitrary types to be written directly to the clipboard without re-encoding or dropping metadata. This is considered a separate feature as there are a lot of security concerns involving unsanitized data, and is dependent on this feature to support sanitized image data.
Reply all
Reply to author
Forward
0 new messages