Intent to Ship: Async Clipboard API

[Gary Kačmarčík (Кошмарчик)]

Contact emails

gar...@chromium.org

Spec

Editor's Draft Clipboard Spec: https://w3c.github.io/clipboard-apis/

Explainer: https://github.com/garykac/clipboard/blob/master/clipboard.md

Summary

Ship the plain-text portion of the new async clipboard API that is based on Promises as an initial step to shipping improved clipboard access.  As part of this we will implement/ship Clipboard Permissions that will initially only be used for this new API.

The API for text support (clipboard.readText() and clipboard.writeText()) has been stable (behind a flag) for many months. We are not requesting to ship the general API (clipboard.read() and clipboard.write()) since it is likely to undergo changes in the near future.

There are 2 permissions that control access to this API: "clipboard-read" and "clipboard-write" (see Clipboard Permission) 

All clipboard access is gated behind:

• Secure context (https)
  
• Chome is current app
  
• Script is running in the active tab

"clipboard-write" access is auto-granted (given the above constraints) while read access will require the user to grant "clipboard-read" permission for the site.

Note that this Intent to Ship does *not* cover changing the existing clipboard APIs (event- or execCommand-based) to make use of the new Permission. That will be a separate Intent to Implement/Ship since there is more risk due to the UX changes.

If you have a recent local/Dev/Canary build, you can use the API at https://permission.site.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Interoperability and Compatibility Risk

Low.

This intent is to add a new async API without changing the current clipboard API, so no existing content will break. Existing clipboard APIs will not (yet) be changed to use the new clipboard Permission.

General response from other browser vendors (Apple, Mozilla, Microsoft) has been positive and it was generally agreed that this API should move forward. See discussion at https://www.w3.org/2016/09/22-webapps-minutes.html#item17 and in the github repo.

Further discussion to resolve outstanding issues (images, mimetypes, delayed content generation) is ongoing.

Note that proper web platform support for clipboard cut/copy/paste has been a longstanding request from users and web developers. See related bugs: 424968, 443355, 593475 (and also 150835, 390583, mozilla/1356543, which cover images and/or rich text).

Link to "Intent to Implement"

https://groups.google.com/a/chromium.org/forum/#!topic/Blink-dev/pFI8AMdDChM

Link to web-platform-tests

https://github.com/w3c/web-platform-tests/tree/master/clipboard-apis

https://wpt.fyi/clipboard-apis (feature is currently behind a flag, some tests are manual since they require a permission and access the shared system clipboard)

Also:

https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/clipboard/

Launch tracking bug

https://crbug.com/677565

Link to entry on the feature dashboard

https://www.chromestatus.com/feature/5861289330999296

Requesting approval to ship?

Yes for the plain text APIs, but not for the general or image-specific APIs:

• clipboard.readText() YES
  
• clipboard.writeText() YES
  
• clipboard.read() NO
  
• clipboard.write() NO
  

Yes for the Clipboard Permission, but only as it applies to the new API (not to existing clipboard APIs).

[Anne van Kesteren]

Are you perhaps using an older template? I don't see a W3C TAG review
listed. I also don't think it's accurate to claim Mozilla has taken a
position on this API.

[Gary Kačmarčík (Кошмарчик)]

On Fri, Jan 5, 2018 at 1:19 AM, Anne van Kesteren <ann...@annevk.nl> wrote:

Are you perhaps using an older template? I don't see a W3C TAG review listed.

My apologies -- I didn't notice that it had been updated since I originally drafted the i2s.

Additional information:

TAG:

For the initial text-only part of the API, we did not submit a TAG review because returning a Promise with a basic text string seemed non-contentious when it was discussed. I opened a TAG review for this initial part here: https://github.com/w3ctag/design-reviews/issues/222

Our plan was to trigger a TAG review once we have a proposal for handling images and delayed content generation, since that is a far more significant and complex addition.

Debuggability: No changes required

Risks: Ergonomics: The purpose of this API is to provide *async* r/w access to the clipboard, so it is intended to improve on the current APIs that provide synchronous access.

Risks: Interop

Edge: No concerns expressed when discussed at TPAC

Firefox: No concerns expressed when discussed at TPAC

Safari: Expressed positive sentiment about having asynchronous clipboard access

Web developers: See list of related bugs given above. This feature has been a long-standing request.

Risks: Activation

Developers could use this API immediately, as-is.

Polyfills would be difficult to create, but existing clipboard libraries could be updated to use the new API.

 

I also don't think it's accurate to claim Mozilla has taken a position on this API.

smaug/Olli stated in the Sept 2016 TPAC meeting that there were no concerns with this API work moving forward, so "No signals" doesn't seem to be entirely accurate either. Do we have something in-between?

[Philip Jägenstedt]

LGTM1

This looks great, and I see issues filed by at least Gecko and WebKit engineers on https://github.com/w3c/clipboard-apis/issues, so I would expect any strong disagreement with the model to have surfaced by now.

None of the open issues seem to mention "readText" or "writeText", so that's good. (Except one I just filed, not blocking.)

Out of the tests in https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/clipboard/, could all of those be put in web-platform-tests if there were a way to set permissions? +Jonathon Kereliuk is working on that, so maybe this is a good candidate to try it with? (Many APIs using permissions also have some other part that's tricky to automate.)

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGnkXoHj0RV_jovoWgecBNnfTdyRfr%3D%3DawkmuUKx4jT3LxP_UA%40mail.gmail.com.

[Gary Kačmarčík (Кошмарчик)]

On Tue, Jan 16, 2018 at 5:19 AM, Philip Jägenstedt <foo...@chromium.org> wrote:

LGTM1

This looks great, and I see issues filed by at least Gecko and WebKit engineers on https://github.com/w3c/clipboard-apis/issues, so I would expect any strong disagreement with the model to have surfaced by now.

None of the open issues seem to mention "readText" or "writeText", so that's good. (Except one I just filed, not blocking.)

Out of the tests in https://cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/clipboard/, could all of those be put in web-platform-tests if there were a way to set permissions? +Jonathon Kereliuk is working on that, so maybe this is a good candidate to try it with? (Many APIs using permissions also have some other part that's tricky to automate.)

Yes. There are existing clipboard WPTs that are currently disabled because of the lack of Permission support, but these LayoutTests are simpler and would be good candidates for initial tests.

[Alex Russell]

Hey Gary,

This was reviewed in today's API Owners meeting. 

Thanks for filing the TAG review (although it has come in only recently and we haven't had a chance to discuss yet; I anticipate it getting time at the F2F meeting in London in 2 weeks). 

While that's in flight, I have a few questions about the API:

• Why is `readText()` not a flag on `read()`? E.g., `read({ types: ["text/plain", ...] })`
• `readText()` doesn't appear in the Explainer. How should a developer choose between them when they're both available?
• Will there always be text content?
• Is `clipboardchange` modeled using the same permission as reading?
• I see coverage of the Permissions API integration in the spec (great!). Would be great to see examples of how to use this API in the Explainer. Specifically, it seems that the ClipboardPermissionDescriptor would naturally be the same option bag that one would pass to read()/write() (or perhaps I'm missing something there?)

As a nit, might be good to have links from the Explainer to the spec and vise versa. Also, it'd be great if the explainer lived in the same repo.

Regarding interop risk, are other browsers implementing this?

Regards

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGnkXoFA0oJj70tU%2BaQ0CMFPAd3Kj5axLUMuTpLiF%2BCdgjyyKA%40mail.gmail.com.

[Mahfuzur Rahman]

Logesitcs with fellow

[Gary Kačmarčík (Кошмарчик)]

I've updated the explainer based on your comments here and moved it into the clipboard-apis repo:

https://github.com/w3c/clipboard-apis/blob/master/explainer.adoc

On Thu, Jan 18, 2018 at 10:51 AM, Alex Russell <sligh...@chromium.org> wrote:

While that's in flight, I have a few questions about the API:

• Why is `readText()` not a flag on `read()`? E.g., `read({ types: ["text/plain", ...] })`

Because text is a special (simple) case and other types are not: (1) 'text/plain' is the most commonly used clipboard data type so this unblocks a number of use cases, (2) it doesn't require special processing (like, e.g., images) so delayed generation is not a concern, (3) it doesn't have multiple representations (although other types may have a 'text' equivalent).

For this reason, we felt it warranted a special case.

Having 'text' passed in as a type param would work, but:

(1) That implies the API should work for other data types as well (which is possibly useful, but that complicates the "simple" API)

(2) Allowing multiple return types would require a wrapper (like DataTransfer) which would make this no simpler than the full `read()` method.

(3) The return type couldn't be strongly typed (Promise<object> instead of Promise<DOMString>).

Also recent issues (like this one for multiple representations of the same data), are making us reconsider the appropriateness of DataTransfer for the general read/write API. So we don't want to commit to a general `read()` API that relies on DataTransfer.

• `readText()` doesn't appear in the Explainer. How should a developer choose between them when they're both available?

I've updated the explainer. (Thank you for the comments on it!)

It's a convenience method, so if the developer only wants text data, then `readText()` is easier to use. I've added examples to the explainer that demonstrate the difference.

• Will there always be text content?

No, the clipboard is not guaranteed to contain text data. readText() returns an empty string in that case. 

• Is `clipboardchange` modeled using the same permission as reading?

Yes. [but see note below]

• I see coverage of the Permissions API integration in the spec (great!). Would be great to see examples of how to use this API in the Explainer. Specifically, it seems that the ClipboardPermissionDescriptor would naturally be the same option bag that one would pass to read()/write() (or perhaps I'm missing something there?)

The Permission check is automatically triggered when the API is called, and user agents may have a permission prompt as part of this check. Because of this, there is no need to call something like "GetClipboardPermission" before accessing the APIs.

The one exception to this is with the `clipboardchange` event. In that case, there is no API call to trigger the Permission check. When we implement that event, we'll need to resolve how to request permission (either something like "navigator.clipboard.requestPermission" or "permission.request", depending on how this permission bug is resolved).

As a nit, might be good to have links from the Explainer to the spec and vise versa. Also, it'd be great if the explainer lived in the same repo.

Done. and Done.

 

Regarding interop risk, are other browsers implementing this?

Not at the moment. AIUI, other browsers are waiting until the remaining issues with delayed generation and multiple representations are resolved before implementing. Apple and Mozilla have both been involved in (a) filing issues and (b) discussing solutions. Mozilla has landed a temporary patch to support image copy and would like to use this API once the remaining issues with images and delay-generation-content have been addressed.

Note that these outstanding issues only affect `read()` and `write()`, not `readText()` and `writeText()`.

[nehte...@gmail.com]

Hello! I use a new email filtering service called BitBounce to better filter my spam. To deliver your email to my inbox, please click the link below and pay the small Credo fee. Thank you! You can sign up or get more info about BitBounce by clicking here. To deliver your email: We’ve never met. I’ll pay your fee. I know you. Add me to your whitelist. Email That Pays BitBounce is a product by: Turing Technology, Inc. BitBounce.com Redwood City, CA

[Philip Jägenstedt]

Thanks for the explainer, Gary, good stuff!

An additional reason to keep read() and readText() separate is feature detection. If there's likely going to be a long period in any browser where just readText() but not read() is supported, simple and synchronous feature detection for that is useful.

[mk...@chromium.org]

Skimming through the bug, it looks like there was some discussion around the permission prompts that was never resolved (https://bugs.chromium.org/p/chromium/issues/detail?id=677564#c9). Were there additional conversations that weren't captured in the bug? +estark@ to make sure that Enamel has signed off on this from a security UX perspective.

-mike

[Emily Stark]

Raymes, I think you might have state on the permission prompts for this API?

[Gary Kačmarčík (Кошмарчик)]

The discussion around that moved to the Privacy Review bug: https://bugs.chromium.org/p/chromium/issues/detail?id=772426#c23

The decision was that:

(1) The Async Clipboard Write API would be allowed as long as (a) Chrome was the foreground app, and (b) the request is from the currently active tab

(2) The Async Clipboard Read API has the same requirements as the Write API, but it also requires explicit Permission to be granted

There is no need for a user gesture in either case.

On Wed, Feb 7, 2018 at 12:17 AM, <mk...@chromium.org> wrote:

[Ben Wells]

(answering for Raymes who is away this week)

On Thu, Feb 8, 2018 at 4:45 AM 'Gary Kačmarčík (Кошмарчик)' via blink-dev <blin...@chromium.org> wrote:

The discussion around that moved to the Privacy Review bug: https://bugs.chromium.org/p/chromium/issues/detail?id=772426#c23

The decision was that:

(1) The Async Clipboard Write API would be allowed as long as (a) Chrome was the foreground app, and (b) the request is from the currently active tab

(2) The Async Clipboard Read API has the same requirements as the Write API, but it also requires explicit Permission to be granted

There is no need for a user gesture in either case.

^^^ is correct.

 

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGnkXoEcSyozsf_XhurZ0gSFC-pFF2UCCggy1oqqiVE6HLSe2g%40mail.gmail.com.

[Gary Kačmarčík (Кошмарчик)]

I'm not aware of any remaining issues with this feature (other than Raymes flipping the Security bit when he gets back next week).

But I only see one LGTM. Is there anything further that I need to do here?

[Chris Harrelson]

Hi Gary,

Sorry for the delay. It was requested off-thread to wait for TAG review of this API since it is new. I see it was tagged as being possibly looked at at the last TAG meeting; I have pinged the issue there to see. If I don't hear back soon I will assume all is well and LGTM this review.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGnkXoEjj2YeeLSh47aDq7H-46h_hEUiiqWbOXE-BxLBzOKSTQ%40mail.gmail.com.

[Alex Russell]

Hey Gary,

Apologies for not responding to this thread sooner. Thanks for updating the explainer. It's much easier to see what's going on with the intended use of the API now.

I continue to have a few concerns:

• Do you anticipate the set of read*/write* methods growing in the future (e.g., for JSON, images, etc.)?
• No image formats beyond SVG are supported in writing. This is a security precaution? If so, I can understand why this might be perhaps reasonable on legacy OSes, but is it required of all OSes? Also, surely, SVG is the most complex image format...how is that safe when the others aren't?
• Is the set of readable formats knowable synchronously? If so, should there be an introspection API for this that can guard read*() methods?
• The style of reading multiple different types asynchronously is pretty far from what you'd get with, e.g., a Response object. For the sake of consistency, you might imagine `.text()`, `.json()`, etc. methods. Maybe this doesn't make sense because it isn't deserializing a single stream (as in, it isn't a single consumable source of data fronted by many methods, but rather many values hosted on a single object), but the naming at least feels out of step.
• The example in 7.2.2 shows `read()`, not `readText()`

Thanks again for your patience.

[Gary Kačmarčík (Кошмарчик)]

On Thu, Feb 15, 2018 at 11:12 AM, Alex Russell <sligh...@chromium.org> wrote:

Hey Gary,

Apologies for not responding to this thread sooner. Thanks for updating the explainer. It's much easier to see what's going on with the intended use of the API now.

I continue to have a few concerns:

Comments inline: Note that we're only interested in shipping the text-only API at this point. We have a bug to file another TAG review once we have a stable proposal that supports images, delayed generation of content, and multiple formats for the same clipboard object. 

Please let me know if any of these concerns are blocking for the text parts of the API.

• Do you anticipate the set of read*/write* methods growing in the future (e.g., for JSON, images, etc.)?

No, we do not intend to expand the convenience methods beyond "text". Text is a special case that is (a) very common and (b) simple.

We would certainly entertain proposals for additional convenience methods, but at the moment, there doesn't seem to be a need for anything beyond "text".

• No image formats beyond SVG are supported in writing. This is a security precaution? If so, I can understand why this might be perhaps reasonable on legacy OSes, but is it required of all OSes? Also, surely, SVG is the most complex image format...how is that safe when the others aren't?

[Note that that section is describing the situation where the clipboard events ("cut", "copy" and "paste") are being overridden by an EventListener. It is documenting the part of the API that has already shipped in all UAs and is unrelated to the Async API.]

Yes, it was a security requirement to prevent malicious images (special crafted to exploit bugs in the OS decoder) from being pasted to the clipboard. i was not around for the original discussion WRT SVG, but I believe that the difference is that SVG is (or was) typically parsed by the UA.

I've moved that section under the "Clipboard Event API" section to make it more clear that it applies to the ClipboardEvent APIs.

In any case, the long-term solution for this problem is to transcode images to clean up any exploits, but this cannot be done with the current APIs because they are synchronous and pasting a large image could freeze the browser. Addressing this was one of the motivations for introducing the Async API.

Once we have agreement on how to support images with the new API, we'll be able to have an expanded set of formats supported (although the current APIs will still be restricted by this list because of the security concerns discussed above).

• Is the set of readable formats knowable synchronously? If so, should there be an introspection API for this that can guard read*() methods?

No, we do not have an API for that. I've opened an issue to track this proposal.

• The style of reading multiple different types asynchronously is pretty far from what you'd get with, e.g., a Response object. For the sake of consistency, you might imagine `.text()`, `.json()`, etc. methods. Maybe this doesn't make sense because it isn't deserializing a single stream (as in, it isn't a single consumable source of data fronted by many methods, but rather many values hosted on a single object), but the naming at least feels out of step.

The current proposal for `read()` and `write()` is attempting to leverage the existing DataTransfer object (from Drag and Drop), but we are encountering some challenges with that approach and are considering using a different object type for clipboard. This is one of the issues that we need to resolve before we can finalize the general read/write APIs.

But whatever decision we make here doesn't impact the text APIs.

• The example in 7.2.2 shows `read()`, not `readText()`

Fixed!

[Chris Harrelson]

LGTM2

Per offline discussion with Gary, Alex and others, the questions raised at the end of this thread have either been fixed or bugs filed for future discussion, but AIUI the consensus is that the issued raised don't block shipping the current API iteration. That's good enough for me.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGnkXoEiA4BPcX2A9zfy9w-1kfTgJZzNgMAEOmPyhM6KRP7OZg%40mail.gmail.com.

[Yoav Weiss]

LGTM3

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGnkXoEiA4BPcX2A9zfy9w-1kfTgJZzNgMAEOmPyhM6KRP7OZg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOMQ%2Bw-7CVAccqu0p3mCxq-x94NwORv3Rt9njZC-f74O4sR6mA%40mail.gmail.com.

[mysteriou...@gmail.com]

When we would be able tocopy images using this command?

[PhistucK]

See crbug.com/150835.

☆PhistucK

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/c3a1f016-1c32-4bba-8ec4-c747ed9ca526%40chromium.org.