Intent to Prototype: Pickling for Async Clipboard API

[Anupam Snigdha]

Contact emails

sni...@microsoft.com, huang...@chromium.org, pc...@microsoft.com

 

Explainer

https://github.com/dway123/clipboard-pickling/blob/main/explainer.md#pickling-for-async-clipboard-api

 

Design Doc

https://docs.google.com/document/d/1afc45MQuwxEWgoUeJCO-sOWRSzs31V4JS-kKXJNMTXw/edit?usp=sharing 

 

Specification

None

 

Summary

Pickle Clipboard API lets websites read and write arbitrary unsanitized payloads using a standardized pickling format, as well as read and write a limited subset of OS-specific formats (for supporting legacy apps).

The name of the clipboard format is mangled by the browser in a standardized way to indicate that the content is from the web, which allows native applications to opt-in to accepting the unsanitized content.

 

Blink component

Blink>DataTransfer

 

Motivation

Powerful web applications would like to exchange data payloads with web and native applications via the OS clipboard (copy-paste). The existing Web Platform has an API that supports the most popular standardized data types (text, image, rich text) across all platforms. However, this API does not scale to the long tail of specialized formats. In particular, custom formats, non-web-standard formats like TIFF (a large image format), and proprietary formats like .docx (a document format), are not supported by the current Web Platform.

 

Initial public proposal

None

 

TAG review

https://github.com/w3ctag/design-reviews/issues/636 

 

TAG review status

Pending

 

Risks

 

Interoperability and Compatibility

Other browsers implement an ability to write custom clipboard data in varied shapes.  Part of the value of this work is to standardize the names of the formats that will be written per platform and to ensure a common shape of data on the clipboard so that browsers can read and write from this standard set of pickled formats.

 

Gecko: https://bugzilla.mozilla.org/show_bug.cgi?id=860857 

 

WebKit: No signal

 

Web developers: Positive signal from Figma & Sketchup. Internal MS office products have shown interest in supporting this API.

 

Is this feature fully tested by web-platform-tests?

No

 

Flag name

TBD

 

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=106449

 

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5649558757441536

 

[Mike Taylor]

Hi there,

On 5/13/21 10:46 AM, 'Anupam Snigdha' via blink-dev wrote:

Interoperability and Compatibility

Other browsers implement an ability to write custom clipboard data in varied shapes.  Part of the value of this work is to standardize the names of the formats that will be written per platform and to ensure a common shape of data on the clipboard so that browsers can read and write from this standard set of pickled formats.

 

Gecko: https://bugzilla.mozilla.org/show_bug.cgi?id=860857

It would probably be good to file an issue for a position from Mozilla now, ahead of an I2S: https://github.com/mozilla/standards-positions/issues/new
(that issue indeed is related, but there's no real discussion of pickling, beyond a mention in Comment 19)

WebKit: No signal

You can also email the webkit-dev list and ask for a position. See bit.ly/blink-signals.

Is this feature fully tested by web-platform-tests?

No

Do y'all intent do add WPTs (even just manual ones)?

thanks,
Mike

[Anupam Snigdha]

Ah, sorry I forgot to mention that we’ve actually filed request for positions from Mozilla and Apple:

https://github.com/mozilla/standards-positions/issues/525
https://lists.webkit.org/pipermail/webkit-dev/2021-May/031855.html

 

Re: Do y'all intent do add WPTs (even just manual ones)?
Yes, before sending i2s, we will update the Clipboard API spec and add WPT tests as well.

 

Thanks,

Anupam

[Gary Kačmarčík (Кошмарчик)]

I would like to see a Risks section added to the Explainer because there are a few aspects that may not be immediately apparent on a casual reading.

There are 2 parts to this proposal:

(a) The shape of the API to read/write the pickled data

(b) The format of pickled data on the native clipboard

Each of these carries different risks if we get it wrong and need to make an update:

For (a), we need to update all browsers and convince devs to migrate to the new API

For (b), we need to update all browsers and get all native apps that use the data to update.

(a) is pretty typical for a web api, but (b) is very different and backwards compatibility is a concern if we need to make a change. How do we envision handling making an update to the picking format?

Related to this, I think we need to:

(1) Have a design for all platforms before we commit to this. In particular, I'm concerned about support for Apple's clipboard/pasteboard format.

(2) Explicitly request feedback on both (a) and (b) when we solicit comments from browser vendors, native app developers and the TAG.

Also, I don't recall the details but ISTR an earlier discussion that raised some concerns about PII (like a file path or owner name) possibly being included in the clipboard header for raw data on macOS. We should investigate to make sure that's not a problem. Adding a Privacy section would be good and it could also cover any concerns around having these custom types on the clipboard (eg: fingerprinting).

And while we're at it, a Security section that discusses using this API to place specially-crafted malicious data on the clipboard should be included. We sanitize common data types to avoid this with the regular clipboard API, but this new API opens up (by design) a whole new world of new data types to attack, and we should acknowledge that and make it easy for readers to discover.

Finally, do we have a spec that covers the details of the pickling format on the native clipboards? I started working one on a year or so ago, but never finished a complete draft. I feel that getting the details right on this will be hard without this format being spelled out.

Having said all that, I'm excited to see this moving forward!

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CH2PR00MB0680F0B192999874F93B889ACF519%40CH2PR00MB0680.namprd00.prod.outlook.com.

[Anupam Snigdha]

Thank you Gary for the detailed feedback and support for this proposal!

>  I would like to see a Risks section added to the Explainer

Added Risks section to the explainer: https://github.com/dway123/clipboard-pickling/blob/main/explainer.md#risks

 

>  How do we envision handling making an update to the picking format?

Once the Pickling format is standardized, native apps and sites would have to explicitly opt-in to use this API. This doesn't affect reading/writing of the standard formats such as html, plain-text etc if these formats are written along with custom formats so we don't expect any copy-paste regressions for the existing formats.

 

>  (1) Have a design for all platforms before we commit to this. In particular, I'm concerned about support for Apple's clipboard/pasteboard format.

Will investigate this and get back to you.

 

Added privacy and security section in the explainer: https://github.com/dway123/clipboard-pickling/blob/main/explainer.md#privacy-and-security

 

>  do we have a spec that covers the details of the pickling format on the native clipboards

Do you mean https://github.com/dway123/clipboard-pickling/blob/main/explainer.md#os-interaction-format-naming?

[Darwin Huang]

Thank you for taking lead on this and driving it forward! :)

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.