Intent to Prototype: Capture handle

190 views
Skip to first unread message

Elad Alon

unread,
May 11, 2021, 9:01:49 AM5/11/21
to blink-dev

Contact emails

elad...@chromium.org

Explainer

https://docs.google.com/document/d/1oSDmBPYVlxFJxb7ZB_rV6yaAaYIBFDphbkx5bXLnzFg/edit?usp=sharing

Specification

None

Summary

We introduce a mechanism that allows an application to opt-in to exposing certain information to other applications which are video-capturing it. This allows collaboration between capturing and captured applications. For example, a VC application that's video-capturing a tab where a presentation application lives, could expose user-facing controls in the VC tab for navigating the presentation in the captured tab.



Blink component

Blink>GetUserMedia>Tab

Motivation

Display-capturing applications can offer the user improved functionality if they know what they are capturing, assuming the captured application collaborates. This is already possible using steganography. We offer a simpler, more reliable and more secure mechanism, by which an application may advertise select information - origin, handle, ID, etc. Additionally, the mechanism we propose allows the captured application to select the audience - the entire web or select origins. Finally, our mechanism offers some confidence in the capturer that certain parts of the message, if included, are not spoofed: 1. Only the top-level application can set the capture-handle, whereas steganography-based methods are exposed to messages being displayed by embedded content. 2. The browser provides the origin of the captured application (if it opts-in to exposing it).



Initial public proposal

https://github.com/w3c/mediacapture-screen-share/issues/166

TAG review

None

TAG review status

Pending

Risks

Interoperability and Compatibility

None


Gecko: No official signal. Some resistance voiced over the argument that this makes getDisplayMedia "too attractive" and could hinder adoption of getViewportMedia. (There are no current plans to deprecate getDisplayMedia.)

WebKit: No official signal, but they contributed ideas to early revisions and seemed generally positive.

Web developers: Google Meet very interested. Additional developers will be approached soon.


Is this feature fully tested by web-platform-tests?

No

Flag name

CaptureHandle

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1200910

Launch bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1200907

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/4854125411958784

This intent message was generated by Chrome Platform Status.

Yoav Weiss

unread,
May 17, 2021, 9:42:12 AM5/17/21
to Elad Alon, blink-dev
On Tue, May 11, 2021 at 3:01 PM 'Elad Alon' via blink-dev <blin...@chromium.org> wrote:

Contact emails

elad...@chromium.org

Explainer

https://docs.google.com/document/d/1oSDmBPYVlxFJxb7ZB_rV6yaAaYIBFDphbkx5bXLnzFg/edit?usp=sharing

Specification

None

Summary

We introduce a mechanism that allows an application to opt-in to exposing certain information to other applications which are video-capturing it. This allows collaboration between capturing and captured applications. For example, a VC application that's video-capturing a tab where a presentation application lives, could expose user-facing controls in the VC tab for navigating the presentation in the captured tab.



Blink component

Blink>GetUserMedia>Tab

Motivation

Display-capturing applications can offer the user improved functionality if they know what they are capturing, assuming the captured application collaborates. This is already possible using steganography. We offer a simpler, more reliable and more secure mechanism, by which an application may advertise select information - origin, handle, ID, etc. Additionally, the mechanism we propose allows the captured application to select the audience - the entire web or select origins. Finally, our mechanism offers some confidence in the capturer that certain parts of the message, if included, are not spoofed: 1. Only the top-level application can set the capture-handle, whereas steganography-based methods are exposed to messages being displayed by embedded content. 2. The browser provides the origin of the captured application (if it opts-in to exposing it).



Initial public proposal

https://github.com/w3c/mediacapture-screen-share/issues/166

TAG review

None

Might be worthwhile to start an early design review, to avoid surprises/delays further down the line.
 


TAG review status

Pending

Risks

Interoperability and Compatibility

None


Gecko: No official signal. Some resistance voiced over the argument that this makes getDisplayMedia "too attractive" and could hinder adoption of getViewportMedia. (There are no current plans to deprecate getDisplayMedia.)

WebKit: No official signal, but they contributed ideas to early revisions and seemed generally positive.

Web developers: Google Meet very interested. Additional developers will be approached soon.


Is this feature fully tested by web-platform-tests?

No

Flag name

CaptureHandle

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1200910

Launch bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1200907

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/4854125411958784

This intent message was generated by Chrome Platform Status.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMO6jDMCtcN8Kzu0d5B3DKh97-_eXD32K9QfgOx%2Bo8KLEJdvuQ%40mail.gmail.com.

Elad Alon

unread,
May 20, 2021, 12:05:04 PM5/20/21
to blink-dev, yoav...@chromium.org, blink-dev, Elad Alon
Thanks for the advice. My plan is laid out here on the intent-to-experiment. In a nutshell, implementation before the cut, then continue the discussion over this feature in the WebRTC WG meeting (2021-05-25), then present to TAG the (potentially-modified) design after that meeting. And since you've been involved in this feature now, I've picked you to review the last file on the last core CL. :-)
Reply all
Reply to author
Forward
0 new messages