Intent to Experiment: Captured Surface Control

563 views
Skip to first unread message

Elad Alon

unread,
Jan 17, 2024, 8:47:43 AMJan 17
to blink-dev

Contact emails

elad...@chromium.org

Explainer

https://github.com/screen-share/captured-surface-control/blob/main/README.md

Specification

TBD, but will be hosted on https://screen-share.github.io/captured-surface-control once produced.
For the time being, please refer to the explainer, which has a detailed description of the API as well as sample use of it.
A demo is also available. (Some of the functionality is still being landed in Chromium atm.)

Design docs

https://docs.google.com/document/d/10UojDvTJ6ojBEOP7cgBIIaE7WZEfes_Qv1eN3A2A0nM/edit?usp=sharing

Summary

Web API that allows Web applications to: 1. Produce wheel events in a captured tab or window. 2. Read/write the zoom level of a captured tab.



Blink component

Blink>GetDisplayMedia

TAG review

Not started.

TAG review status

Pending. We expect that developer feedback during the origin trial might lead to non-trivial changes to the API shape, and would therefore like to hold off on TAG review until after those changes.

Risks



Interoperability and Compatibility



Gecko: No signal

WebKit: No signal

Web developers: No signals

Other signals:

Security

https://github.com/screen-share/captured-surface-control?tab=readme-ov-file#security-and-privacy-concerns



WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?



Goals for experimentation

Validate the API shape with Web developers and gather feedback on such topics as:
  • Usefulness of the API as currently implemented.
  • Usefulness of allowing the API to be invoked while the capturing page is not focused (currently, the capturing page must be focused).
  • Possible pain points with scrolling as currently specified and implemented. (Is more fine-grained control necessary? Is scrolling too janky as currently specified and implemented?)

Ongoing technical constraints

None

Debuggability

N/A

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)?

No (Supported on all desktop platforms. Screen-sharing is not currently supported on mobile platforms.)

Is this feature fully tested by web-platform-tests?

No (but we're working on it)

Flag name on chrome://flags

captured-surface-control

Finch feature name

CapturedDisplaySurface

Requires code in //chrome?

False

Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1466247

Launch bug

https://launch.corp.google.com/launch/4268170

Estimated milestones

OriginTrial desktop last127
OriginTrial desktop first122
DevTrial on desktop122


Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5092615678066688

Links to previous Intent discussions

Intent to prototype: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMO6jDPSgR3kX39drHd9t-JvTKBk%2B7Dg03O6dvowzw-LjQ__1A%40mail.gmail.com

This intent message was generated by Chrome Platform Status.

Elad Alon

unread,
Jan 17, 2024, 9:09:29 AMJan 17
to blink-dev, Elad Alon
A quick clarification, btw, that we have a partner in mind that's eager to start origin trial as soon as m122.
We also heard interest from other parties during Screen Capture CG meetings.

Chris Harrelson

unread,
Jan 17, 2024, 7:48:41 PMJan 17
to Elad Alon, blink-dev
LGTM to experiment M122 - M127

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/f6f5c1b3-d203-4fa5-bdef-394e4c04ab02n%40chromium.org.

Mike Taylor

unread,
Jan 17, 2024, 8:32:32 PMJan 17
to Elad Alon, blink-dev

LGTM to experiment from M122 to M127 inclusive.

--

Mike Taylor

unread,
Jan 17, 2024, 8:33:24 PMJan 17
to Elad Alon, blink-dev

Oops, missed Chris' email on this. But you're double approved now. :)

Elad Alon

unread,
May 8, 2024, 4:13:01 PMMay 8
to blink-dev, mike...@chromium.org, Elad Alon

Hello Blink owners,


We are asking that the OT period on this experiment be shifted, and instead of the originally requested period of m122-m127 (inclusive), we consider it retroactively as m124-m129 (inclusive).


As of the time of writing, there has only ever been one active OT participant (Meet), and they only started widespread use of the feature in m124. Further, the feature was not fully implemented until m124. (The initial target of m122 was set based on timelines that did not materialize, both in terms of the feature’s implementation and of participants’ implementation.)


With respect to standardization:

When this API was first presented to Web developers in the Screen Capture CG, we heard a lot of support for it. Since then, we prioritized implementation and validation of the API with partners. Now that we have the thumbs-up on implementation from partners, we are prioritizing standardization. An initial outreach to Mozilla and Apple during the WebRTC WG editors meeting started out positively, and I am looking forward to presenting this to a wider audience during the May interim meeting of the WebRTC WG. The prospects on achieving consensus before m129 runs its course appear good.


Thanks,

Elad

Chris Harrelson

unread,
May 8, 2024, 4:21:00 PMMay 8
to Elad Alon, blink-dev, mike...@chromium.org
Sounds fine to me - consider it renumbered from 124.

Elad Alon

unread,
May 8, 2024, 4:23:09 PMMay 8
to Chris Harrelson, blink-dev, mike...@chromium.org
Reply all
Reply to author
Forward
0 new messages