Intent to Ship: Element Capture
Elad Alon
Contact emails
Explainer
https://github.com/screen-share/element-capture/blob/main/README.md
Specification
https://screen-share.github.io/element-capture
Summary
API for capturing a subtree of the DOM.
Given a video MediaStreamTrack obtained through pre-existing means to initiate tab-capture, Element Capture allows mutating the track to only capture a subtree of the DOM starting at a given Element.
The API bears some resemblance to the Region Capture API, but affords greater flexibility for applications, because occluding and occluded content are both excluded from the capture.
Blink component
TAG review
https://github.com/w3ctag/design-reviews/issues/954
TAG review status
Issues addressed
Chromium Trial Name
ElementCapture
Origin Trial documentation link
https://github.com/screen-share/element-capture/blob/main/README.md
WebFeature UseCounter name
kElementCapture
Risks
Interoperability and Compatibility
1. Other browser vendors have not indicated an intention to implement and ship this feature; however, we are working with them to ensure that our own implementation would most closely align with their own vision; see details in “Anticipated spec changes” below.
2. There is no compatibility risk because this is a completely new feature.
Gecko:
https://github.com/mozilla/standards-positions/issues/857
Currently marked as negative, but discussions in the joint SCCG – WebRTC WG meeting during TPAC 2024 (minutes), indicate that Mozilla’s position is likely to change if we only allow track.restrictTo() on tracks derived by getViewportMedia(). We intend to adopt this restriction once getViewportMedia() is shipped. Mozilla has been formally asked whether this would change their position; no response yet given.
WebKit:
No signal
https://github.com/WebKit/standards-positions/issues/280
Web developers: Strongly positive
LibreStream
“This is perfect. We have a video collaboration app where we need to share documents (e.g. PDFs) so users can walk through the document together. As part of this, we have a document viewer that the sharer needs to share dur-ing the call and scroll through different pages where others can annotate on the shared document. Prior to Element Capture, this was a lousy user experience since the entire window needed to be displayed and didn’t allow annotations from the sharer. Thank you!”
Engagely
“I highly support adopting this specification. We at Engageli intend to use it for our session recorder, and for collaboration tools.”
Tango
“I can't emphasize enough how instrumental this specification would be for our product and user experience.”
RGO Communications
“We support this API. We have not participated in the origin trial due to prioritization issues; however, we intend to use the API after it's shipped.”
AudioPump
“At AudioPump, Inc., we have several projects that use similar APIs and will likely also implement Element Capture. There are no specific plans at the moment, and the current restrictions cut down on the use cases, but I'm sure once it becomes available for users, we will implement something.”
Tella
“I support adopting the Element Capture specification. We could in the future use this to do cleaner screen captures in our chrome extension. We'd let the user "inspect" the page and pick the part they'd want to capture more specifically.”
Mux
“I support adopting the Element Capture specification.“
Zoom
“Element capture has its use case. it is different as region capture. Up for it.“
Dialpad
“I support adopting the Element Capture specification.”
Chromatic
“It could also help for tools like https://www.chromatic.com/ for visual regression testing!”
Storitto + Audiocado
“We use the browser to create videos for Storrito.com and Audiocado.com, the described feature would have saved me many headaches.”
Remotion
“I am also very interested in this API and second that it will be important to get the frame in an uncompressed way. For Remotion, we will be able to provide in-browser video rendering if this API ships and generally, this would open the door for web-based offline video editors to become more powerful (the status quo is that only canvas-based graphics can be exported as images).“
Debuggability
No changes to DevTools are intended.
Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)?
No
This API is supported on all desktop platforms. Mobile platforms are unsupported because screen-capture itself is unsupported on those platforms.
Is this feature fully tested by web-platform-tests?
Yes
Flag name on about://flags
element-capture
Finch feature name
ElementCapture
Tracking bug
https://bugs.chromium.org/p/chromium/issues/detail?id=1418194
Launch bug
https://launch.corp.google.com/launch/4240472
Sample links
https://element-capture-demo.glitch.me
Estimated milestones
Anticipated spec changes
At the moment, Chromium’s implementation of Element Capture works with `getDisplayMedia({preferCurrentTab: true})`, which is Chromium’s temporary stopgap for getViewportMedia(). In the future, after getViewportMedia() is fully specified, implemented and gains wide adoption, we will limit Element Capture to only work with tracks derived from getViewportMedia() calls, and deprecate the ability to call restrictTo() on tracks obtained with anything other than getViewportMedia().
Link to entry on the Chrome Platform Status
https://chromestatus.com/feature/5198989277790208?gate=5129718719840256
Links to previous Intent discussions
Intent to Prototype: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMO6jDO6y5b6y3q9QEd2scsYPWuWLJBnPLgwm%2BaHpKx36CYMwA%40mail.gmail.com
Intent to Experiment: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMO6jDN8mcO%2BYqaVA5nb5BBv-dZB0wqwfh9580wMc-e%2BNuP7yw%40mail.gmail.com
Daniel Clark
Do you have a rough idea of how long do we expect to be in this intermediate state where Element Capture is available with getDisplayMedia() rather than getViewportMedia()? Presumably the longer that lasts the harder it will be to remove the getDisplayMedia() way of doing it once getViewPortMedia() is available.
-- Dan
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
blink-dev+...@chromium.org.
To view this discussion visit
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMO6jDPwMu%3D0_RCdRKrwa-w5Egj29priVsB%3DJwgXOby2QKkrQQ%40mail.gmail.com.
Elad Alon
Once cross-origin isolation enjoys wide adoption, I see no blockers to deprecating EC-over-gDM in favor of EC-over-gVM.
Chris Harrelson
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/d85a8a96-a5ca-4c99-ad91-0de9a07c3baen%40chromium.org.
Mike Taylor
LGTM2
Daniel Bratell
LGTM3
I think the privacy concerns are similar enough to getRegionCapture that it's acceptable to ship this without the suggested getViewportMedia hooks. That said, I do find it concerning that we're not able to create cross-browser compatibility in the near future.
/Daniel
To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/c4a0a878-f573-400c-8479-e7dd8e704397%40chromium.org.