Intent to Experiment: Element Capture

1.020 görüntüleme
İlk okunmamış mesaja atla

Elad Alon

okunmadı,
20 Kas 2023 06:41:0020.11.2023
alıcı blink-dev

Contact emails

elad...@chromium.org


Explainer

https://github.com/screen-share/element-capture/blob/main/README.md


Specification

https://screen-share.github.io/element-capture


Summary

API for capturing a subtree of the DOM.


Given a video MediaStreamTrack obtained through pre-existing means to initiate tab-capture, Element Capture allows mutating the track to only capture a subtree of the DOM starting at a given Element.


The API bears some resemblance to the Region Capture API, but affords greater flexibility for applications, because occluding and occluded content are both excluded from the capture.



Blink component

Blink>GetDisplayMedia


TAG review

We are holding off on the request for a TAG review until we get some more developer feedback about the API shape.


TAG review status

Pending


Risks



Interoperability and Compatibility



Gecko: Under consideration (https://github.com/mozilla/standards-positions/issues/857)


WebKit: No signal (https://github.com/WebKit/standards-positions/issues/280)


Web developers:

Positive See upvotes and comments on the following:


Other signals:


WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?



Goals for experimentation

* Solicit more informed Web developer feedback to validate the API shape.

* Ensure that the feature works correctly in conjunction with adjacent features.



Debuggability

No changes to DevTools are intended.



Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

No

This API is supported on all desktop platforms. Mobile platforms are unsupported because screen-capture itself is unsupported on those platforms.



Is this feature fully tested by web-platform-tests?

Not yet (but we’re working on extending coverage)


Flag name on chrome://flags

element-capture


Finch feature name

ElementCapture


Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=1418194


Launch bug

https://launch.corp.google.com/launch/4240472


Estimated milestones

Shipping on desktop

121



Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5198989277790208


Links to previous Intent discussions

Intent to prototype: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMO6jDO6y5b6y3q9QEd2scsYPWuWLJBnPLgwm%2BaHpKx36CYMwA%40mail.gmail.com


This intent message was generated by Chrome Platform Status.


Mike Taylor

okunmadı,
20 Kas 2023 11:29:3120.11.2023
alıcı Elad Alon, blink-dev

Hi Elad,

Can you clarify which milestones you intend to experiment in?

thanks,
Mike

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMO6jDN8mcO%2BYqaVA5nb5BBv-dZB0wqwfh9580wMc-e%2BNuP7yw%40mail.gmail.com.

Elad Alon

okunmadı,
20 Kas 2023 11:35:1120.11.2023
alıcı blink-dev, mike...@chromium.org, blink-dev, Elad Alon
Hi Mike. I'm asking for m121 to m126 (inclusive).

Mike Taylor

okunmadı,
20 Kas 2023 11:39:2820.11.2023
alıcı Elad Alon, blink-dev

Thanks, LGTM to experiment from 121 to 126 inclusive.

Jan-Ivar Bruaroey

okunmadı,
27 Kas 2023 09:53:5127.11.2023
alıcı blink-dev, Elad Alon
The Gecko position has been updated to negative. See https://github.com/mozilla/standards-positions/issues/857

Elad Alon

okunmadı,
27 Kas 2023 10:00:1127.11.2023
alıcı blink-dev, jbru...@mozilla.com, Elad Alon
Thank you for updating, Jan-Ivar. I have now updated this information in the ChromeStatus entry. I am looking forward to carry on the discussion about your position in your standards-positions GitHub repo, in the Element Capture spec repo, in the Screen Capture CG and in the WebRTC WG.

Elad Alon

okunmadı,
10 May 2024 06:44:5310 May
alıcı blink-dev, Elad Alon, jbru...@mozilla.com

Hello Blink owners,


We are asking for a breaking period of 3 weeks to this API, followed by a renewed experiment for the traditional 6 milestones (2024-May-14 to 2024-Jun-04). A request that seems similar to me was granted in January 2022 for another API, and later shipped successfully with consensus with Mozilla and Apple.


At the time of writing, we have public support for this API from such companies as Zoom, Jitsi, Mux, DialPad, Whereby, Intel and Tella. Tango’s co-founder wrote “I can't emphasize enough how instrumental this specification would be for our product and user experience.” However, none of them have signed up for the OT as of yet. 


From the OT perspective:

This API allows Web developers to build novel new features; however, it requires a non-trivial investment. We are hoping that after giving more time, we will see more participation.


From the standardization perspective:

We need time to pick up discussions with Mozilla and Apple again. As Chrome’s Security and Privacy experts do not share the concerns Mozilla and Apple have previously voiced, it stands to reason that additional discussions will allow us to converge - and we will prioritize these discussions now.


Additionally, some possibilities remain for API changes that could perhaps allow for a compromise, mostly around cross-origin isolation. (Full disclosure - this is not my ideal outcome, but it’s a fallback possibility worth mentioning.)


Progress made:

  • Spec: The spec has evolved and is now more mature, dealing better with such edge cases as loss of “eligibility for restriction.”

  • TAG: Previously we held off on the request for a TAG review until we got some more developer feedback about the API shape. Having received this initial feedback, the TAG request has now been submitted.

  • Signals: Signals have been requested. Mozilla responded. We intend to prioritize this discussion with them now.

  • Feedback: Outreach for feedback from the spec community - multiple issues were filed on the spec by Web developers.

  • WPT: Coverage has recently been extended.


Reasons to run a new trial:

Gain additional feedback from new participants. Examples for remaining areas where new feedback could help include:

  • Uncover new edge cases which were not uncovered by the spec authors and reviewers, implementers and current OT participants. The current edge cases here demonstrate how non-obvious these may be.

  • Validate (or refute) the assumptions we have made about the viability of an MVP that is missing some functionality. Examples:

    • Events notifying apps when an element stops/starts being “eligible for restriction”.

    • Mechanism to force elements into an “eligible for restriction” state.

  • Encourage future OT participation by Web developers, by demonstrating that risks associated with relying on an origin trial, while real, are partially offset by a commitment to keep the OT going until discussions conclude.


Thanks,

Elad

Mike Taylor

okunmadı,
10 May 2024 16:25:0310 May
alıcı Elad Alon, blink-dev, jbru...@mozilla.com

Note: Because you're asking for a renewal of 6 milestones, this will require 3 LGTMs.

Can you clarify which milestones you're asking for? You've written "(2024-May-14 to 2024-Jun-04)" - that range maps to M125 and M126 stable releases.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Elad Alon

okunmadı,
14 May 2024 12:08:1614 May
alıcı blink-dev, mike...@chromium.org, jbru...@mozilla.com, Elad Alon, Panos Astithas
I'm asking for a break-off period starting July 1 and ending July 22 (inclusive). That means that the original trial will end on June 30, and be restarted on July 23. This corresponds to breaking the original experiment early during the cycle of what would have otherwise been its final milestone (m126), the restarting the experiment over the m127-m132 range. (CC Panos for feasibility.)

Mike Taylor

okunmadı,
14 May 2024 14:36:4314 May
alıcı Elad Alon, blink-dev, jbru...@mozilla.com, Panos Astithas

OK - thanks for the clarification. I would ask that you get in touch with every registered origin - expiring the experiment early may be surprising.

LGTM1 to experiment from M127 to M132 inclusive.

Chris Harrelson

okunmadı,
14 May 2024 14:40:4214 May
alıcı Mike Taylor, Elad Alon, blink-dev, jbru...@mozilla.com, Panos Astithas

Yoav Weiss (@Shopify)

okunmadı,
15 May 2024 10:41:3615 May
alıcı Chris Harrelson, Mike Taylor, Elad Alon, blink-dev, jbru...@mozilla.com, Panos Astithas

Elad Alon

okunmadı,
21 May 2024 06:05:5821 May
alıcı blink-dev, Yoav Weiss, mike...@chromium.org, Elad Alon, blink-dev, jbru...@mozilla.com, Panos Astithas, Chris Harrelson
Thank you for the LGTMs.

A quick update that we have discussed the matter with the OT Core team, and it appears to us all better to have the breaking period between June 24 (first day of break) and July 15 (first day of renewed experiment). This ensures that partners have time to mint a new token and roll it out to their production enviornment ahead of m127 shipping to Stable.

Danny Moerkerke

okunmadı,
17 Tem 2024 15:21:2617 Tem
alıcı blink-dev, Elad Alon, Yoav Weiss, mike...@chromium.org, blink-dev, jbru...@mozilla.com, Panos Astithas, Chris Harrelson
I registered a token for the origin trial but it doesn't seem to work. Is it not active yet?

Op dinsdag 21 mei 2024 om 12:05:58 UTC+2 schreef Elad Alon:

François Beaufort

okunmadı,
22 Tem 2024 02:48:1122 Tem
alıcı Danny Moerkerke, origin-tri...@google.com, blink-dev, Elad Alon, Yoav Weiss, mike...@chromium.org, jbru...@mozilla.com, Panos Astithas, Chris Harrelson
Note that I can also reproduce this issue with https://element-capture-demo.glitch.me/

image.png
@origin-tri...@google.com Can you help us fix this?
My understanding was that the breaking period was between June 24 and July 15. The issue happens for both old and new tokens.


François Beaufort

okunmadı,
22 Tem 2024 10:18:5222 Tem
alıcı James Scott, Danny Moerkerke, origin-tri...@google.com, blink-dev, Elad Alon, Yoav Weiss, mike...@chromium.org, jbru...@mozilla.com, Panos Astithas, Chris Harrelson
I see "Origin Trials - Version: 1.0.0.17" in chrome://components.
@James Scott Can you also reproduce on your side by going to https://element-capture-demo.glitch.me/?

On Mon, Jul 22, 2024 at 4:04 PM James Scott <james...@google.com> wrote:
Hi all,
The latest configuration (v1.0.0.17) to re-enable the feature went out last week. Could you go to chrome://components/ and check the version of the Origin Trials component and report back what you see?
image.png

Users will need to restart their browser after getting the latest version.

Best,
James

James Scott

okunmadı,
22 Tem 2024 10:56:5322 Tem
alıcı François Beaufort, Danny Moerkerke, origin-tri...@google.com, blink-dev, Elad Alon, Yoav Weiss, mike...@chromium.org, jbru...@mozilla.com, Panos Astithas, Chris Harrelson
Hi François,
On my Chromebook, the feature is enabled at https://element-capture-demo.glitch.me/. However, it only started working after I restarted my Chromebook following the 1.0.0.17 update.  The trial is also active on my Linux machine. Similarly, I had to run the `killall chrome` command in my Linux terminal to get it to work.
I have attached two screenshots.

Please let me know if you are still experiencing issues, and if I can be of any further assistance.

Best,
James
 
image.png
image.png

James Scott

okunmadı,
22 Tem 2024 10:56:5422 Tem
alıcı François Beaufort, Danny Moerkerke, origin-tri...@google.com, blink-dev, Elad Alon, Yoav Weiss, mike...@chromium.org, jbru...@mozilla.com, Panos Astithas, Chris Harrelson
Hi all,
The latest configuration (v1.0.0.17) to re-enable the feature went out last week. Could you go to chrome://components/ and check the version of the Origin Trials component and report back what you see?
image.png

Users will need to restart their browser after getting the latest version.

Best,
James

On Mon, Jul 22, 2024 at 2:47 AM François Beaufort <fbea...@google.com> wrote:

Danny Moerkerke

okunmadı,
24 Tem 2024 08:31:3124 Tem
alıcı blink-dev, James Scott, Danny Moerkerke, origin-tri...@google.com, blink-dev, Elad Alon, Yoav Weiss, mike...@chromium.org, jbru...@mozilla.com, Panos Astithas, Chris Harrelson, François Beaufort
Thanks James, that works! 
So users should just restart their browser once they have v1.0.0.17 of the component?

Op maandag 22 juli 2024 om 17:56:54 UTC+3 schreef James Scott:

James Scott

okunmadı,
24 Tem 2024 13:21:5724 Tem
alıcı Danny Moerkerke, blink-dev, origin-tri...@google.com, Elad Alon, Yoav Weiss, mike...@chromium.org, jbru...@mozilla.com, Panos Astithas, Chris Harrelson, François Beaufort
Hi Danny,

Happy to help! Yes, restarting the browser after updating to v1.0.0.17 is the best way to make sure the previous version's configuration is cleared and the component's new settings take effect immediately.

Best,
James
Tümünü yanıtla
Yazarı yanıtla
Yönlendir
0 yeni ileti