Intent to Extend Experiment: Capture all screens
Simon Hangl
Hello blink-dev,
We’d like to ask for an extension to our Origin Trial, from M124 to M130. This is due to the dependency on isolated web apps.
Contact emails
Explainer
https://github.com/screen-share/capture-all-screens/blob/main/README.md
Specification
https://screen-share.github.io/capture-all-screens
Design docs
https://screen-share.github.io/capture-all-screens
https://github.com/screen-share/capture-all-screens/blob/main/README.md
Summary
Capture all the screens currently connected to the device using getAllScreensMedia(). Calling getDisplayMedia() multiple times requires multiple user gestures, burdens the user with choosing the next screen each time, and does not guarantee to the app that all the screens were selected. getAllScreensMedia() improves on all of these fronts. (As this feature has extreme privacy ramifications, it is only exposed behind an enterprise policy, and users are warned before recording even starts, that recording *could* start at some point.)
Blink component
TAG review
https://github.com/w3ctag/design-reviews/issues/856
TAG review status
Complete
Chromium Trial Name
GetAllScreensMedia
Link to origin trial feedback summary
https://github.com/screen-share/capture-all-screens/issues
Origin Trial documentation link
https://github.com/screen-share/capture-all-screens
Risks
Interoperability and Compatibility
This API is only available to origins allowlisted by administrators through a policy. The policy itself is non-standard, limiting even theoretical interoperability.This API rejects requests from pages that are not allow-listed through an administrator. The likelihood of this API being adopted by a browser that does not provide administrators mechanisms to manage clients is low.
Gecko: N/A
WebKit: N/A
Web developers: Positive (https://github.com/screen-share/capture-all-screens/issues/9)
Other signals:
Ergonomics
No
Activation
The challenge for developers is the limitation of the API to origins allowlisted by an enterprise policy.
Security
1. Risk of malicious sites exploiting the API and gaining access to sensitive information on users' devices. This risk is mitigated by the API only being accessible to origins allowlisted by an enterprise policy.
2. Risk of users loading private information that gets recorded and made available to apps affiliated with their device's admin. This risk is mitigated by informing users that recording might start at any moment before the API becomes accessible. (In CrOS, this warning is delivered in the log-in screen, and when users log-in despite the warning, this is tantamount to assent.)
3. Risk of users forgetting that their screens are being recorded. This risk is mitigated through a persistent notification.
Goals for experimentation
Learn about the experience of web developers and how this API fulfills their needs.
Reason this experiment is being extended
This API will eventually be released for isolated contexts, which are delayed. Hence, we are asking for an extension of the origin trial.
Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, ChromeOS, Android, and Android WebView)?
No
This API is initially implemented on CrOS, where demand for it is greatest, and where we have the most flexibility in offering users early warning that their screens may be recorded if they proceed past the log-in screen. Lessons learned from shipping this API on CrOS will be used when deciding how to correctly implement such warnings on other platforms.
Is this feature fully tested by web-platform-tests?
No, as WPTs don’t support setting of managed policies. The API is tested by a number of unit- and browser- tests (Test files).
DevTrial instructions
https://github.com/screen-share/capture-all-screens/blob/main/HOWTO.md
Flag name on chrome://flags
enable-get-all-screens-media
Finch feature name
None
Non-finch justification
None
Requires code in //chrome?
True
Tracking bug
https://issues.chromium.org/issues/40216442
Launch bug
https://launch.corp.google.com/launch/4201060
Estimated milestones
Link to entry on the Chrome Platform Status
https://chromestatus.com/feature/6284029979525120
Links to previous Intent discussions
Intent to prototype: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAEdDZo9N354i6eST0x19TXwpeBtgs5_gJUYVF%2BTKLpiJySDADg%40mail.gmail.com
Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/6TRT0XsVOE4/m/NOm-YEQCAgAJ
Simon Hangl
Software Engineer
Google Germany GmbH
Erika-Mann-Straße 33
80636 München
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Diese E-Mail ist vertraulich. Falls Sie diese fälschlicherweise erhalten haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail an die falsche Person gesendet wurde.
This e-mail is confidential. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person.
Yoav Weiss (@Shopify)
Hello blink-dev,
We’d like to ask for an extension to our Origin Trial, from M124 to M130. This is due to the dependency on isolated web apps.
Contact emails
Explainer
https://github.com/screen-share/capture-all-screens/blob/main/README.md
Specification
https://screen-share.github.io/capture-all-screens
Design docs
https://screen-share.github.io/capture-all-screens
https://github.com/screen-share/capture-all-screens/blob/main/README.md
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAP0TkgE%2B58Z59SF%3Dpbvrbx8ovaUFC8V0uki7bovVSKg6GAbxOg%40mail.gmail.com.
Vladimir Levin
Hello blink-dev,
We’d like to ask for an extension to our Origin Trial, from M124 to M130. This is due to a dependency on isolated web apps, which are delayed.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAP0TkgF1BfhsLRadATibKed4vQUoV8_PqA_xUUZdXSSFcGZW%2Bw%40mail.gmail.com.
Simon Hangl
Daniel Bratell
Any reason to not make it available for everyone? Asking for a friend.
Another thing, when extending experiments we want to see evidence of substantial progress on the feature so that it doesn't just roll along until it's burned in by pure inertia. Could you please tell us about the progress since the last extension?
/Daniel
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/dad681d8-8adb-4530-bf59-3604c8bc5047n%40chromium.org.
Simon Hangl
Simon Hangl
Yoav Weiss (@Shopify)
- Dependency injection
- The initial intent indicated dependency on Enterprise Policy, rather than IWAs.
- I see some reasoning for the new dependency in the design doc's security considerations, but it seems incomplete
- e.g. why couldn't you enforce CSP and TrustedTypes as a requirement for this regardless of IWA? How does bundling help when allowing one app to leak information from others? Wasn't there controls in place limiting the origins that can do that as part of the Enterprise Policy?
- I may be missing context as a lot of the links in that doc are still Google-only
- Timelines
- The initial trial went from 118 to 124.
- On this thread I see you started by asking for an extension from 124 to 130, and then switched to asking for 129 to 132.
- At the same time, I don't believe the OT was put on hold when 124 was released.
- What happened between M124 and M128?
- Progress towards shipping
- On top of that, no evidence of substantial progress towards shipping was demonstrated. Again, the design doc still contains many Google-only links, so I may be missing context here, but this section feels very much like a soft launch. The Origin Trial risks we are trying to avoid don't seem to have been carefully considered.
Reilly Grant
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6aee109d-77a7-4a01-b4d9-3fcbb4e06b36n%40chromium.org.
Simon Hangl
Thanks for your response, Yoav. Please find my answers to your questions below:
ad 1) “Why not CSP + trusted types instead of IWAs”: We discussed this with Artur, who initially flagged the vulnerability here. We do enforce these requirements. The API is only exposed in contexts that meet certain requirements on client-side XSS mitigation. These are necessary but not sufficient, as server-side XSS remains a meaningful risk in the absence of the packaging/signing guarantees of IWAs. We're managing that risk during this experimental period via Enterprise policy requirements on the one hand, and OT registration on the other.
ad 2) “What happened between M124 and M128”: We did clarify with the Blink owners whether we could extend the origin trial until M136, to ensure partners can already work with the API followed by the transition to IWAs. The origin trial accidentally was created longer than the formal 6 milestones (see discussions here), which I realized after I applied for extension on this thread. While we did clarify whether we can extend the origin trial with the timelines above, I sincerely apologize for not following the formal process.
ad 3) “Progress towards shipping”: We acknowledge that with our approach we went beyond the intent of an origin trial. We did however check in advance with the Blink owners whether we could follow this approach due to exceptional circumstances in order to
get this API into the hands of selected web developers and
timebox the temporary solution through origin trial to make sure this API does not remain on the drive-by web.
The origin trial is essential to keep current developer momentum and grant enough time for the selected developers to prepare the API launch in context of IWA. Good evidence for progress towards shipping can be seen by the multitude of IWA related work to prepare the upcoming launch (IWA Launch).
I hope to have answered your questions sufficiently. Please let me know if you have any further concerns or follow-up questions.