Intent to Ship: Screen Capture

[emi...@chromium.org]

Contact emails

emi...@chromium.org, nik...@chromium.org

 

Explainer

See design doc.

Design doc/Spec

W3C Spec: https://w3c.github.io/mediacapture-screen-share/

Design doc

Summary

This extension enables the acquisition of a user's display, or part thereof, in the form of a MediaStream. This enables a number of applications, including screen sharing using WebRTC.

Link to “Intent to Implement” blink-dev discussion

Link

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes. We support all the screen capture functionality, that has already been surfaced by extensions API.

Demo link

https://webrtc.github.io/samples/src/content/getusermedia/getdisplaymedia/

Risks

Interoperability and Compatibility

Chromium is late to implement this API as the functionality already exists in other browsers. We are looking to be compatible with all other implementations.

Edge: Shipped

Firefox: Shipped (an earlier version of API)

Safari: Added in WebKit, but there’s no UI implementation yet.

Web developers: They are interested in using this common and much simpler API rather than going through extensions in multiple steps to get the same result.

Ergonomics

MediaStream output should be compatible with other WebRTC 1.0 endpoints.

Activation

It should be relatively easy for developers to use this API quickly. However, we expect developers to adapt this API over time according to their application's needs; so the current usage of extension API should gradually decrease over time as the usage of this increases.

Is this feature fully tested by web-platform-tests? Link to test suite results from wpt.fyi.

Please link to the test suite.

Additional content browser tests.

Entry on the feature dashboard

Entry

[Tom Jones]

This sound like a huge security hole. How can we be sure it won't be the next fiasco on the front page of the newspaper?

Peace ..tom

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/644e6201-c380-4971-be38-1e92d6f4877a%40chromium.org.

[Daniel Bratell]

I'm very happy to see this coming! A couple of questions though.

How stable is the specification and will/can Firefox change to support it when it's finished? Also, has the spec had a TAG review? We do want one of those unless it's not needed for some reason.

Security, as exciting as this is, it's also scary. Is the explainer up to date here? If I understood it correctly, an approval will be persistent, but there will still be a media selector that prevents instant access (for a screenshot or similar) to the user's desktop? Just looking for confirmation.

The wpt.fyi test results are FAIL or non-existent for all browsers. Shouldn't this be supported by Edge already? I do see that it works in Edge (funny to capture your own window and get windows all the way down).

/Daniel - excited

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/644e6201-c380-4971-be38-1e92d6f4877a%40chromium.org.

--

/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

[Wez]

+1 to a dedicated section on the security story for this feature - there were some very specific forms of attack that prevented us moving forward with screen/window/tab capture via getUserMedia() before now.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/op.zqmmquq0rbppqq%40cicero2.linkoping.osa.

[Harald Alvestrand]

The "security and permissions" section of the spec is pretty hefty:

https://w3c.github.io/mediacapture-screen-share/#security-and-permissions

We've got some experience now due to having shipped the extension-gated screenshare API for a few years; the controls that the extension API has implemented seem, at first look, to have prevented the security catastophes that some predicted.

I think a security discussion at this particular time needs to focus on the differences between having an extension-gated API and having an API with a browser-specific permission popup - if there are factors that make one significantly worse than the other, we need to address them - but we're not in a greenfield here.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/644e6201-c380-4971-be38-1e92d6f4877a%40chromium.org.

--

/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/op.zqmmquq0rbppqq%40cicero2.linkoping.osa.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CALekkJfucMB6Xks--88wckxSyqvj9MypNBiyKyLYLBL%3D3VsZYQ%40mail.gmail.com.

[Philipp Hancke]

Am Di., 9. Okt. 2018 um 18:58 Uhr schrieb Daniel Bratell <bra...@opera.com>:

I'm very happy to see this coming! A couple of questions though.

So am I, given the planned removal of inline installation in M71 which will make installation of the extension that is required for this more painful.

 

How stable is the specification and will/can Firefox change to support it when it's finished? Also, has the spec had a TAG review? We do want one of those unless it's not needed for some reason.

+jib for Firefox.

Question: this is still throwing on passing constraints, e.g.

navigator.getDisplayMedia({video: {width: 1280, height: 720}}).catch(e => console.error(e.name, e))

even though constraints were recently reintroduced in the specification?

Security, as exciting as this is, it's also scary.

How was it not scary when this relied on extensions in the webstore? (or for hangouts a built-in extensions; some vendors are more equal than others? https://bugs.chromium.org/p/chromium/issues/detail?id=850536 didn't see much interest)

Mozilla has been shipping an equivalent API for a while now (Firefox 52+; prior to that this was whitelist-based) and we haven't heard of major security disasters.

Does this include tab capture for which the specification recommends elevated permissions in https://w3c.github.io/mediacapture-screen-share/#elevated-permissions.

Presumable thinking of something like the webstore but given the way webstore has killed inline installation without notice doesn't make me confident that this is a platform i will ever trust again as a developer.

Is the explainer up to date here? If I understood it correctly, an approval will be persistent, but there will still be a media selector that prevents instant access (for a screenshot or similar) to the user's desktop? Just looking for confirmation.

Give it a try in Chrome with experimental web platform features enabled:

https://webrtc.github.io/samples/src/content/getusermedia/getdisplaymedia/

The wpt.fyi test results are FAIL or non-existent for all browsers.

Shouldn't this be supported by Edge already? I do see that it works in Edge (funny to capture your own window and get windows all the way down).

Automating this is the tough part, i.e. which window to automagically pick.

https://bugs.chromium.org/p/chromium/issues/detail?id=459532 has some of the story.

Works well enough in practice even though I haven't tried with getDisplayMedia.

 

/Daniel - excited

Philipp -- tired of the lack of communication displayed in in https://bugs.chromium.org/p/chromium/issues/detail?id=326740#c93

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/op.zqmmquq0rbppqq%40cicero2.linkoping.osa.

[Wez]

Thanks Harald, I had meant in the Chromium-specific doc but as you point out the spec of course has links to discussion of the threads, including to the web security model.  What I don't see, though, is anything spelling out how the UA requirements for screen/window sharing mitigate those threats.

IIRC correctly the two main mitigations extensions had, versus web sites, were install-time permissions prompt, and the fact that an extension can be more-easily blocked/removed if it is reported as abusing the permissions it was granted, than a site can be. Obviously things have moved on since then, but as you suggest, spelling out the remaining differences/risks, and their mitigations, seems valuable.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/644e6201-c380-4971-be38-1e92d6f4877a%40chromium.org.

--

/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/op.zqmmquq0rbppqq%40cicero2.linkoping.osa.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[thomascli...@gmail.com]

and we haven't heard of major security disasters.

 

Boy that is a low bar. Has anyone performed a threat analysis of this?

For example, embed attack in image and some how get some other app to reclassify the file and make it executable.

 

thx ..tom

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADxkKiLAGyA8h7uG5XOf_3aHPrv_ZJMStSs401raQyKsgmHiUQ%40mail.gmail.com.

 

[win...@gmail.com]

On Tuesday, October 9, 2018 at 2:56:52 PM UTC-4, Philipp Hancke wrote:

Am Di., 9. Okt. 2018 um 18:58 Uhr schrieb Daniel Bratell <bra...@opera.com>:

How stable is the specification and will/can Firefox change to support it when it's finished? Also, has the spec had a TAG review? We do want one of those unless it's not needed for some reason.

+jib for Firefox.

Hi, getDisplayMedia() already works in Firefox using the adapter.js polyfill. Example: https://jsfiddle.net/jib1/q75yb8pf/

That polyfill maps the spec API to the older non-spec implementation Firefox has supported for a long time. We intend to implement and ship getDisplayMedia() natively, and are actively working on it in bug 1321221 (the native version will let the user choose between fullscreen or a window in the same prompt.)

The spec is fairly stable at this point.

Question: this is still throwing on passing constraints, e.g.

navigator.getDisplayMedia({video: {width: 1280, height: 720}}).catch(e => console.error(e.name, e))

even though constraints were recently reintroduced in the specification?

Firefox supports constraints here FWIW.

Security, as exciting as this is, it's also scary.

How was it not scary when this relied on extensions in the webstore? (or for hangouts a built-in extensions; some vendors are more equal than others? https://bugs.chromium.org/p/chromium/issues/detail?id=850536 didn't see much interest)

Mozilla has been shipping an equivalent API for a while now (Firefox 52+; prior to that this was whitelist-based) and we haven't heard of major security disasters.

We haven't, but it's also true it's scary. There are non-obvious risks around sharing web surfaces in particular, discussed in the spec, and in a threat analysis.

Firefox attempts to mitigate these concerns by showing an extra warning message in the user prompt's preview window whenever the user attempts to share a browser window or the full screen (where a browser window may be visible). See this blog for details.

With the dropping of the extension installation step, I hope Chrome is planning to show a similar warning.

Is the explainer up to date here? If I understood it correctly, an approval will be persistent,

The spec forbids persisting approval (it allows persisting denial only).

Cheers,

.: Jan-Ivar :.

[Niklas Enbom]

Screen capture security concerns is covered quite a bit in existing W3C and IETF specs:

https://w3c.github.io/mediacapture-screen-share/#security-and-permissions

https://tools.ietf.org/html/draft-ietf-rtcweb-security-10#section-4.1.1

I think it could be constructive to sort concerns into either disagreeing with the spec, or whether the Chrome implementation isn't honoring the spec. It could be worth noting that in regards to the W3C guidance we consider any screen content to belong to the elevated permission class, where the user needs to give active consent every time. 

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/1a11ddca-aa38-4e9d-95f8-a0c499e3916c%40chromium.org.

[jbru...@mozilla.com]

On Tuesday, October 9, 2018 at 5:53:11 PM UTC-4, Niklas Enbom wrote:

I think it could be constructive to sort concerns into either disagreeing with the spec, or whether the Chrome implementation isn't honoring the spec. It could be worth noting that in regards to the W3C guidance we consider any screen content to belong to the elevated permission class, where the user needs to give active consent every time. 

I'm confused. The spec defines active user consent and elevated permissions as two different forms of consent interaction. Which one are you referring to here?

.: Jan-Ivar :.

[Niklas Enbom]

Right, that wasn't clearly formulated. I meant that we don't split up the surface types in elevated or non-elevated permissions. Any surface type is treated the same.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/8d6e6cb2-9bdf-4e99-9a1f-961a3d5d9778%40chromium.org.

[jbru...@mozilla.com]

That would be an area where Chrome is not honoring the spec, which says that "Active user consent is sufficient where there is little or no risk of an application gaining information that the user did not intend to share." - which is true only of non-web surfaces.

It explicitly says: "It is strongly advised that elevated permissions be required to access any display surface that might be used to circumvent cross-origin protections for content. The key goal of this consent process is not just to demonstrate that a user intends to share content, but to also determine that the user exhibits an elevated level of trust in the application that is being granted access."

I don't see how this can be claimed to have been determined solely from the user selecting between surfaces.

.: Jan-Ivar :.

[Daniel Vogelheim]

Hi Emircan,

This feature needs both security & privacy reviews before shipping. Please file a launch bug and request the appropriate reviews (if not already done).

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/644e6201-c380-4971-be38-1e92d6f4877a%40chromium.org.

[emi...@chromium.org]

This feature needs both security & privacy reviews before shipping. Please file a launch bug and request the appropriate reviews (if not already done).

Here is the launch bug. Feel free to add comments there regarding reviews.

Question: this is still throwing on passing constraints, e.g.
navigator.getDisplayMedia({video: {width: 1280, height: 720}}).catch(e => console.error(e.name, e))
even though constraints were recently reintroduced in the specification?

I was holding on until the spec is updated to handle constraints/capabilities/settings in the initial implementation. It is still in WIP, bug tracker will be updated once this CL lands.

[Philip Jägenstedt]

We discussed this briefly in the API owners meeting today (Ojan, Chris, Daniel, Jochen, Yoav, me)

Given that this has shipped in other browsers and removes the need to use an extension to get this functionality, the case for shipping this is very strong.

The remaining risk is security and privacy, and that will be handled in the internal launch bug. This intent won't be blocked on that review, but concerns there would block shipping.

LGTM1

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/bc54856c-cfc5-4754-b2c1-74049103b91b%40chromium.org.

[oj...@google.com]

LGTM2

[Jochen Eisinger]

lgtm3

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/ea1272f1-9caa-4128-a003-3eb63fdaf56d%40chromium.org.