Intent to Ship: RTCPeerConnection.id

[Elad Alon]

Contact emails

elad...@google.com

Explainer

Explainer doc

Spec

Link to public discussion

Summary

A field is added to RTCPeerConnection, which exposes an identifier for the peer connection, which can be used to refer to it outside of a JavaScript application.

The first user of this feature is Chrome’s Hangouts extension, for the purpose of enabling WebRTC event logging.

Link to “Intent to Implement” blink-dev discussion

Link

Link to Origin Trial feedback summary

The feedback (see here and here) indicated that this is unlikely to be added to the standard. We would like to ship this anyway, to benefit Chrome’s use case, described in the summary.

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Risks

Interoperability and Compatibility

There is no risk to breaking existing applications, as this is new, and is not yet used by any applications.

Interoperability with other browsers is a concern. Attempts to standardize this field were rejected. If other browsers are interested in the future, they might reopen the discussion to standardize.

Ergonomics

No performance issues are expected to be introduced by this feature.

Activation

• It will not be challenging for developers to take advantage of this feature immediately, as-is.

• This feature will not benefit from having polyfills, significant documentation and outreach, and/or libraries built on top of it.

Is this feature fully tested by web-platform-tests? Link to test suite results from wpt.fyi.

Since standardization was rejected, web-platform-tests will not be created. Instead, the feature is tested using these LayoutTests.

Entry on the feature dashboard

Link (will be modified to not refer to Origin Trial if shipping is approved)

[Boris Zbarsky]

On 4/20/18 8:17 AM, 'Elad Alon' via blink-dev wrote:
> The first user of this feature is Chrome’s Hangouts extension, for the
> purpose of enabling WebRTC event logging.

If this API is meant for a specific extension, does it make sense to
consider only exposing it to extensions, not the web at large?

-Boris

[Elad Alon]

Is there a way to do this, that I might be unaware of?

[Philipp Hancke]

2018-04-20 14:17 GMT+02:00 'Elad Alon' via blink-dev <blin...@chromium.org>:

Contact emails

elad...@google.com

Explainer

Explainer doc

Spec

Link to public discussion

Summary

A field is added to RTCPeerConnection, which exposes an identifier for the peer connection, which can be used to refer to it outside of a JavaScript application.

The first user of this feature is Chrome’s Hangouts extension, for the purpose of enabling WebRTC event logging.

Looking at https://cs.chromium.org/chromium/src/chrome/browser/resources/hangout_services/thunk.js?q=choosedesktopmedia&dr=CSs

this api seems to be private:

chrome.webrtcLoggingPrivate

Will anything but the hangouts extension be able to use this and be published to the chrome webstore?

FWIW, putting this id into the "id" field of the https://w3c.github.io/webrtc-stats/#dom-rtcpeerconnectionstats (which is currently 'RTCPeerConnection') would not require any changes to the API.

Link to “Intent to Implement” blink-dev discussion

Link

Link to Origin Trial feedback summary

The feedback (see here and here) indicated that this is unlikely to be added to the standard. We would like to ship this anyway, to benefit Chrome’s use case, described in the summary. 

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Risks

Interoperability and Compatibility

There is no risk to breaking existing applications, as this is new, and is not yet used by any applications.

Interoperability with other browsers is a concern. Attempts to standardize this field were rejected. If other browsers are interested in the future, they might reopen the discussion to standardize.

Ergonomics

No performance issues are expected to be introduced by this feature.

Activation

• It will not be challenging for developers to take advantage of this feature immediately, as-is.

• This feature will not benefit from having polyfills, significant documentation and outreach, and/or libraries built on top of it.

Is this feature fully tested by web-platform-tests? Link to test suite results from wpt.fyi.

Since standardization was rejected, web-platform-tests will not be created. Instead, the feature is tested using these LayoutTests.

Entry on the feature dashboard

Link (will be modified to not refer to Origin Trial if shipping is approved)

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMO6jDNGcsZZ4dPNRQrhJ64HynpPVKmFJf9500AnD4qFJ17Vpg%40mail.gmail.com.

[Boris Zbarsky]

On 4/20/18 12:20 PM, 'Elad Alon' via blink-dev wrote:
> Is there a way to do this, that I might be unaware of?

Exposing APIs to extensions but not web content? I assume this can be
done, since extensions can do things web content isn't allowed to do...

-Boris

[Elad Alon]

• Yes, we want to make this available to anyone who wants to use it; we still think it would be good to eventually add it to the standard.
  
• Putting the ID in RTCPeerConnectionStats, as well as other approaches, were discussed and rejected. (My apologies that I cannot produce the rejection reason for this particular approach atm. If you think it's important, I could dig through earlier correspondences and find it.)
  

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Jochen Eisinger]

Why does this ID need to be in the web platform, if it's only used by an extension?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMO6jDP4M0eV6_rWa-Qz16oOMb8GjZL_jVGseDKDaBBYhEUpBg%40mail.gmail.com.

[Elad Alon]

The extension lives on a different process, and so cannot be given a reference to the RTCPeerConnection. Connecting the two by use of an ID exposed to JS, was the best solution we've managed to come up with so far. Alternative suggestions are always welcome, though.

[PhistucK]

The fact that there are (already) private APIs just for Hangouts makes me think that the web platform does not provide everything that is needed for a serious web messaging/call/conference application. Instead of adding yet another private/proprietary API, perhaps consider adding the other stuff that Hangouts requires to the standard?

This way -

- You might not need that thunk.js (and others) and could get the statistics from the actual peer connection object, which the web platform already provides, as far as I know.

- The web platform would be more usable for serious web applications.

☆PhistucK

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMO6jDPZR16bdUdWK8REHeMMV8rErFPgU%3DfgOx1w-yRDUdPbHQ%40mail.gmail.com.

[Rick Byers]

We discussed this proposal in an API Owners meeting last week, and then foolip@ and I met with Elad and Björn to discuss it in greater detail.

At a high level, our minimum bar for shipping APIs which lack support from other implementors/standards groups is that there is compelling demand from developers (or a single developer of a large/popular site) where they will use the API to address an important use case on the web.  It's conceivable that someone might have such a use case for this API, but at the moment the only demand we're aware of is for supporting non-standard extensions which doesn't itself contribute to the "moving the web forward" part of the equation in the launch process.  

The API owners feel that extension-focused use cases should be addressed only via extension APIs, not by adding APIs to the standards-based web.

Event.isTrusted is the only example I know of where we've shipped an API primarily due to the value it provides in extension use cases.  But in that case there was already broad implementation support and an established specification for the API, so we were actually lowering interop risk by shipping support in Chrome, and so felt the tradeoff was justified.  If other browsers were interested in shipping RTCPeerConnection.id for their own extension-specific use cases, then I'd be happy for Chrome to ship it as well.

Elad is exploring other ways to achieve his use case without shipping a public API, while also continuing to explore whether there might be developers interested in using this API outside the context of extensions.

Rick

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMO6jDNGcsZZ4dPNRQrhJ64HynpPVKmFJf9500AnD4qFJ17Vpg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMO6jDP4M0eV6_rWa-Qz16oOMb8GjZL_jVGseDKDaBBYhEUpBg%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAMO6jDPZR16bdUdWK8REHeMMV8rErFPgU%3DfgOx1w-yRDUdPbHQ%40mail.gmail.com.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABc02_LreFjrjePd15EbM8ktJRG2QyTKcFUPH_OvBr0W4NeHNQ%40mail.gmail.com.

[Daniel Bratell]

Jumping over to blink-api-owners-discuss for a higher level question.

On Mon, 07 May 2018 19:24:46 +0200, Rick Byers <rby...@chromium.org> wrote:

We discussed this proposal in an API Owners meeting last week, and then foolip@ and I met with Elad and Björn to discuss it in greater detail.

Did you also discuss the bigger question: How to solve the use cases that have forced hangouts/meet to build an extension? Is there something that the web platform should get that would make that extension unnecessary?

/Daniel

--

/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

[Philip Jägenstedt]

On Tue, May 8, 2018 at 4:24 PM Daniel Bratell <bra...@opera.com> wrote:

Jumping over to blink-api-owners-discuss for a higher level question.

On Mon, 07 May 2018 19:24:46 +0200, Rick Byers <rby...@chromium.org> wrote:

We discussed this proposal in an API Owners meeting last week, and then foolip@ and I met with Elad and Björn to discuss it in greater detail.

Did you also discuss the bigger question: How to solve the use cases that have forced hangouts/meet to build an extension? Is there something that the web platform should get that would make that extension unnecessary?

We didn't spend too much time on that higher level question, because we'd need more stakeholders involved to really get anywhere.

In that area, however, I did take a look at thunk.js earlier today, pasting for your information. Quoting:

From skimming https://chromium.googlesource.com/chromium/src/+/cdb8e9984e33e2a59d3c5e80e8b80bc9a2ec7b5f/chrome/browser/resources/hangout_services/thunk.js#64:

 - Using https://developer.chrome.com/apps/system_cpu to get more information about the CPU than with web-exposed APIs

 - Many bits related to logging

 - Use of chrome.webrtcAudioPrivate.getSinks and getAssociatedSink. Ends up AudioSystem::GetDeviceDescriptions which presumably provides different information than navigator.mediaDevices.enumerateDevices(). (Not tested.)

- Something called "getNaclArchitecture", so there's NaCL being used for something somewhere (or it's dead code)

- setAudioExperiments, presumably for experiments within libwebrtc

- Screen sharing

- CPU, memory and network usage monitoring

End quote.

It would require a lot more investigation/discussion to work out a concrete plan to make the extension unnecessary. 

[Daniel Bratell]

On Tue, 08 May 2018 23:16:47 +0200, 'Philip Jägenstedt' via blink-api-owners-discuss <blink-api-ow...@chromium.org> wrote:

On Tue, May 8, 2018 at 4:24 PM Daniel Bratell <bra...@opera.com> wrote:

Jumping over to blink-api-owners-discuss for a higher level question.

On Mon, 07 May 2018 19:24:46 +0200, Rick Byers <rby...@chromium.org> wrote:

We discussed this proposal in an API Owners meeting last week, and then foolip@ and I met with Elad and Björn to discuss it in greater detail.

Did you also discuss the bigger question: How to solve the use cases that have forced hangouts/meet to build an extension? Is there something that the web platform should get that would make that extension unnecessary?

We didn't spend too much time on that higher level question, because we'd need more stakeholders involved to really get anywhere.

In that area, however, I did take a look at thunk.js earlier today, pasting for your information. Quoting:

From skimming https://chromium.googlesource.com/chromium/src/+/cdb8e9984e33e2a59d3c5e80e8b80bc9a2ec7b5f/chrome/browser/resources/hangout_services/thunk.js#64:

 - Using https://developer.chrome.com/apps/system_cpu to get more information about the CPU than with web-exposed APIs

Seems like this would have to be a protected API at best. But might also not be absolutely necessary for an app.

 - Many bits related to logging

Hmm. Ought to be possible to do without an extension you'd believe,

 - Use of chrome.webrtcAudioPrivate.getSinks and getAssociatedSink. Ends up AudioSystem::GetDeviceDescriptions which presumably provides different information than navigator.mediaDevices.enumerateDevices(). (Not tested.)

And this might be a chance for an improvement in  the official specs. Unless there is security reasons for what is there now. If it's actually needed.

- Something called "getNaclArchitecture", so there's NaCL being used for something somewhere (or it's dead code)

...

- setAudioExperiments, presumably for experiments within libwebrtc

Which maybe should run through the same system as other web platform experiments.

- Screen sharing

Ouch. This is the big one. I've never tried Hangouts screen sharing in Opera (no extension) so I assume it will not work and can not work while it works out of the box in Chrome thanks to the extension.

- CPU, memory and network usage monitoring

....

It would require a lot more investigation/discussion to work out a concrete plan to make the extension unnecessary. 

Yes. 

Screen sharing seems to be the big one and I can imagine that part never being possible through standard web platform APIs. (for some value of "never")

Thanks for checking. Still not sure how to get from here to a perfect future. :)

[PhistucK]

> Screen sharing seems to be the big one and I can imagine that part never being possible through standard web platform APIs. (for some value of "never")

Available in Edge now. navigator.getDisplayMedia. Chrome is simply late to the game.

Chrome also has it, behind a command line flag for web content (except Hangouts which got it despite the fact that no command line flag was specified, or it least so it was at some point if I understand correctly) and behind an extension API for extensions (Hangouts could have asked the users to install an extension, it was a poor design choice and some abuse of power, in my opinion).

☆PhistucK

--
You received this message because you are subscribed to the Google Groups "blink-api-owners-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To post to this group, send email to blink-api-ow...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/op.ziq7iwkorbppqq%40cicero2.linkoping.osa.

[Jochen Eisinger]

On Wed, May 9, 2018 at 7:40 PM PhistucK <phis...@gmail.com> wrote:

> Screen sharing seems to be the big one and I can imagine that part never being possible through standard web platform APIs. (for some value of "never")

Available in Edge now. navigator.getDisplayMedia. Chrome is simply late to the game.

The challenge is that this is a huge cross-origin leak (potentially), but probably hard to convey to the user: if you share your screen, the site might read confidential information from whatever other site you have open or might open while sharing your screen.

I guess one way to address this would be to only allow sharing same origin tabs (good enough for presenting a doc), or otherwise filter out cross origin pixels. Or have a CORS like mechanism (again, a presentation website could probably be convinced to send an appropriate CORS response).

[PhistucK]

Edge does not let you pick a tab, only entire application windows and screens ("displays", I assume they mean screens).

Your warning can show up on the source picker. :)

I do not necessarily agree that the user cannot understand that, but if this is in the way, then have a meeting and solve it. ;) It is better to have this, than to show blunt favoritism towards Hangouts (on Chrome OS, I can maybe understand, but on desktop Chrome? Nope).

☆PhistucK

You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CALjhuieCdtTaN1YaVDCwfDhOcsDG%3DU%3DL7WDNypmL33o2NJ_Erg%40mail.gmail.com.

[Philipp Hancke]

2018-05-11 13:01 GMT+02:00 Jochen Eisinger <joc...@chromium.org>:

On Wed, May 9, 2018 at 7:40 PM PhistucK <phis...@gmail.com> wrote:

> Screen sharing seems to be the big one and I can imagine that part never being possible through standard web platform APIs. (for some value of "never")

Available in Edge now. navigator.getDisplayMedia. Chrome is simply late to the game.

The challenge is that this is a huge cross-origin leak (potentially), but probably hard to convey to the user: if you share your screen, the site might read confidential information from whatever other site you have open or might open while sharing your screen.

I guess one way to address this would be to only allow sharing same origin tabs (good enough for presenting a doc), or otherwise filter out cross origin pixels. Or have a CORS like mechanism (again, a presentation website could probably be convinced to send an appropriate CORS response).

IIRC the CORS issue was discussed quite a bit. Martin Thomson who authored https://w3c.github.io/mediacapture-screen-share/ can probably discuss this for hours.

Firefox used to have a whitelist for this which could be modified either through an open process or by using an extension that modified the about:config settings.

They got rid of this in FF52 and now screensharing through getUserMedia (either display or window) is available to any secure origin.

This is not a new discussion, I filed https://bugs.chromium.org/p/chromium/issues/detail?id=416856 back in 2014...

Speaking having a screensharing extension with almost a million users in the store, forcing an inferior UX on everyone else is a (somewhat tiny) unfair advantage.

The UX is not that bad actually, inline installation can work without a reload if you know how. 

WRT to RTCPeerConnection.id, giving Hangouts the ability to use another private API that allows "better" WebRTC debugging is somewhat concerning (and its not even clear from the intent to ship what problem is solved).

This would be ok if the extension API was public and reasonably well documented, but the lack of documentation for chrome://webrtc-internals suggests this is not a priority.

 

 

Chrome also has it, behind a command line flag for web content (except Hangouts which got it despite the fact that no command line flag was specified, or it least so it was at some point if I understand correctly) and behind an extension API for extensions (Hangouts could have asked the users to install an extension, it was a poor design choice and some abuse of power, in my opinion).

☆PhistucK

On Wed, May 9, 2018 at 6:48 PM Daniel Bratell <bra...@opera.com> wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-discuss+unsub...@chromium.org.
To post to this group, send email to blink-api-owners-discuss@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/op.ziq7iwkorbppqq%40cicero2.linkoping.osa.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CALjhuieCdtTaN1YaVDCwfDhOcsDG%3DU%3DL7WDNypmL33o2NJ_Erg%40mail.gmail.com.

[Harald Alvestrand]

Webrtc-internals is only documented in source......

To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owners-d...@chromium.org.
To post to this group, send email to blink-api-ow...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/op.ziq7iwkorbppqq%40cicero2.linkoping.osa.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Jochen Eisinger]

If edge and ff already ship screensharing, maybe somebody should just send an intent to ship and see what security review comes back with?

[Philip Jägenstedt]

I agree, actually trying to move it forward towards shipping is the only thing that will get this issue resolved.

[PhistucK]

Do we know for sure whether this is the only real hinderance (forget logging for a moment)?

☆PhistucK

[Philip Jägenstedt]

I would expect the other bits to be real problems as well.

[Christian Biesinger]

Looks like Hangouts works in Firefox now:
https://blog.mozilla.org/webrtc/firefox-is-now-supported-by-google-hangouts-and-meet/

Does it support screensharing there?

Christian

https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYdwckTBp%3DacQ3CCNdOzOoO5b-TJrHgeDhCXPw3StvjajA%40mail.gmail.com
.

[Philip Jägenstedt]

Yes, I've tested locally with a Chrome-to-Firefox meet.google.com meeting, and I could share my whole screen with Firefox.

[Ojan Vafai]

Do we know how Edge and Firefox are handling the cross-origin leak? Is it just a permission? That's a pretty scary permission IMO (roughly: allow this site to access any of your information on any website that can be put in an iframe).

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners-discuss/CAARdPYew2shxAQD2gaDh0uM0HHG5sNGQmcVpZY%3D93uyr_WcO1Q%40mail.gmail.com.

[Ojan Vafai]

But ditto what Jochen said: it seems like a security review via an intent-to-ship is the right next step.

[Philip Jägenstedt]

Firefox had a chooser UI similar to getUserMedia with a yellow box with warning text about risks. Can't take a screenshot now I'm afraid.