Intent to ship WebRTC RTCRtpReceiver contributing and synchronization sources

[Nico Grunbaum]

Background: RTCRtpReceiver's[1] getSynchronizationSources() method
allows WebRTC applications to monitor volume levels of call participants
without having to use the costly getStats()[2] call. RTCRtpReceiver's
getContributingSources() method enables WebRTC applications to monitor
the volume level of participants whose audio is being mixed upstream if
the mixer includes this data in the RTP extension headers.

We intend to ship these interfaces in 59, and they are on nightly now.

Tracking: https://bugzilla.mozilla.org/show_bug.cgi?id=1363667

[1] https://w3c.github.io/webrtc-pc/#dom-rtcrtpreceiver
[2] https://w3c.github.io/webrtc-pc/#dom-rtcrtpreceiver-getstats()

Cheers,
Nico Grunbaum
-. --.

[Boris Zbarsky]

On 12/6/17 12:19 AM, Nico Grunbaum wrote:
> We intend to ship these interfaces in 59, and they are on nightly now.
>
> Tracking: https://bugzilla.mozilla.org/show_bug.cgi?id=1363667
>
> [1] https://w3c.github.io/webrtc-pc/#dom-rtcrtpreceiver
> [2] https://w3c.github.io/webrtc-pc/#dom-rtcrtpreceiver-getstats()

Would you mind filling out the other parts of the intent templates at
https://wiki.mozilla.org/WebAPI/ExposureGuidelines#Intent_to_Implement
and https://wiki.mozilla.org/WebAPI/ExposureGuidelines#Intent_to_Ship ?

At the very least, what is the status in other implementations and what
is the testing situation?

-Boris

[Nico Grunbaum]

Thank you for the link to the template Boris.

Intent to ship WebRTC RTCRtpReceiver contributing and synchronization
sources

Summary: RTCRtpReceiver's[1] getSynchronizationSources() method allows

WebRTC applications to monitor volume levels of call participants
without having to use the costly getStats()[2] call. RTCRtpReceiver's
getContributingSources() method enables WebRTC applications to monitor
the volume level of participants whose audio is being mixed upstream if
the mixer includes this data in the RTP extension headers.

It is in nightly now, and we intend to ship these interfaces in 59.
Chrome shipped a partial implementation[3] of getContributingSources()
in 59. Edge has partial support[4] for getContributingSources().

Bug to turn on by default:
https://bugzilla.mozilla.org/show_bug.cgi?id=1363667
Link to standard: https://w3c.github.io/webrtc-pc/#rtcrtpreceiver-interface
Testing: WPT are disabled, as the tests are themselves not functional[5]
and there is ongoing spec discussion as how to fix them[6].

[1] https://w3c.github.io/webrtc-pc/#dom-rtcrtpreceiver
[2] https://w3c.github.io/webrtc-pc/#dom-rtcrtpreceiver-getstats()

[3] https://bugs.chromium.org/p/chromium/issues/detail?id=703122&desc=2
[4] https://msdn.microsoft.com/en-us/library/mt502507(v=vs.85).aspx
[5] https://bugzilla.mozilla.org/show_bug.cgi?id=1419900
[6] https://github.com/w3c/webrtc-pc/pull/1668#issuecomment-346951615

-- Nico Grunbaum

On 12/6/17 5:42 AM, Boris Zbarsky wrote:
> Would you mind filling out the other parts of the intent templates at
> https://wiki.mozilla.org/WebAPI/ExposureGuidelines#Intent_to_Implement
> and https://wiki.mozilla.org/WebAPI/ExposureGuidelines#Intent_to_Ship ?
>
> At the very least, what is the status in other implementations and
> what is the testing situation?
>
> -Boris

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

[Boris Zbarsky]

On 12/6/17 3:22 PM, Nico Grunbaum wrote:
> It is in nightly now, and we intend to ship these interfaces in 59.
> Chrome shipped a partial implementation[3] of getContributingSources()
> in 59. Edge has partial support[4] for getContributingSources().

OK. Neither of those ships getSynchronizationSources(), right?

> Testing: WPT are disabled, as the tests are themselves not functional[5]
> and there is ongoing spec discussion as how to fix them[6].

Do we have other (non-wpt) tests for this running in our CI?

Have we done any checking of how interoperable what we're proposing to
ship is with what the other UAs have shipped?

I just filed a few issues on the spec here, because some parts of it are
not making sense to me:

* https://github.com/w3c/webrtc-pc/issues/1688
* https://github.com/w3c/webrtc-pc/issues/1689
* https://github.com/w3c/webrtc-pc/issues/1690

For the first two, it's a question about the API shape that is being
specified. For the last one, it's an issue where the spec is not clear
enough to implement interoperably, from what I can tell, and testing to
see what various shipping implementations do might be interesting.

-Boris

[Nico Grunbaum]

Answers can be found inline.

-Nico

On 12/7/17 12:45 PM, Boris Zbarsky wrote:
> On 12/6/17 3:22 PM, Nico Grunbaum wrote:
>> It is in nightly now, and we intend to ship these interfaces in 59.
>> Chrome shipped a partial implementation[3] of
>> getContributingSources() in 59. Edge has partial support[4] for
>> getContributingSources().
>
> OK.  Neither of those ships getSynchronizationSources(), right?
>

Correct those are not shipping in Chrome or Edge yet. Chrome has issued
an intent to ship:
https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/I39cDWKyMJo
.  I have not found any such signals with regards to Edge.

>> Testing: WPT are disabled, as the tests are themselves not
>> functional[5] and there is ongoing spec discussion as how to fix
>> them[6].
>
> Do we have other (non-wpt) tests for this running in our CI?
>

Yes, we have coverage via mochitests.

> Have we done any checking of how interoperable what we're proposing to
> ship is with what the other UAs have shipped?

No, not yet, and I agree with your statements below on why this is
important to complete.

>
> I just filed a few issues on the spec here, because some parts of it
> are not making sense to me:
>
> * https://github.com/w3c/webrtc-pc/issues/1688
> * https://github.com/w3c/webrtc-pc/issues/1689
> * https://github.com/w3c/webrtc-pc/issues/1690
>
> For the first two, it's a question about the API shape that is being
> specified.  For the last one, it's an issue where the spec is not
> clear enough to implement interoperably, from what I can tell, and
> testing to see what various shipping implementations do might be
> interesting.
>

Yes, the last one in particular is an interesting point and an open
issue https://github.com/w3c/webrtc-pc/issues/1497. I looks to be an
2018 interim topic.  There also appears to be discussion of this issue
on the Chrome bug for getContributingSources,
https://bugs.chromium.org/p/chromium/issues/detail?id=625208#c11

FWIW Our implementation makes sure that timestamp field is comparable to
Date.now() so that it will also be comparable to timestamps in
getStats(), https://www.w3.org/TR/webrtc/#dom-rtcstats-timestamp .

[Tanvi Vyas]

Is there a pref to turn this added functionality off? That way users who
are worried about their fingerprint can change the about:config pref? Or
perhaps it can be disabled when privacy.resistFingerprinting is set to true?

Thanks!

~Tanvi

On Tue, Dec 5, 2017 at 9:19 PM, Nico Grunbaum <ngru...@mozilla.com> wrote:

> Background: RTCRtpReceiver's[1] getSynchronizationSources() method allows

> WebRTC applications to monitor volume levels of call participants without
> having to use the costly getStats()[2] call. RTCRtpReceiver's
> getContributingSources() method enables WebRTC applications to monitor the
> volume level of participants whose audio is being mixed upstream if the
> mixer includes this data in the RTP extension headers.
>

> We intend to ship these interfaces in 59, and they are on nightly now.
>
> Tracking: https://bugzilla.mozilla.org/show_bug.cgi?id=1363667
>

> [1] https://w3c.github.io/webrtc-pc/#dom-rtcrtpreceiver
> [2] https://w3c.github.io/webrtc-pc/#dom-rtcrtpreceiver-getstats()
>

> Cheers,
> Nico Grunbaum
> -. --.
>

[Eric Rescorla]

Can you explain why you think this is an increased fingerprinting surface?
The data in question here is the audio level of *incoming* media, and as
the bug indicates, there are other ways to obtain it.

-Ekr

[Boris Zbarsky]

On 12/7/17 4:47 PM, Nico Grunbaum wrote:
> Correct those are not shipping in Chrome or Edge yet. Chrome has issued
> an intent to ship:
> https://groups.google.com/a/chromium.org/forum/#!topic/blink-dev/I39cDWKyMJo

Thank you for the link!

> Yes, we have coverage via mochitests.

Great.

> No, not yet, and I agree with your statements below on why this is
> important to complete.

Great. Let's make sure we do that before we ship this.

> FWIW Our implementation makes sure that timestamp field is comparable to
> Date.now()

How? They use different clocks.... What are we actually doing to
produce this number?

> so that it will also be comparable to timestamps in
> getStats(), https://www.w3.org/TR/webrtc/#dom-rtcstats-timestamp .

As I noted in the issue, those aren't specified in a sane way either.

-Boris

[Tanvi Vyas]

Thanks for clarifying ekr! I hadn't read the full details in the bug and
github pages.