Intent to Implement: WebRTC Frame Logging API

69 views
Skip to first unread message

Sergey Ulanov

unread,
May 23, 2017, 6:04:31 PM5/23/17
to blin...@chromium.org, h...@chromium.org
Contact emails
 
Spec
 
This specification has been proposed as a WICG item, but not yet transferred there.
 
Summary
WebRTC Frame Event Logging API will allow web applications that use WebRTC to record timestamps when each frame is processed by different parts of the processing pipeline. Each frame is uniquely identified, which allows to correlate timestamps on both ends of the connection and then measure total end-to-end latency for each individual frame. This will allow to measure click-to-effect performance, camera-to-screen lag and so on.
 
Motivation
Applications that use WebRTC APIs do not control directly the data sent over media streams. Browser is responsible for all steps of media processing from the capture device on the sender side (microphone or camera) to the render device on the receiver (audio playback device or screen). This means that currently it’s not possible for an application to measure end-to-end performance of a WebRTC connection. It's also not possible to know how much time is spent in each part of the pipeline.
 
Interoperability and Compatibility Risk
The API will be used mainly for performance evaluation. Application that use WebRTC will still work on browsers that don’t support the new API, but they won’t be able to measure end-to-end performance.
 
Edge: No signals
Firefox: No signals
Safari: No signals
Web developers: Positive. Several teams at Google expressed interest. Particularly it will be useful for Chrome Remote Desktop.
 
Compatibility risk
None. This is a new API that doesn’t affect other features.

Ongoing technical constraints
None.
 
Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?
Yes.

OWP launch tracking bug
 
Link to entry on the feature dashboard
 
Requesting approval to ship?
No
 
 

Timothy Dresser

unread,
May 24, 2017, 8:03:13 AM5/24/17
to Sergey Ulanov, blin...@chromium.org, igri...@chromium.org, pani...@chromium.org, h...@chromium.org
+igri...@chromium.org +pan...@chromium.org 

Could this information be exposed via the Performance Timeline / Performance Observer?
At a cursory glance, there appears to be a fair bit of overlap.

Tim

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2B1DSjib6i41Es4zf7tYZOOFtuHUokAV5c7nP9O_EnKatrfabQ%40mail.gmail.com.

Rick Byers

unread,
May 24, 2017, 11:58:35 AM5/24/17
to Timothy Dresser, Sergey Ulanov, blink-dev, igri...@chromium.org, pani...@chromium.org, Harald Alvestrand
Working on this use case sounds awesome to me, thank you!  In general, having the ability to measure latency in the wild is really important :-)

But yes please work with the web perf folks to try to come up with a design that fits well with our broader web perf design.  Eventually you'll want to request a TAG review, and fitting in with the rest of the platform is one of the main things the TAG looks for.  I assume there could be scenarios where someone really wants to combine WebRTC and nativation or input performance data, right?

On Wed, May 24, 2017 at 8:02 AM, Timothy Dresser <tdre...@chromium.org> wrote:
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHTsfZDCockqbvxuaeicSYXD-HAuGENV2Nk%3DPuTyxsO1n9ikHw%40mail.gmail.com.

Sergey Ulanov

unread,
May 24, 2017, 5:37:39 PM5/24/17
to Timothy Dresser, blin...@chromium.org, igri...@chromium.org, pani...@chromium.org, h...@chromium.org
On Wed, May 24, 2017 at 5:02 AM, Timothy Dresser <tdre...@chromium.org> wrote:
Could this information be exposed via the Performance Timeline / Performance Observer?
At a cursory glance, there appears to be a fair bit of overlap.

We haven't considered it. It does appear related.  I'll take closer look to the Performance Timeline spec to figure out if it can work for this use-case.
Thank you!

Shubhie Panicker

unread,
May 24, 2017, 5:45:17 PM5/24/17
to Sergey Ulanov, Timothy Dresser, blink-dev, igri...@chromium.org, pani...@chromium.org, h...@chromium.org
Re: Performance Observer
We discussed this briefly but didn't follow up, I bumped the relevant thread from previous discussion

On Wed, May 24, 2017 at 2:37 PM, Sergey Ulanov <ser...@chromium.org> wrote:
On Wed, May 24, 2017 at 5:02 AM, Timothy Dresser <tdre...@chromium.org> wrote:

Jochen Eisinger

unread,
May 29, 2017, 11:09:32 AM5/29/17
to Shubhie Panicker, Sergey Ulanov, Timothy Dresser, blink-dev, igri...@chromium.org, pani...@chromium.org, h...@chromium.org
If you go ahead with the proposed API, please also add a security & privacy considerations section!

On Wed, May 24, 2017 at 11:45 PM 'Shubhie Panicker' via blink-dev <blin...@chromium.org> wrote:
Re: Performance Observer
We discussed this briefly but didn't follow up, I bumped the relevant thread from previous discussion
On Wed, May 24, 2017 at 2:37 PM, Sergey Ulanov <ser...@chromium.org> wrote:
On Wed, May 24, 2017 at 5:02 AM, Timothy Dresser <tdre...@chromium.org> wrote:
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAK7ODi_2SKxbe%2BiqODiSP_8Wy%2BBpniNRCpbhaTaZkoTLZ8MM1Q%40mail.gmail.com.

Ilya Grigorik

unread,
Jun 2, 2017, 7:57:49 PM6/2/17
to Jochen Eisinger, Shubhie Panicker, Sergey Ulanov, Timothy Dresser, blink-dev, pani...@chromium.org, h...@chromium.org
FWIW, we did review WebRTC Stats API [1], which resulted in a few open issues:

- https://github.com/w3c/webrtc-stats/issues/119 - closed with no action.. and still has me wondering.

The latter discussion is the core of the issue: as designed, it doesn't play well with third-party/RUM use cases, but it's also not clear to me if that's an important use case for WebRTC -- my personal hunch is that it should be, but hey.. :). Whatever is the resolution there, applies to this extension as well..

ig


On Mon, May 29, 2017 at 8:09 AM, Jochen Eisinger <joc...@chromium.org> wrote:
If you go ahead with the proposed API, please also add a security & privacy considerations section!

On Wed, May 24, 2017 at 11:45 PM 'Shubhie Panicker' via blink-dev <blin...@chromium.org> wrote:
Re: Performance Observer
We discussed this briefly but didn't follow up, I bumped the relevant thread from previous discussion
On Wed, May 24, 2017 at 2:37 PM, Sergey Ulanov <ser...@chromium.org> wrote:
On Wed, May 24, 2017 at 5:02 AM, Timothy Dresser <tdre...@chromium.org> wrote:
Reply all
Reply to author
Forward
0 new messages