Intent to Deprecate and Remove: WebRTC's RTCStats of type "track" and "stream".

669 views
Skip to first unread message

Henrik Boström

unread,
Jan 27, 2023, 5:49:12 AM1/27/23
to blink-dev, Harald Alvestrand, Yoav Weiss, Alex Russell
Contact emails

Background
I attempted to remove this feature before but had forgotten to file an intent to deprecate, background here.

Specification
The getStats() API spec is here and it contains all the metrics. The deprecated metrics are also listed, but in the obsolete section. There's an open issue to remove obsolete metrics from the spec as soon as they are unshipped from modern browsers. This is considered a blocker for the document to reach Proposed Recommendation status.

Summary
WebRTC is a set of JavaScript APIs (spec) that allow real-time communication between browsers. For the relevant metrics being removed, we're only talking about the WebRTC use case that is sending or receiving audio or video (typically Video Conferencing use cases), not the data channel use cases that is a popular WebRTC use case, since data channel only use cases would never have any tracks/streams.

RTCPeerConnection.getStats() returns a map of string-to-objects, where each object is one of the dictionaries defined in the stats spec. The reason an app calls getStats() is mostly to report quality metrics (send and receive resolutions, bitrates, glitches, video QP, etc) which can be important for A/B experimentation. It can also be used in a way that impacts app logic or even UX inside the app. Most common use case I can think of: poll getStats() at 10 Hz and render volume bars for each participant based on volume levels from stats objects.

The deprecation in question is to remove some stats objects that were made obsolete several years ago: all metrics on the "track" dictionary have been moved to non-obsolete objects ("inbound-rtp", "outbound-rtp", "media-source"). Reasons for wanting to deprecate include:
  • Spec-compliance: needed for browser implementations to align and for the spec to become Proposed Recommendation.
  • Web compat: Firefox never implement "track" or "steam" due to them being obsolete.
  • Performance: the duplicated metrics make up ~40% of the stats report size, which can be a significant number of bytes in larger meetings and it is common for apps to poll getStats() 10 times per second.
  • Tech debt: unblock removal of 1400 LOC.
In the meantime, the obsolete metrics is duplicated in several places of the stats report.

Risks
- Impossible to properly measure usage
Because stats objects are exposed as JavaScript dictionaries, and because apps have to iterate all objects of the stats report in order to find the ones they are interested in or if they just dump all the data without filtering, there is no way to measure how big the dependency is on track in the real world.

While we lack use counters, we have some positive signs:
  • Because Firefox does not have "track" or "stream" stats, any app that can run on Firefox already exercises the paths of these not existing.
  • An experiment to "unship deprecated metrics" has been running at 50% Canary since October, giving developers testing Canary a heads-up. Nobody complained until the experiment reached Stable.
  • We got to 50% Stable in M109 and while we're in the process of rolling it back now due to breaking twilio-video.js, it's interesting to note that this is the only breakage we are aware of (that does not mean there aren't more breakages, but I believe this at least says something about the severity).
- Selenium et al typically starts browsers from fresh profiles and hence does not know the finch trial seed
The most likely explanation for breakage is not testing Canary or test environments not having access to Finch experiments. This makes the behavior on Stable a surprise.

- To have a Reverse Origin Trial or not to have a Reverse Origin Trial?
Migrating should require so few lines of code (look for stats.type == 'inbound-rtp' instead of stats.type == 'track', for example) that it seems to be a bigger hurdle for a developer to enroll in the trial than to simply fix their code.

- Compatiblity risk
There is one particular metric out of all metrics that, if you run Safari, does not exist on "inbound-rtp" yet. This can be a problem, but again is probably not a big problem because this particular metric was never implemented on Firefox so apps already need to survive without it, and it is very easy to write a fallback path for the Safari case:

let trackIdentifer = null;  // In Firefox this will never be set regardless.
if (inboundRtp.trackIdentifier) {
  // Spec-compliant browser.
  trackIdentifier = inboundRtp.trackIdentifier;
} else if (inboundRtp.trackId) {
  // Fallback-path for Safari or 1+ year old Chromium browsers.
  trackIdentifier = report.get(inboundRtp.trackId).trackIdentifier;
}

Proposal
Rollback to 0% Stable but keep the "unship deprecated" experiment at 50% Canary/Beta. Wait for Twilio to fix their issue and do another rollout attempt. Keep a slower rollout pace next time.

I see limited amount of value in a Reverse Origin Trial since it appears to be more effort to register to the trial than to fix the issue, if you are affected.

I do prefer to have the feature enabled-by-default in M111+ and overwrite that default via Finch rather than the other way around as to not "turn off the fire alarm" for non-Finch testing environments. I realize that is not perfect (what if you run in a non-Finch environment) but it would reduce overall risk.

Philipp Hancke

unread,
Jan 27, 2023, 6:41:23 AM1/27/23
to Henrik Boström, blink-dev, Harald Alvestrand, Yoav Weiss, Alex Russell
Thank you Henrik. I agree with one suggestion: only do default-off in M112+ (which is branching so you would just need to revert this commit on the M111 branch).
This gives developers another month to update (which itself should be quick) and then rolling it out to their customers and users (which takes time).

I hope that the 50% rollout caused enough incidents (even though you may never hear about some of them) to get the fixes in ASAP.

cheers

Philipp

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/5ecf1ea6-c16c-464a-b529-439e05e4feedn%40chromium.org.

Henrik Boström

unread,
Jan 27, 2023, 8:16:15 AM1/27/23
to blink-dev, philipp...@googlemail.com, blink-dev, Harald Alvestrand, Yoav Weiss, Alex Russell, Henrik Boström
Delaying the enabled-by-default to M112 is fine by me but I'll wait for a resolution here before taking action. Currently it is enabled-by-default in Canary.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Johnny Stenback

unread,
Jan 27, 2023, 1:24:58 PM1/27/23
to Henrik Boström, blink-dev, philipp...@googlemail.com, Harald Alvestrand, Yoav Weiss, Alex Russell
Is there an enterprise policy in place for this deprecation already? If not, adding one seems appropriate given the challenges of rolling out even simple fixes in some enterprise environments.

Thanks,
Johnny

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6a0b5bb9-addd-4a56-b053-1429bbaabe2dn%40chromium.org.

Henrik Boström

unread,
Jan 30, 2023, 5:13:29 AM1/30/23
to blink-dev, Johnny Stenback, blink-dev, philipp...@googlemail.com, Harald Alvestrand, Yoav Weiss, Alex Russell, Henrik Boström
On Friday, January 27, 2023 at 7:24:58 PM UTC+1 Johnny Stenback wrote:
Is there an enterprise policy in place for this deprecation already? If not, adding one seems appropriate given the challenges of rolling out even simple fixes in some enterprise environments.

One does not exist at the moment but I can add one.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Harald Alvestrand

unread,
Jan 30, 2023, 5:16:59 AM1/30/23
to Henrik Boström, blink-dev, Johnny Stenback, philipp...@googlemail.com, Yoav Weiss, Alex Russell
I'm not sure an enterprise policy is appropriate - I see the same problem with sunsetting the policy as with sunsetting the stat in general, and usage of enterprise policies is (as far as I know) far more opaque to us than origin trials or Finch feature usage.

Henrik Boström

unread,
Jan 30, 2023, 5:20:27 AM1/30/23
to blink-dev, Harald Alvestrand, blink-dev, Johnny Stenback, philipp...@googlemail.com, Yoav Weiss, Alex Russell, Henrik Boström
On Monday, January 30, 2023 at 11:16:59 AM UTC+1 Harald Alvestrand wrote:
I'm not sure an enterprise policy is appropriate - I see the same problem with sunsetting the policy as with sunsetting the stat in general, and usage of enterprise policies is (as far as I know) far more opaque to us than origin trials or Finch feature usage.

Oh, we definitely don't want that. If a flag is needed (other than the Finch one) then I would much rather do a Reverse Origin Trial in that case. I still think that has limited value but if that mitigates concerns then I'm supportive of it.

Yoav Weiss

unread,
Jan 31, 2023, 5:39:36 AM1/31/23
to Henrik Boström, blink-dev, Harald Alvestrand, Johnny Stenback, philipp...@googlemail.com, Alex Russell
Thanks for filing an intent and moving the trial to 0% on stable.

From responses on the other thread, it seems like there may be a few months of lag between the time developers notice this upcoming change and the time it'd reach users.
Do you know if a 3P deprecation trial would have better deployment latency?

Henrik Boström

unread,
Jan 31, 2023, 6:45:20 AM1/31/23
to blink-dev, Yoav Weiss, blink-dev, Harald Alvestrand, Johnny Stenback, philipp...@googlemail.com, Alex Russell, Henrik Boström, avas...@twilio.com
On Tuesday, January 31, 2023 at 11:39:36 AM UTC+1 Yoav Weiss wrote:
Thanks for filing an intent and moving the trial to 0% on stable.

From responses on the other thread, it seems like there may be a few months of lag between the time developers notice this upcoming change and the time it'd reach users.
Do you know if a 3P deprecation trial would have better deployment latency?

If a 3P deprecation trial still requires the affected websites to get the latest version of the affected library in order to get that trial then, if I understand correctly, I don't think this will help because it sounds like the next version of Twilio will contain the fix for this in which case the trial would not be needed.

It sounds like it is just a matter of waiting for updates to be pushed, but please correct me if I'm wrong about this +Anna and if a trial would help.

Henrik Boström

unread,
Feb 1, 2023, 5:49:36 AM2/1/23
to blink-dev, Henrik Boström, Yoav Weiss, blink-dev, Harald Alvestrand, Johnny Stenback, philipp...@googlemail.com, Alex Russell, avas...@twilio.com
Ping. Does my proposal to do enabled-by-default (so that testing environments don't miss this again) prior to having ramped this up to 100% make sense or is that considered bad practise?
Current status is ENABLED_BY_DEFAULT in M111 (currently Canary), which would need to be reverted if we either want to delay this to M112 or not do ENABLED_BY_DEFAULT at all until 100% launch.

Yoav Weiss

unread,
Feb 1, 2023, 10:35:03 AM2/1/23
to Henrik Boström, blink-dev, Harald Alvestrand, Johnny Stenback, philipp...@googlemail.com, Alex Russell, avas...@twilio.com
Enabling by default on Canary and then turning it off right before branch could definitely be a good way to get lab testers that test on Canary be aware of this.
At the same time, the release managers may not be excited about it, because it reduces Canary's testing relevance.

Maybe we can settle on having this enabled at the early stages of the Canary milestone, and then revert as we approach Beta?

Henrik Boström

unread,
Feb 2, 2023, 4:41:20 AM2/2/23
to blink-dev, Yoav Weiss, blink-dev, Harald Alvestrand, Johnny Stenback, philipp...@googlemail.com, Alex Russell, avas...@twilio.com, Henrik Boström
That makes sense, I'll work on a revert of the enabled-by-default soon.

By the looks of this comment it sounds like Twilio has released a fix to the bug, but I assume it will take some time for users to update their library.

Mike Taylor

unread,
Feb 3, 2023, 12:14:16 PM2/3/23
to Harald Alvestrand, Henrik Boström, blink-dev, Johnny Stenback, philipp...@googlemail.com, Yoav Weiss, Alex Russell
I agree with Johnny that an enterprise policy would be useful, at least for a few milestones.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOqqYVG_P0ji6Z-n0DoCua4nzJZN7S48o23%3DUpx_ELerqphfUg%40mail.gmail.com.


Harald Alvestrand

unread,
Feb 4, 2023, 3:28:49 AM2/4/23
to Mike Taylor, Henrik Boström, blink-dev, Johnny Stenback, philipp...@googlemail.com, Yoav Weiss, Alex Russell
What's the imagined scenario in which an enterprise policy would be useful?

The only place I could imagine it being relevant is if there exists a WebRTC application that is only used within a single enterprise (neither hosting nor usage exists outside the enterprise), and that WebRTC application depends on non-upgraded Twilio libraries.

I don't know that we have evidence that such applications exist.


Mike Taylor

unread,
Feb 6, 2023, 3:41:35 PM2/6/23
to Harald Alvestrand, Henrik Boström, blink-dev, Johnny Stenback, philipp...@googlemail.com, Yoav Weiss, Alex Russell
We don't know what we don't know, but it's not hard to imagine an in-house enterprise WebRTC application that is using "stats" or "track". Twilio is the breakage we know about (because a developer took the time to report a bug). Having a policy so an app continues to work while a fix is made is a good thing - and comes with the nice side effect of appearing on the Enterprise release notes, increasing awareness.

Harald Alvestrand

unread,
Feb 7, 2023, 3:10:40 AM2/7/23
to Mike Taylor, Henrik Boström, blink-dev, Johnny Stenback, philipp...@googlemail.com, Yoav Weiss, Alex Russell
Do we have trackable statistics on the usage of corporate policies?
ie if nobody uses the policy in 2 milestones, can we detect that and decide that it is not needed and delete it, or will we be as unsure as we are now?

Henrik Boström

unread,
Feb 7, 2023, 5:36:43 AM2/7/23
to blink-dev, Harald Alvestrand, Henrik Boström, blink-dev, Johnny Stenback, philipp...@googlemail.com, Yoav Weiss, Alex Russell, Mike Taylor
I have the same concern as Harald regarding corporate policies. Why not a Deprecation Origin Trial in that case for list of users and more concrete timeline?

Manjesh Malavalli

unread,
Feb 7, 2023, 3:03:46 PM2/7/23
to blink-dev, Henrik Boström, Harald Alvestrand, blink-dev, Johnny Stenback, philipp...@googlemail.com, Yoav Weiss, Alex Russell, Mike Taylor
Hi Team,

Enabling the removal of track and stream stats by default starting from M112 gives our customers time to migrate to the latest version of the Twilio Video SDK which stops depending on the removed stats.
Thank you very much for agreeing to postpone the rollout.

Manjesh

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Manjesh Malavalli

unread,
Feb 7, 2023, 4:17:10 PM2/7/23
to blink-dev, Henrik Boström, Harald Alvestrand, blink-dev, Johnny Stenback, philipp...@googlemail.com, Yoav Weiss, Alex Russell, Mike Taylor
Hi team,

Thank you for postponing the rollout to M112. This gives our customers some time to migrate to the newest version of the Twilio Video SDK, which does not depend on the removed stats. We will do an internal post-mortem regarding why our tests did not catch this sooner and make the necessary corrections on our end.

Thanks,

Manjesh

On Tuesday, February 7, 2023 at 2:36:43 AM UTC-8 Henrik Boström wrote:
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Chris Thompson

unread,
Feb 7, 2023, 4:19:35 PM2/7/23
to Manjesh Malavalli, blink-dev, Henrik Boström, Harald Alvestrand, Johnny Stenback, philipp...@googlemail.com, Yoav Weiss, Alex Russell, Mike Taylor
> Do we have trackable statistics on the usage of corporate policies?
> ie if nobody uses the policy in 2 milestones, can we detect that and decide that it is not needed and delete it, or will we be as unsure as we are now?

It is quite common in my experience to handle removals with an enterprise policy that is stated upfront to be temporary (i.e., the enterprise release notes give a deadline milestone), with no metrics needed.

Henrik Boström

unread,
Feb 8, 2023, 12:13:29 PM2/8/23
to blink-dev, Chris Thompson, blink-dev, Henrik Boström, Harald Alvestrand, Johnny Stenback, philipp...@googlemail.com, Yoav Weiss, Alex Russell, Mike Taylor, Manjesh Malavalli
Proposal based on our meeting:
- Requesting to make this ENABLED_BY_DEFAULT in M112 but to have a Reverse Origin Trial available with an end date date 3 milestones out: M115.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

Johnny Stenback

unread,
Feb 8, 2023, 12:45:14 PM2/8/23
to Henrik Boström, blink-dev, Harald Alvestrand, philipp...@googlemail.com, Yoav Weiss, Alex Russell, Mike Taylor
Deprecation trials require changes to servers which an enterprise may not have access to or ability to affect changes to, whereas enterprise policies are entirely within the enterprise's control. I'd recommend reaching out to the enterprise team for their perspective on usage policies and perspective on whether a policy should be included here or not (I'm still of the opinion that it's in our interest to include one).

Thanks,
Johnny

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Philip Jägenstedt

unread,
Feb 8, 2023, 5:54:12 PM2/8/23
to Johnny Stenback, Henrik Boström, blink-dev, Harald Alvestrand, philipp...@googlemail.com, Yoav Weiss, Alex Russell, Mike Taylor
LGTM1 for the plan to remove in 112 with reverse OT until 115.

We discussed in today's meeting whether we should also have an enterprise policy, but landed on not requesting it. The reason is that it's quite possible that this change only breaks Twilio and nothing else, at all. That's because of the shape of the API and that the change got to 50% stable in M109 without other breakage being reported. An enterprise policy would be the "belts and braces" approach, but there's also a cost to making changes like these more laborious that has to be weighed against the risk.

Removing things from the web platform is usually thankless work, but it is important to reduce complexity and converge with other browsers (interoperability) so I want to applaud the effort put into this 👏

Rick Byers

unread,
Feb 10, 2023, 11:23:58 AM2/10/23
to Philip Jägenstedt, Johnny Stenback, Henrik Boström, blink-dev, Harald Alvestrand, philipp...@googlemail.com, Yoav Weiss, Alex Russell, Mike Taylor
+1 to Philip's sentiment, thanks for putting the effort into this! Given that it got to 50% stable and this was the only issue reported and Twilio is happy with the plan, then I'm also reluctant to ask for more opt-outs. So LGTM2

But please circle back here if you hear of any other non-trivial breakage. If necessary we can postpone again with finch and add an enterprise opt-out. +1 to what Johnny says about enterprise policies being the only effective knob for some environments - we've heard of many cases where updating a software package is a multi-month process for enterprise customers of a software system, but where asking those customers to enable a policy in their fleet is relatively easy. It's generally not about software developed by the enterprise itself, but bought from a 3p who has a support contract. More details on the policy and purpose of it are here.

Rick

Rick Byers

unread,
Feb 10, 2023, 12:31:34 PM2/10/23
to Philip Jägenstedt, Johnny Stenback, Henrik Boström, blink-dev, Harald Alvestrand, philipp...@googlemail.com, Yoav Weiss, Alex Russell, Mike Taylor
Sorry missed the link, details at https://www.chromium.org/developers/enterprise-changes/

Rick

Yoav Weiss

unread,
Feb 15, 2023, 6:03:26 AM2/15/23
to Rick Byers, Philip Jägenstedt, Johnny Stenback, Henrik Boström, blink-dev, Harald Alvestrand, philipp...@googlemail.com, Alex Russell, Mike Taylor
LGTM3
Reply all
Reply to author
Forward
0 new messages