Re: [discuss-webrtc] Deprecated "track" and "stream" stats are unshipped in M109.

[Philipp Hancke]

Henrik,

while I understand that this has been ongoing for a while in Canary and Beta for a while without anyone raising issues,

the timeline "tomorrow" (with ramp-up) comes as a surprise, in particular considering practices like code freezes over the holidays.

Can you explain why you think you don't need an intent to deprecate on blink-dev (cc'd)?

(I agree they should go away but don't see a need for such an aggressive timeline)

Am Mo., 9. Jan. 2023 um 10:40 Uhr schrieb Henrik Boström <hb...@webrtc.org>:

With the Stable release of M109 tomorrow, the deprecated "track" and "stream" stats objects returned by RTCPeerConnection.getStats() will no longer be available.

This is unshipped at 1% Stable, but it will soon ramp up further with the goal of 100% unshipped in M109.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/0ebcadca-6d66-4f7c-b7e5-d2ac707a6e84n%40googlegroups.com.

[Alex Russell]

Thanks for adding blink-dev, Philipp. CC-ing the API OWNERs as this seems related to a pattern of breaking changes without Blink intents that we've been informed of by customers.

Do I understand correctly that this deprecation is being managed via Finch for 109 Stable?

Best,

Alex

You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADxkKiJ-vkiyVTH%2BsSNOjv67zx0p7qLra1ePdGJS3JW3GTzfKQ%40mail.gmail.com.

[Aaron Boushley]

Can you help me understand exactly which objects are being removed here? We rely on `RTCPeerConnection.getStats()` although we pass in a stream selector. We then iterate over the returned stats reports looking for ones containing the values we need.

Is this a removal of the stats objects that have the fixed ID of "track" and "stream"?

Is there any more documentation I can look at beyond the 2 sentences above?

Aaron

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CADxkKiJ-vkiyVTH%2BsSNOjv67zx0p7qLra1ePdGJS3JW3GTzfKQ%40mail.gmail.com.

Confidentiality Note: We care about protecting our proprietary information, confidential material, and trade secrets. This message may contain some or all of those things. Cruise will suffer material harm if anyone other than the intended recipient disseminates or takes any action based on this message. If you have received this message (including any attachments) in error, please delete it immediately and notify the sender promptly.

[Yoav Weiss]

+Henrik Boström - was there an intent sent for this removal? Any form of developer communication?

You received this message because you are subscribed to the Google Groups "blink-api-owners" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-api-owne...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-api-owners/CAA44PQjegbaaorsT_r-iNtUAg2Gxa5f6WohHAaLeGw2tmBW%2BwA%40mail.gmail.com.

[Henrik Boström]

On Tue, Jan 10, 2023 at 8:51 AM Yoav Weiss <yoav...@chromium.org> wrote:

+Henrik Boström - was there an intent sent for this removal? Any form of developer communication?

There was developer communication dating as far back as July but I admit I had forgotten to send out a formal blink-dev intent to deprecate!

- I should have done that.

The getStats() API in question is not being deprecated, but the RTCStatsReport (an id-to-stats-object map) report will stop containing the stats object which were made obsolete in the spec several years ago due to the contents of these stats objects having been moved to other stats objects that are still being returned. Same values, different location. In other words, the report is being trimmed down by removing duplicate information. Stats processing code in an application is gated on stats type for knowing which metrics to look for on an individual stats object which should make this lower risk compared to other depracations. The motivation for this is performance optimizations (~40% report size reduction), technical debt reduction (-1400 LOC) and web compat ("track" does not exist in Firefox).

The communication channel used was WebRTC's official google group, discuss-webrtc. History:

• July 25, 2022 PSA announced the plan to deprecate at a milestone TBD. This was also the time where the "DEPRECATED_" prefix was added to the JavaScript-exposed stats object IDs, which made it into M106. The deprecation prefix is also visible in the chrome://webrtc-internals/ developer page when a page uses WebRTC.
• There was another PSA on September 6, 2022 about other stats news with a reminder of the imminent stats deprecation.
• The October 19, 2022 PSA announced "track" stats being removed at 50% Canary.
• The follow-up October 27, 2022 PSA announced it would also be removed at 50% Beta (where M109 Beta was released on December 1st). This PSA also clarifies that "The goal is to continue ramping it up on Stable when M109 is released".
• Lastly we have yesterday's PSA announcing that the removal was advanced to 1% Stable which this conversation is a response to.

On Mon, Jan 9, 2023 at 9:42 PM Alex Russell <sligh...@chromium.org> wrote:

Thanks for adding blink-dev, Philipp. CC-ing the API OWNERs as this seems related to a pattern of breaking changes without Blink intents that we've been informed of by customers.

Do I understand correctly that this deprecation is being managed via Finch for 109 Stable?

Yes, as to minimize risk of breakage the deprecation is managed via a Finch flag, which is currently 1% Stable + 50% Canary/Beta.

To my knowledge, no issues have been reported since the rollout started in Canary in November, 2022 or "DEPRECATED_" prefix was added in July, 2022.

This, combined with the fact that apps usually gate on type, is why I thought it would be safe to gently roll out further to Stable.

 

Best,

Alex

On Tue, Jan 10, 2023 at 12:34 AM 'Aaron Boushley' via blink-dev <blin...@chromium.org> wrote:

Can you help me understand exactly which objects are being removed here? We rely on `RTCPeerConnection.getStats()` although we pass in a stream selector. We then iterate over the returned stats reports looking for ones containing the values we need.

The selector (be it an RTCRtpSender, RTCRtpReceiver or MediaStreamTrack) continues to work, it's just that the report no longer contains the removed stats objects.

 

Is this a removal of the stats objects that have the fixed ID of "track" and "stream"?

It is the removal of the stats objects where .type == "track" or .type == "stream".

In the spec this refers to dictionaries RTCMediaStreamTrackStats and RTCMediaStreamStats which are part of the "Obsolete" section of the spec. See RTCStatsType for complete list of stats object types.

Regarding the track stats dictionary, the same metrics are still available, but you have to look at the non-deprecated locations: RTCOutboundRtpStreamStats and RTCInboundRtpStreamStats dictionaries instead (type == "outbound-rtp" and type == "inbound-rtp"). See also the type "media-source" referenced from outbound-rtp.mediaSourceId.

Is there any more documentation I can look at beyond the 2 sentences above?

The spec as well as the implementation are good resources in general.

 

Aaron

[Philipp Hancke]

Am Di., 10. Jan. 2023 um 14:55 Uhr schrieb Henrik Boström <hb...@google.com>:

On Tue, Jan 10, 2023 at 8:51 AM Yoav Weiss <yoav...@chromium.org> wrote:

+Henrik Boström - was there an intent sent for this removal? Any form of developer communication?

There was developer communication dating as far back as July but I admit I had forgotten to send out a formal blink-dev intent to deprecate!

- I should have done that.

The getStats() API in question is not being deprecated, but the RTCStatsReport (an id-to-stats-object map) report will stop containing the stats object which were made obsolete in the spec several years ago due to the contents of these stats objects having been moved to other stats objects that are still being returned. Same values, different location. In other words, the report is being trimmed down by removing duplicate information. Stats processing code in an application is gated on stats type for knowing which metrics to look for on an individual stats object which should make this lower risk compared to other depracations. The motivation for this is performance optimizations (~40% report size reduction), technical debt reduction (-1400 LOC) and web compat ("track" does not exist in Firefox).

This apparently breaks users of Twilio's Video SDK:

https://github.com/twilio/twilio-video.js/issues/1968#issuecomment-1404316610

WRT web-compat thanks for pointing out to Safari that they missed adding at least trackIdentifier to inbound-rtp.

The communication channel used was WebRTC's official google group, discuss-webrtc. History:

Given the moderation status of discuss-webrtc that qualifies as a disused lavatory with a sign on the door saying ‘Beware of the Leopard.” :-)

I am in favor of using that list as much as possible in terms of process since it is still read by some "WebRTC developers" but I do not think this is a substitute for talking to developers using WebRTC.

• July 25, 2022 PSA announced the plan to deprecate at a milestone TBD. This was also the time where the "DEPRECATED_" prefix was added to the JavaScript-exposed stats object IDs, which made it into M106. The deprecation prefix is also visible in the chrome://webrtc-internals/ developer page when a page uses WebRTC.
• There was another PSA on September 6, 2022 about other stats news with a reminder of the imminent stats deprecation.
• The October 19, 2022 PSA announced "track" stats being removed at 50% Canary.
• The follow-up October 27, 2022 PSA announced it would also be removed at 50% Beta (where M109 Beta was released on December 1st). This PSA also clarifies that "The goal is to continue ramping it up on Stable when M109 is released".

Oh I missed that that the goal was stated there too -- otherwise we would have the "it is a bad idea to do this during a time of code freezes" discussion back then.

• Lastly we have yesterday's PSA announcing that the removal was advanced to 1% Stable which this conversation is a response to.

On Mon, Jan 9, 2023 at 9:42 PM Alex Russell <sligh...@chromium.org> wrote:

Thanks for adding blink-dev, Philipp. CC-ing the API OWNERs as this seems related to a pattern of breaking changes without Blink intents that we've been informed of by customers.

Do I understand correctly that this deprecation is being managed via Finch for 109 Stable?

Yes, as to minimize risk of breakage the deprecation is managed via a Finch flag, which is currently 1% Stable + 50% Canary/Beta.

To my knowledge, no issues have been reported since the rollout started in Canary in November, 2022 or "DEPRECATED_" prefix was added in July, 2022.

This, combined with the fact that apps usually gate on type, is why I thought it would be safe to gently roll out further to Stable.

Rolling out the removal via finch as opt-in has quite nasty implications for CI testing that might(*) be in place, no?

Selenium et al typically starts browsers from fresh profiles and hence does not know the finch trial seed? (**)

I haven't seen documentation showing how CI should pre-seed the profiles used with the finch trial seed (and for deprecations it might be better to recommended a command line for the CI).

I'm still in favor of the removal but removing by default (possibly guarded by a killswitch) seems like a less error-prone process.

Philipp

(*) I am somewhat pessimistic about the actual state of the "webrtc industry" wrt to automated tests

(**) at least that was my conclusion from a post-mortem of the VP8 temporal layer reduction that is, in what must be the longest "trial" ever, still ongoing?

[Sudheer Boynapally]

Hi,

Is the deprecation of 'track' and 'stream' objects completed 100% on all the versions of chrome v109 or any specific sub version of 109? 

Thanks,

[Sudheer Boynapally]

Was there any change on January 23rd corresponding to this? like rolling out this deprecation?

Thanks,

[Henrik Boström]

The track and stream removal experiment is at 50% Stable for M109. On January 23rd it went from 10% to 50%. The intent is to ramp up to 100%.

Which experiment group you end up with (have or not have the deprecated stats) are chosen with a dice roll every time the user restarts their browser.

In M111 (which is currently Canary) the removal is enabled-by-default so in that version there is no dependency on getting finch configs pushed anymore.

[Alex Russell]

It appears this change is causing breakage at scale, including for our partners.

Given how this is going, I'd like to suggest the following course of action:

• Turn down the Finch rollout to 0% immediately
• Revert the change in 111
• File an intent-to-deprecate here ASAP w/ whatever data on compat we can gather (as it looks like usecounters won't work) so we can discuss a new plan to land this

We have options that might be more suitable (e.g., reverse OT), or maybe the change just needs devrel in wider channels + time to breathe. Either way, backing off of this rollout seems like the right thing to do. In future, web-visible WebRTC changes also need to come through the main Blink intents process. Let's start down that path now.

Curious for other API OWNERs feedback, so cc-ing blink-api-owners-discuss@.

Best,

Alex

[Sudheer Boynapally]

Is the experiment rolled back on January 25th ? or it is still at 50% stable for M 109?

--

Thanks,
Sudheer

[Henrik Boström]

I'll roll it back and file an intent to deprecate thread. It was a big mistake that I forgot to do that in the first place, and it's unfortunate that we have to roll this back. But I take having reached 50% Stable and only knowing for sure about Twilio as a positive sign (if this was very widespread we would have heard about it earlier than we did). Still it seems like Finch is a problem for Selenium test environments in that it's possible to run test code for several months without noticing a feature is on. That's a bit sad.

[Anna Vasilko]

Hi Henrik,

Just wanted to double check that the latest plan is to revert the experiment, correct? Asking because there was somewhat different comment on the PR here. 

At Twilio we really hope the experiment can be reverted asap given that it is breaking for our customers and their end users in the wild. The adoption of our new sdk release (with the above PR change) will take long time for most of the customers as they need to 1) learn about the issue 2) pick-up the new sdk version 3) roll out their applications with the new sdk version. For some customers it takes months. 

Basically the only way to really stop the breaking impact is to stop the experiment and give some notice and time to adopt the changes.

I have to acknowledge it took surprisingly long time for us to discover this breaking change and act on it. Holidays probably played a role here. But we will do a retrospective internally to understand the reasons better and to catch such issues sooner in the future. 

[Henrik Boström]

Hi Anna, yes the experiment is being rolled back to 0% Stable (it's still 50% on Canary/Beta though).

The rollback has already been submitted, but unfortunately we must wait for the next config push. I'm told this happens on Monday, 9AM Pacific Time. Users will need to restart their browser to get the config.

In preparation for the next rollout, I've filed the much needed intent to deprecate here: https://groups.google.com/a/chromium.org/g/blink-dev/c/NZVXsJQ7tV8. Please provide input on the proposed plans.

[Sudheer Boynapally]

Hi Henrik,

Just want to confirm if this is rolled-back today at 9am PST ?

--

Thanks,
Sudheer

[Henrik Boström]

Yes, the experiment has been rolled back on Stable. Note that users need to restart their browser to get the new config.

[Alex Russell]

Thanks so much for driving the rollback and Intent to Deprecate, Henrik.