Re: Enforce a limit on the number of WebMediaPlayers per frame.

[Dale Curtis]

bcc: issue, cc:media-dev

Hi Riley, are you actually trying to show 70+ videos on the screen at the same time? If not you'll want to use preload=none or clear the src on the unused ones.

If you are, please let us know more about your use case.

- dale

On Wed, Jul 21, 2021 at 1:07 PM Riley Dutton <ri...@dutton.us> wrote:

Hey all,

I'm one of the Developers on Roll20 (https://roll20.net), and online VTT that makes heavy usage of multi-media components. We use WebRTC for audio and video chat, audio tags for our Jukebox system, and video tags for things like animated tokens.

This update has basically broken our software. Has any consideration been given to presenting this as an option that can be toggled (i.e. once the limit is reached, an alert pops up with a box where you can opt-in to allowing more tags on the page?) I've also seen conversations starting to happen for things like Jitsi Meet where essentially this is now putting a cap on the number of folks that can be in a call. Seems like something that the user should be able to control.

Best,

Riley

On Wednesday, June 9, 2021 at 3:41:36 PM UTC-5 John Warburton (Gerrit) wrote:

Attention is currently required from: Chris Hamilton.

Thank you very much for this suggestion.

It sounds entirely practical and I'll try it tomorrow.

With best wishes,

JW

View Change

To view, visit change 2816118. To unsubscribe, or for help writing mail filters, visit settings.

Gerrit-Project: chromium/src

Gerrit-Branch: main

Gerrit-Change-Id: I0cb30c9a9ef95981fb4d3e8563d6f5cbfdb30f42

Gerrit-Change-Number: 2816118

Gerrit-PatchSet: 9

Gerrit-Owner: Chris Hamilton <chr...@chromium.org>

Gerrit-Reviewer: Chris Hamilton <chr...@chromium.org>

Gerrit-Reviewer: Chromium LUCI CQ <chromiu...@luci-project-accounts.iam.gserviceaccount.com>

Gerrit-Reviewer: Dale Curtis <dalec...@chromium.org>

Gerrit-Reviewer: Kentaro Hara <har...@chromium.org>

Gerrit-Reviewer: Nasko Oskov <na...@chromium.org>

Gerrit-CC: John Warburton <jo...@johnwarburton.net>

Gerrit-CC: Tricium <triciu...@appspot.gserviceaccount.com>

Gerrit-CC: chromium...@chromium.org

Gerrit-CC: srirama chandra sekhar <srir...@samsung.com>

Gerrit-Attention: Chris Hamilton <chr...@chromium.org>

Gerrit-Comment-Date: Wed, 09 Jun 2021 20:41:26 +0000

Gerrit-HasComments: No

Gerrit-Has-Labels: No

Gerrit-MessageType: comment

[Dale Curtis]

Merging threads:

I'd like to second Riley's concerns here in that this should be toggle-able / disable-able setting as I also have several applications that create dynamic audio players via the use of "new Audio();" constructor in JavaScript. Some of my users have reported issues using my apps and I have had to instruct them to revert to a previous version of Chrome to overcome this restriction.

Unless you're actually trying to play 70+ tags at once, this is likely a bug in your application. You can clear the src of unused tags or set preload=none to avoid them from loading until activated. 

- dale

[Joseph Anderson]

Dale:

I'll take a closer look at my code tonight. This is what I am doing and my apps will work fine again but only if I comment out the last line of code for "SoundArray" push. My app keeps track of all past sounds that were instantiated dynamically, regardless if they were actively playing or not. Then, I have a master "StopAll" function that would loop through the array and forcibly stop any sound left playing.

var sound = new Audio(filename);

sound.onpause = function(){
    sound = null;
};

sound.onended = function(){
    sound = null;
};

sound.play();

// will have to find alternative way to handle this

window.SoundArray.push(sound);

Thoughts?

Regards.

[Dale Curtis]

You should add sound.src = "" prior to calling sound = null. Otherwise the sound object will stick around at least until playback finishes and then until garbage collection collects it. The SoundArray.push is still fine.

- dale

[Riley Dutton]

So the  main initial problem for us is we were pre-loading around 80 <audio> tags of various SFX that we play while using the app. Even though they were in the kSuspended state they were counting against the limit. We've re-worked that and are switching to a pool-based setup there and I think that should solve the primary blocker for us in the short-term.

That said, here would be the current state for us; of the "slots" we have to work with:

- 10 or so reserved for WebRTC video chat elements

- 5-10 reserved for playing SFX audio

- 10 reserved for our Jukebox feature (people playing multiple sound files at once to create sound-scapes for the game session)

That leaves around 40 slots for video tags to use with our animated tokens feature. We use hidden video tags to load video content such as WebM and then blit that into a canvas element. Most of our users aren't going to have that many animated graphics on a single page, but some might.

I guess the bottom-line on this for me, as a developer and a long-time Chrome user, is that while I realize 99% of sites weren't using media elements to this degree, there are legitimate use cases out there, and adding this sort of hard cut-off to a major web feature, with no way to override it on a site-by-site basis for the end user, seems like a step in the wrong direction. Of course that's just my two cents, so take it for what it's worth :-)

Best,

Riley

[Chris Hamilton]

The main intent of this limit is to prevent poorly written sites from accumulating too much memory, as each WebMediaPlayer brings to life a bunch of "heavy" objects and interfaces across both the renderer and the browser. Since much of the memory is actually accumulated in the browser process, this is an indirect way where a poorly written website can take down the whole browser and all other tabs. There are poorly written "infinite scrolling" video sites that accumulate an effectively unlimited number of players (we have out-of-memory reports where there are many 1000s of media players in memory).

That being said, I don't want us to break legitimate use cases like these. A simple solution would be to just lift the limit significantly higher. This would still put an effective cap on infinitely scrolling sites.

Cheers,

Chris

[Dale Curtis]

Raising the cap is fine with me if we find a truly broken case, but despite the unfortunate short term pain it's healthy that the current limit is catching large preload cases like this. I appreciate it's painful to discover this out of the blue on the stable channel though, so if we make further restrictions like this we'll send out some guidance to blink-dev@.

Guiding authors to a more dynamic creation and destruction mechanism will provide a better user experience. Loading this many tags at once can hit the network cap of 6 open streams to a given origin, so these end up being serialized or stuck for reasons unbeknownst to the page author. Often an audio tag isn't necessarily the right mechanism for this use case either. Generally folks switch to WebAudio for large sound boards like this. E.g., using a single AudioContext w/ each sound effect loaded via decodeAudioFileData().

- dale

[Riley Dutton]

As I said in the first part of my message, we are fine with switching the audio pre-loads to a different system. I think it's also likely that we will be switching to something like Howler which I believe does use WebAudio to help alleviate the SFX issue. However, the WebRTC + using video tags to show animated sprites on a canvas is, I think, a broken use case that the new hard limit will simply make impossible going forward (open to ideas on how to work around this though if there is one).

If raising the cap is an option, and the goal is to prevent crashes from websites with 1000s of video files all playing at once in an infinitely scrolling page, then my suggestion would be to increase the cap to something like 500/300...there's a pretty big gap between the current cap of 75 and the intended prevention of 1000s. At least for our use case I think that would be more than sufficient.

Appreciate you all taking the time to address this for us.

Best,

Riley

[Chris Hamilton]

On Thu, Jul 22, 2021 at 2:27 PM Riley Dutton <ri...@dutton.us> wrote:

As I said in the first part of my message, we are fine with switching the audio pre-loads to a different system. I think it's also likely that we will be switching to something like Howler which I believe does use WebAudio to help alleviate the SFX issue. However, the WebRTC + using video tags to show animated sprites on a canvas is, I think, a broken use case that the new hard limit will simply make impossible going forward (open to ideas on how to work around this though if there is one).

If raising the cap is an option, and the goal is to prevent crashes from websites with 1000s of video files all playing at once in an infinitely scrolling page, then my suggestion would be to increase the cap to something like 500/300...there's a pretty big gap between the current cap of 75 and the intended prevention of 1000s. At least for our use case I think that would be more than sufficient.

The current values are taken as the 99.9th%ile from the per platform distributions. There's precedent from other "interventions" like this that it's roughly the sweet spot where things to one side are legitimate usage and things to the other are abusive. That being said, it's not a perfect measure by any means and it could be that we want to move that line a little further out.

 

Appreciate you all taking the time to address this for us.

Best,

Riley

On Thu, Jul 22, 2021 at 1:03 PM Dale Curtis <dalec...@chromium.org> wrote:

Raising the cap is fine with me if we find a truly broken case, but despite the unfortunate short term pain it's healthy that the current limit is catching large preload cases like this. I appreciate it's painful to discover this out of the blue on the stable channel though, so if we make further restrictions like this we'll send out some guidance to blink-dev@.

Guiding authors to a more dynamic creation and destruction mechanism will provide a better user experience. Loading this many tags at once can hit the network cap of 6 open streams to a given origin, so these end up being serialized or stuck for reasons unbeknownst to the page author. Often an audio tag isn't necessarily the right mechanism for this use case either. Generally folks switch to WebAudio for large sound boards like this. E.g., using a single AudioContext w/ each sound effect loaded via decodeAudioFileData().

Overall I'm agreed with it being a good thing that it's forcing authors to think about active memory management.

[Alex Russell]

Hey all,

I've been fielding out-of-band escalations related to this intervention, and I wasn't able to spot a blink-dev thread or an API OWNERS LGTM for this developer-exposed behavior change. Was one sent that I wasn't able to find?

Regards

[Chris Hamilton]

I think the blame lands squarely on me, as a naive perf dev not really intimate with Blink process. So this didn't happen. We're immediately rolling back the M92 change (which will require a stable respin, unfortunately). Further changes to lower the limit will go through the normal "intent to intervene" process, happen much more slowly, and will also have a Finch pressure-relief valve. I've also reached out to web.dev folks about a best practices article.

See https://chromium-review.googlesource.com/c/chromium/src/+/3057111

Cheers,

Chris

[Alex Russell]

Thanks, and cheers for the quick reply.