Intent to implement and ship: HTMLMediaElement.allowedToPlay

[Chris Pearce]

Summary: HTMLMediaElement.allowedToPlay allows web authors to determine in advance of calling HTMLMediaElement.play() whether the HTMLMediaElement in its current state would be allowed to play, or would be blocked by the browser's autoplay blocking policies.

This is useful to web authors as if they can't autoplay they may prefer to download a poster image instead of paying the price of downloading media data.

This feature is particularly useful for Firefox, as web authors can poll HTMLMediaElement.allowedToPlay to determine whether a permission prompt would show if they were to call play().

Bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1478208

Spec: being developed here https://github.com/whatwg/html/issues/3617

Platform coverage: All

Target release: The same release as block autoplay, so 63, or soon after.

Preference: None

Support in other engines: There is consensus in the WhatWG github issue, but no explicit commitments from other implementors yet. We decided we should push ahead with this for Firefox because of our strategy of prompting the user for permission makes this feature particularly useful for us, as this feature provides a way for sites to avoid the prompt if desired.

Secure Context: No. HTMLMediaElement is already shipping elsewhere outside a secure context.

[Jan-Ivar Bruaroey]

On 7/29/18 10:39 PM, Chris Pearce wrote:
> Summary: HTMLMediaElement.allowedToPlay allows web authors to determine in advance of calling HTMLMediaElement.play() whether the HTMLMediaElement in its current state would be allowed to play, or would be blocked by the browser's autoplay blocking policies.
>
> This is useful to web authors as if they can't autoplay they may prefer to download a poster image instead of paying the price of downloading media data.
>
> This feature is particularly useful for Firefox, as web authors can poll HTMLMediaElement.allowedToPlay to determine whether a permission prompt would show if they were to call play().

Doesn't this amputate the user flow we just implemented?

Without this attribute, Netflix queues up rich media, Firefox asks the
user if they want autoplay, who answers "Duh, it's Netflix", and levels
up to auto-playing Netflix forever.

With this attribute, Netflix sees HTMLMediaElement.allowedToPlay ==
false, and downloads a poster image instead. User must click to get
engaging media, which now takes longer to load, and they never level up
to auto-playing Netflix?

Doesn't seem like an improvement. Am I missing something?

.: Jan-Ivar :.

[Adam Gashlin]

I don't think that Netflix would accept allowedToPlay == false for media
which is the whole point of visiting a page. They probably wouldn't even
check it then, instead allowing for the possibility that the user will be
prompted or that it just won't autoplay if they expressed that preference.
This feature doesn't help in such cases, but I don't see how it hurts. I
guess there's a possibility for overzealous postering.

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>

[Chris Pearce]

Without this attribute, sites that are determined to avoid showing the doorhanger have no option other than to never autoplay ever in Firefox. With this attribute, they can detect when they can "safely" start to autoplay video if the user otherwise unblocks autoplay.

To continue your example of Netflix, if the user clicked somewhere, say to select their Netflix profile, or click to login or a browse a category or side-scroll their catalog, then they'd gesture activate the document and the site would be able to autoplay, without a doorhanger.

I think you're basically correct that sites that actively use allowedToPlay to avoid the doorhanger would also avoid being whitelisted.

I think the only thing that you're missing is how vehemently some sites are in their desire to avoid the doorhanger prompt.

cpearce.

[Jan-Ivar Bruaroey]

On 8/1/18 3:36 AM, Chris Pearce wrote:
> On Tuesday, July 31, 2018 at 9:05:03 AM UTC+12, Jan-Ivar Bruaroey wrote:
>> On 7/29/18 10:39 PM, Chris Pearce wrote:
>>> Summary: HTMLMediaElement.allowedToPlay allows web authors to determine in advance of calling HTMLMediaElement.play() whether the HTMLMediaElement in its current state would be allowed to play, or would be blocked by the browser's autoplay blocking policies.
>>>
>>> This is useful to web authors as if they can't autoplay they may prefer to download a poster image instead of paying the price of downloading media data.
>>>
>>> This feature is particularly useful for Firefox, as web authors can poll HTMLMediaElement.allowedToPlay to determine whether a permission prompt would show if they were to call play().
>>
>> Doesn't this amputate the user flow we just implemented?
>>
>> Without this attribute, Netflix queues up rich media, Firefox asks the
>> user if they want autoplay, who answers "Duh, it's Netflix", and levels
>> up to auto-playing Netflix forever.
>>
>> With this attribute, Netflix sees HTMLMediaElement.allowedToPlay ==
>> false, and downloads a poster image instead. User must click to get
>> engaging media, which now takes longer to load, and they never level up
>> to auto-playing Netflix?
>>
>> Doesn't seem like an improvement. Am I missing something?
>
> Without this attribute, sites that are determined to avoid showing the doorhanger have no option other than to never autoplay ever in Firefox. With this attribute, they can detect when they can "safely" start to autoplay video if the user otherwise unblocks autoplay.

My first reaction to this was:

How can the user unblock autoplay on sites determined to never show the
doorhanger?

But then I realized we're mixing terms.

With "user otherwise unblocks autoplay" you mean "user gesture".
What I call "unblocking autoplay" you call "whitelisting".

It sounds like allowedToPlay would be used to entirely sidestep the
autoplay permission model we just implemented, rendering Firefox
identical to Chrome in behavior.

Except not identical, but much poorer, since Chrome would whitelist
Netflix, whereas our plan for that was to ask users once, with a
"remember=yes" prompt. If sites sidestep this prompt, then what's our
whitelist story?

Don't we need a functional plan to let users whitelist Netflix? Doesn't
this attribute work against that?

> To continue your example of Netflix, if the user clicked somewhere, say to select their Netflix profile, or click to login or a browse a category or side-scroll their catalog, then they'd gesture activate the document and the site would be able to autoplay, without a doorhanger.
>
> I think you're basically correct that sites that actively use allowedToPlay to avoid the doorhanger would also avoid being whitelisted.

Netflix doesn't need a new API to circumvent our doorhanger and blast
audio on the first user gesture. That seems easily shimmable:

1. Test Firefox version
2. If new, avoid autoplay on pageload
2. various.forEach((t, n) => t.addEventListener(n, () => audio.play());

What's the benefit to users from standardizing an API to circumvent our UX?

> I think the only thing that you're missing is how vehemently some sites are in their desire to avoid the doorhanger prompt.

No, I'm also missing why we should listen to them.

If Netflix fights our doorhanger, then they're fighting our best attempt
to whitelist them.

Regardless, something being contentious is usually a bad sign for
standardization.

> cpearce.

.: Jan-Ivar :.

[Boris Zbarsky]

On 8/6/18 4:03 PM, Jan-Ivar Bruaroey wrote:
> whereas our plan for that was to ask users once, with a
> "remember=yes" prompt.

For what it's worth, I've been getting this prompt a _lot_ recently;
presumably I finally updated to a nightly that does the prompt.

As a user, I have been tending to uncheck the "remember" checkbox,
because I don't understand the implications of having the answer
remembered and how it would affect the site's behavior in the future...

-Boris

[Jan-Ivar Bruaroey]

On 8/6/18 4:03 PM, Jan-Ivar Bruaroey wrote:

To clarify, I care about Netflix, which is why I question giving up on
persisting autoplay for them, which is what allowedToPlay does.

AFAICT allowedToPlay's sole purpose is to avoid our doorhanger.

I've heard two reasons to fear our doorhanger:

1. Sites doesn't want to get blocked.

This seems bogus, because "getting blocked" appears no different from
avoiding the prompt with allowedToPlay. Both prevent the prompt.

Try it yourself: https://jsfiddle.net/jib1/rwkLaofx/show
Press "Don't Allow", then click anywhere on the page to play.

In other words, users aren't blocking audio, only un-gestured audio,
which shouldn't matter to sites who already avoid it with allowedToPlay.

Also, allowedToPlay will never suddenly return true if used to suppress
the prompt, because we have no other whitelisting strategy.

2. User testing shows many users don't understand the prompt.

This one makes sense to me. If avoiding our one-time prompt matters more
to them than autoplay itself, it's a sign our prompt isn't great. We
should fix that, not help sites opt out.


Here's my take on our prompt:

I love that we went with a visible user agent feature instead of
breaking the web. Even the permission part, partly. The management part.

But the prompt itself is too complicated. It's hard to glean how little
is at stake: delaying audio by a mere click in many cases.

Try the fiddle again, ignore the prompt and just click somewhere on the
page. Tone.

I think we need to rephrase this as a helpful user agent:

_/\_________________
| |
| Wanna hear sound ? |
| |
| No | Yes |
`--------------------'
or
_/\_______________
| |
| Sound blocked. |
| |
| Don't | Thanks |
`------------------'

Then have the user agent set the permission wisely.

By putting the agent in charge, we might even get away with a path to
whitelisting without a prompt, where the user agent implicitly turns the
permission on for well-behaved sites after users have interacted
sufficiently with gesture-driven audio, without signs of distress.

This seems similar to Google's engagement system, except it's visible,
so users can override the agent should it get it wrong, since it's still
permission-based at heart.

Then allowedToPlay might make sense.

Thoughts?

.: Jan-Ivar :.

>> cpearce.
>
> .: Jan-Ivar :.

[Chris Pearce]

Hi Jib,

I appreciate that you care passionately about our users' best interests.

Seeing as you asked "why don't you just?"... Here's why we "didn't just"...

On Tuesday, August 7, 2018 at 7:51:50 PM UTC-7, Jan-Ivar Bruaroey wrote:
> On 8/6/18 4:03 PM, Jan-Ivar Bruaroey wrote:
> > On 8/1/18 3:36 AM, Chris Pearce wrote:
> >> I think the only thing that you're missing is how vehemently some
> >> sites are in their desire to avoid the doorhanger prompt.
> >
> > No, I'm also missing why we should listen to them.
> >
> > If Netflix fights our doorhanger, then they're fighting our best attempt
> > to whitelist them.
>
> To clarify, I care about Netflix, which is why I question giving up on
> persisting autoplay for them, which is what allowedToPlay does.
>
> AFAICT allowedToPlay's sole purpose is to avoid our doorhanger.

Sites are already trying to feature detect whether they can autoplay audible video and present different behaviour when they can't. There are established solutions for this that pre-date us talking about doing a doorhanger. So I don't think we can conclude that authors are doing this to avoid our doorhanger.

Many of the major US news sites as well as the BBC now try to autoplay audibly, and if that is blocked, they fallback to playing inaudiblby and showing an unmute button.

I was also pointed at https://www.npmjs.com/package/modernizr which which is a web framework which is feature sniffing whether audible autoplay works by trying to play a media with a silent audio track. It does this on load, even if the page author hasn't signaled any intention to use video. NPMJS lists that as getting 41k weekly downloads, and this framework would cause sites that are not obviously using media to have a prompt appear in Firefox for maybe no benefit. I want to avoid that.

Many of these packages which detect whether audible autoplay works have timeouts so that if the play() promise is not resolved within some short time they assume that they can't autoplay, which doesn't play nice with us delaying resolving the play() promise while we wait for the user to click the prompt.

Sites were trying to detect whether they can audibly autoplay before Firefox had a prompt, so making it easier/cheaper/faster for them whilst also having a side benefit of avoiding the prompt seems reasonable.

> I've heard two reasons to fear our doorhanger:
>
> 1. Sites doesn't want to get blocked.
>
> This seems bogus, because "getting blocked" appears no different from
> avoiding the prompt with allowedToPlay. Both prevent the prompt.
>
> Try it yourself: https://jsfiddle.net/jib1/rwkLaofx/show
> Press "Don't Allow", then click anywhere on the page to play.
>
> In other words, users aren't blocking audio, only un-gestured audio,
> which shouldn't matter to sites who already avoid it with allowedToPlay.

It's not clear to me that "Sites doesn't want to get blocked" is in our users' interest, so I'm not going to contest this.

> Also, allowedToPlay will never suddenly return true if used to suppress
> the prompt, because we have no other whitelisting strategy.

allowedToPlay will start returning true when the page is gesture activated.

For example, when I open into Netflix, I am greeted with a profile selection screen from which I can choose between my profile, my wife's profile, or my kids' profile. The click to select a profile gesture activates the tab for me, so I normally don't see the prompt on Netflix FWIW.

>
> 2. User testing shows many users don't understand the prompt.
>
> This one makes sense to me. If avoiding our one-time prompt matters more
> to them than autoplay itself, it's a sign our prompt isn't great. We
> should fix that, not help sites opt out.

The prompt exists because there are some cases where users have a reasonable expectation that autoplay audible audio should work without interacting with the page first. For example deep links into YouTube/Netflix videos, notification sounds for messaging sites, and the long tail of things that Chrome broke in https://bugs.chromium.org/p/chromium/issues/detail?id=840866 (some of which are WebAudio, which we're not blocking yet).

I wanted to solve this by shipping our own custom whitelist (like Chrome and Safari), but we couldn't figure out how to objectively generate one without phoning home with our users' browsing history, and that's against our data collection policies.

>
>
> Here's my take on our prompt:
>
> I love that we went with a visible user agent feature instead of
> breaking the web. Even the permission part, partly. The management part.
>
> But the prompt itself is too complicated. It's hard to glean how little
> is at stake: delaying audio by a mere click in many cases.
>
> Try the fiddle again, ignore the prompt and just click somewhere on the
> page. Tone.
>

We are considering switching from the document gesture activation strategy to only allowing play() calls inside a JS context that originated inside a user generated event handler (like how Safari blocks autoplay). But currently that is blocked on a bunch of work to make EventStateManager::IsHandlingUserInput() return true for JS contexts running in a setTimeout or promise continuation that originated in a user generated event handler. That would eleviate some of your "WTF?" response here I think.

> I think we need to rephrase this as a helpful user agent:
>
> _/\_________________
> | |
> | Wanna hear sound ? |
> | |
> | No | Yes |
> `--------------------'
> or
> _/\_______________
> | |
> | Sound blocked. |
> | |
> | Don't | Thanks |
> `------------------'
>
> Then have the user agent set the permission wisely.

To be clear, you're only unhappy with the wording of the prompt? Everyone has a diffent opinion on what the best wording is FWIW...

> By putting the agent in charge, we might even get away with a path to
> whitelisting without a prompt, where the user agent implicitly turns the
> permission on for well-behaved sites after users have interacted
> sufficiently with gesture-driven audio, without signs of distress.

I had already considered approaches like this. How would you unblock sound on sites like slack/irccloud, or the long tail of sites on the Chroium bug above, that don't have play buttons for their audio playback?

> This seems similar to Google's engagement system, except it's visible,
> so users can override the agent should it get it wrong, since it's still
> permission-based at heart.
>
> Then allowedToPlay might make sense.

It's not clear how this is effectively different from Chrome's MEI, which was poorly received.

FWIW, I see a lot of feedback on my twitter stream of people who love that Firefox puts them in control of whether sites autoplay. Whether the same sentiment will also be present in our release population remains to be seen.


cpearce.

[Boris Zbarsky]

On 8/8/18 12:43 PM, Chris Pearce wrote:
> On Tuesday, August 7, 2018 at 7:51:50 PM UTC-7, Jan-Ivar Bruaroey wrote:
>> To clarify, I care about Netflix, which is why I question giving up on
>> persisting autoplay for them, which is what allowedToPlay does.

So I have a question. Would it be at all useful to have an explicit way
for a site to ask for a persistent permission grant (as opposed to the
implicit "just try to play something")? Would netflix use that? Or
would they avoid it for the same reasons as their current avoidance of
the prompt? Assuming sites would use it, would we be able to sell other
browsers on such an API?

There's a hard problem here at heart where the APIs are being designed
around the UX, and the different browsers have very different UX goals,
afaict.

-Boris

[Chris Pearce]

I think that's a good idea.

I had proposed that autoplay-media permissions be exposed as part of the Permisisons API here https://github.com/w3c/permissions/issues/172 which I think would mean that they could be explicitly requested? Google weren't keen on that idea as it doesn't map well to their implicit MEI method.

[Boris Zbarsky]

On 8/8/18 1:00 PM, Chris Pearce wrote:
> Google weren't keen on that idea as it doesn't map well to their implicit MEI method.

Right, that's the "hard problem" bit.

Feels like we should maybe be able to find non-Google allies here,
especially if we have some weasel-wording in the spec about how the UA
can ignore the grant request or something...

-Boris

[Jan-Ivar Bruaroey]

On 8/8/18 12:53 PM, Boris Zbarsky wrote:
> On 8/8/18 12:43 PM, Chris Pearce wrote:
>> On Tuesday, August 7, 2018 at 7:51:50 PM UTC-7, Jan-Ivar Bruaroey wrote:
>>> To clarify, I care about Netflix, which is why I question giving up on
>>> persisting autoplay for them, which is what allowedToPlay does.
>
> So I have a question.  Would it be at all useful to have an explicit way
> for a site to ask for a persistent permission grant (as opposed to the
> implicit "just try to play something")?  Would netflix use that?

Generally, our position [1] is that standardized permission-oriented
APIs are bad for the web and Firefox. Because:

1) It would encourage permission bundling, e.g. the dreaded "Approve all
56 permissions to use this Android Flashlight app", of olden days. It is
generally agreed these days that asking as late as possible ensures the
most context for users to know what specific features they'll miss out
on if they decline. The way we do that is embed prompts in the access APIs.

2) It would mean effectively standardize one permission model for the
web (probably Chrome's). We consider this a bad thing, because
permission is still an area of differentiation and innovation between
browsers. To the extent we attempt this WITHOUT standardizing, we end up
exposing non-standard user agent behavior, and then we always lose,
because many web developers will then test against the dominant
browser's permission model only, and things end up working poorly in
Firefox, unless we mimic Chrome's model exactly.

I also think it's premature to say that mapping block autoplay to a
permission model is even a perfect fit, so I wouldn't draw any general
lessons about permission from block autoplay. That would be the tail
wagging the dog IMHO.

> Or would they avoid it for the same reasons as their current avoidance of
> the prompt?

I'm missing why this would make the prompt any more palatable to sites
that "vehemently" don't want it. I'm also not sure how we would know,
since this is effectively a negotiation. E.g. if you asked me, and I
didn't want it, I'd say "yes" and then never call it, to get the prompt
out of the way.

> There's a hard problem here at heart where the APIs are being designed
> around the UX, and the different browsers have very different UX goals,
> afaict.

Again I wouldn't generalize from this case. "The APIs" are designed
around requesting resources, not permissions. User agents are separate
from the web, and there's value in allowing diversity among user agents
for hard problems, like guiding users with different privacy needs, ad
blocking, permissions, hard problems like that which are not inherently
rooted in how web content is shaped, but more how users have different
preferences for consuming (viewing and interacting with) said content.

> -Boris
[1]
https://github.com/mozilla/standards-positions/issues/19#issuecomment-370039662

[Jan-Ivar Bruaroey]

On 8/8/18 12:43 PM, Chris Pearce wrote:

> Hi Jib,
>
> I appreciate that you care passionately about our users' best interests.
>
> Seeing as you asked "why don't you just?"... Here's why we "didn't just"...
>
> On Tuesday, August 7, 2018 at 7:51:50 PM UTC-7, Jan-Ivar Bruaroey wrote:
>> On 8/6/18 4:03 PM, Jan-Ivar Bruaroey wrote:
>>> On 8/1/18 3:36 AM, Chris Pearce wrote:
>>>> I think the only thing that you're missing is how vehemently some
>>>> sites are in their desire to avoid the doorhanger prompt.
>>>
>>> No, I'm also missing why we should listen to them.
>>>
>>> If Netflix fights our doorhanger, then they're fighting our best attempt
>>> to whitelist them.
>>
>> To clarify, I care about Netflix, which is why I question giving up on
>> persisting autoplay for them, which is what allowedToPlay does.
>>
>> AFAICT allowedToPlay's sole purpose is to avoid our doorhanger.
>
> Sites are already trying to feature detect whether they can autoplay audible video and present different behaviour when they can't. There are established solutions for this that pre-date us talking about doing a doorhanger. So I don't think we can conclude that authors are doing this to avoid our doorhanger.
>
> Many of the major US news sites as well as the BBC now try to autoplay audibly, and if that is blocked, they fallback to playing inaudiblby and showing an unmute button.

Right, because of Chrome. This is my point. allowedToPlay plays into
Chrome's autoplay policy (pardon the pun), not ours.

Chrome's "path to whitelisting" policy appears to be: Don't bother
trying unless you're whitelisted or until user engagement kicks in.

Our "path to whitelisting" policy appears to be: Try, and we'll ask the
user to whitelist you. If you don't try, we can't help you.

From a web developer's perspective, this seems incompatible.

So allowedToPlay makes sense in Chrome, not in Firefox at the moment.

Seems to me we need a better "path to whitelisting" strategy, before we
help bury the one we have.

Once we have a working "path to whitelisting" policy, that's not
thwarted by allowedToPlay or modernizr, I see no problem with
allowedToPlay, but it appears we don't.

What concerns me, is we seem intent to land without this in place...

Firefox 62: Netflix autoplays
Firefox 63: Netflix no longer autoplays
Chrome: Netflix autoplays

:-(

> I was also pointed at https://www.npmjs.com/package/modernizr which which is a web framework which is feature sniffing whether audible autoplay works by trying to play a media with a silent audio track. It does this on load, even if the page author hasn't signaled any intention to use video. NPMJS lists that as getting 41k weekly downloads, and this framework would cause sites that are not obviously using media to have a prompt appear in Firefox for maybe no benefit. I want to avoid that.
>
> Many of these packages which detect whether audible autoplay works have timeouts so that if the play() promise is not resolved within some short time they assume that they can't autoplay, which doesn't play nice with us delaying resolving the play() promise while we wait for the user to click the prompt.
>
> Sites were trying to detect whether they can audibly autoplay before Firefox had a prompt, so making it easier/cheaper/faster for them whilst also having a side benefit of avoiding the prompt seems reasonable.
>
>
>
>
>> I've heard two reasons to fear our doorhanger:
>>
>> 1. Sites doesn't want to get blocked.
>>
>> This seems bogus, because "getting blocked" appears no different from
>> avoiding the prompt with allowedToPlay. Both prevent the prompt.
>>
>> Try it yourself: https://jsfiddle.net/jib1/rwkLaofx/show
>> Press "Don't Allow", then click anywhere on the page to play.
>>
>> In other words, users aren't blocking audio, only un-gestured audio,
>> which shouldn't matter to sites who already avoid it with allowedToPlay.
>
>
> It's not clear to me that "Sites doesn't want to get blocked" is in our users' interest, so I'm not going to contest this.
>
>
>> Also, allowedToPlay will never suddenly return true if used to suppress
>> the prompt, because we have no other whitelisting strategy.
>
>
> allowedToPlay will start returning true when the page is gesture activated.

Sorry I misspoke. I meant allowedToPlay will never suddenly return true
*on pageload* if used to suppress the prompt, because we have no other
whitelisting strategy.

> For example, when I open into Netflix, I am greeted with a profile selection screen from which I can choose between my profile, my wife's profile, or my kids' profile. The click to select a profile gesture activates the tab for me, so I normally don't see the prompt on Netflix FWIW.
>
>
>>
>> 2. User testing shows many users don't understand the prompt.
>>
>> This one makes sense to me. If avoiding our one-time prompt matters more
>> to them than autoplay itself, it's a sign our prompt isn't great. We
>> should fix that, not help sites opt out.
>
>
> The prompt exists because there are some cases where users have a reasonable expectation that autoplay audible audio should work without interacting with the page first. For example deep links into YouTube/Netflix videos, notification sounds for messaging sites, and the long tail of things that Chrome broke in https://bugs.chromium.org/p/chromium/issues/detail?id=840866 (some of which are WebAudio, which we're not blocking yet).
>
> I wanted to solve this by shipping our own custom whitelist (like Chrome and Safari), but we couldn't figure out how to objectively generate one without phoning home with our users' browsing history, and that's against our data collection policies.

So how are we going to solve this?

>> Here's my take on our prompt:
>>
>> I love that we went with a visible user agent feature instead of
>> breaking the web. Even the permission part, partly. The management part.
>>
>> But the prompt itself is too complicated. It's hard to glean how little
>> is at stake: delaying audio by a mere click in many cases.
>>
>> Try the fiddle again, ignore the prompt and just click somewhere on the
>> page. Tone.
>>
>
>
> We are considering switching from the document gesture activation strategy to only allowing play() calls inside a JS context that originated inside a user generated event handler (like how Safari blocks autoplay). But currently that is blocked on a bunch of work to make EventStateManager::IsHandlingUserInput() return true for JS contexts running in a setTimeout or promise continuation that originated in a user generated event handler. That would eleviate some of your "WTF?" response here I think.

No, it doesn't change that the prompt is too complicated IMHO.

>> I think we need to rephrase this as a helpful user agent:
>>
>> _/\_________________
>> | |
>> | Wanna hear sound ? |
>> | |
>> | No | Yes |
>> `--------------------'
>> or
>> _/\_______________
>> | |
>> | Sound blocked. |
>> | |
>> | Don't | Thanks |
>> `------------------'
>>
>> Then have the user agent set the permission wisely.
>
> To be clear, you're only unhappy with the wording of the prompt? Everyone has a diffent opinion on what the best wording is FWIW...
>
>
>> By putting the agent in charge, we might even get away with a path to
>> whitelisting without a prompt, where the user agent implicitly turns the
>> permission on for well-behaved sites after users have interacted
>> sufficiently with gesture-driven audio, without signs of distress.
>
> I had already considered approaches like this. How would you unblock sound on sites like slack/irccloud, or the long tail of sites on the Chroium bug above, that don't have play buttons for their audio playback?

That's a great (hopefully not rhetorical) question, that I think is
squarely in user agent territory. We should try to solve that and even
refine it over time IMHO.

I keep slack open as a pinned tab, and I have given it notification
permission. I also interact with it a lot. Lots of signals there.

>> This seems similar to Google's engagement system, except it's visible,
>> so users can override the agent should it get it wrong, since it's still
>> permission-based at heart.
>>
>> Then allowedToPlay might make sense.
>
>
> It's not clear how this is effectively different from Chrome's MEI, which was poorly received.

It's different because users can see what mode they're in, thanks to how
we visualize permissions in the UI.

> FWIW, I see a lot of feedback on my twitter stream of people who love that Firefox puts them in control of whether sites autoplay. Whether the same sentiment will also be present in our release population remains to be seen.

I like it too. That doesn't mean the prompt isn't too complicated for
most folks. I don't know about you, but the people I follow on Twitter
tend to be quite technical. ;)

.: Jan-Ivar :.

[Chris Pearce]

Thank you for persisting on this issue. I have thought on it and I agree there is a valid concern here about sites using allowedToPlay being steered away from the path where they'd hit the doorhanger.

Our current thinking here is to have allowedToPlay returned one of {"allowed", "prompt-required", "blocked"} instead. Then sites which assumed they can play if video.allowedToPlay!="blocked" would try to play and hit our prompt.

Does that alleviate your concerns?

> > For example, when I open into Netflix, I am greeted with a profile selection screen from which I can choose between my profile, my wife's profile, or my kids' profile. The click to select a profile gesture activates the tab for me, so I normally don't see the prompt on Netflix FWIW.
> >
> >
> >>
> >> 2. User testing shows many users don't understand the prompt.
> >>
> >> This one makes sense to me. If avoiding our one-time prompt matters more
> >> to them than autoplay itself, it's a sign our prompt isn't great. We
> >> should fix that, not help sites opt out.
> >
> >
> > The prompt exists because there are some cases where users have a reasonable expectation that autoplay audible audio should work without interacting with the page first. For example deep links into YouTube/Netflix videos, notification sounds for messaging sites, and the long tail of things that Chrome broke in https://bugs.chromium.org/p/chromium/issues/detail?id=840866 (some of which are WebAudio, which we're not blocking yet).
> >
> > I wanted to solve this by shipping our own custom whitelist (like Chrome and Safari), but we couldn't figure out how to objectively generate one without phoning home with our users' browsing history, and that's against our data collection policies.
>
> So how are we going to solve this?

We've not been able to come up with a reasonable solution for creating a default whitelist, so we opted to avoid the problem altogether by giving users control over what they want to whitelist.

> >> Here's my take on our prompt:
> >>
> >> I love that we went with a visible user agent feature instead of
> >> breaking the web. Even the permission part, partly. The management part.
> >>
> >> But the prompt itself is too complicated. It's hard to glean how little
> >> is at stake: delaying audio by a mere click in many cases.
> >>
> >> Try the fiddle again, ignore the prompt and just click somewhere on the
> >> page. Tone.
> >>
> >
> >
> > We are considering switching from the document gesture activation strategy to only allowing play() calls inside a JS context that originated inside a user generated event handler (like how Safari blocks autoplay). But currently that is blocked on a bunch of work to make EventStateManager::IsHandlingUserInput() return true for JS contexts running in a setTimeout or promise continuation that originated in a user generated event handler. That would eleviate some of your "WTF?" response here I think.
>
> No, it doesn't change that the prompt is too complicated IMHO.

I don't understand why you consider the prompt complicated. Can you try saying why in another way?

>
> >> I think we need to rephrase this as a helpful user agent:
> >>
> >> _/\_________________
> >> | |
> >> | Wanna hear sound ? |
> >> | |
> >> | No | Yes |
> >> `--------------------'
> >> or
> >> _/\_______________
> >> | |
> >> | Sound blocked. |
> >> | |
> >> | Don't | Thanks |
> >> `------------------'
> >>
> >> Then have the user agent set the permission wisely.
> >
> > To be clear, you're only unhappy with the wording of the prompt? Everyone has a diffent opinion on what the best wording is FWIW...
> >
> >
> >> By putting the agent in charge, we might even get away with a path to
> >> whitelisting without a prompt, where the user agent implicitly turns the
> >> permission on for well-behaved sites after users have interacted
> >> sufficiently with gesture-driven audio, without signs of distress.
> >
> > I had already considered approaches like this. How would you unblock sound on sites like slack/irccloud, or the long tail of sites on the Chroium bug above, that don't have play buttons for their audio playback?
>
> That's a great (hopefully not rhetorical) question, that I think is
> squarely in user agent territory. We should try to solve that and even
> refine it over time IMHO.
>
> I keep slack open as a pinned tab, and I have given it notification
> permission. I also interact with it a lot. Lots of signals there.

That is not a rhetorical question. I considered an approach like this and discarded it.

Pinned tabs and push notification permissions are noisy signals.

People pin tabs for all kinds of reasons. People pin tabs to "super bookmark" things that they don't want to autoplay. We had a raft of bugs filed where people pin youtube tabs of channels they like to watch a lot, but they don't want those to autoplay.

Push notifications aren't a good signal either. Lots of sites that we don't think should be autoplaying request push notification permission, like news sites. YouTube also requests push notifcations under some conditions I don't understand, and as stated above not everyone wants YouTube to autoplay.

Push notifications and autoplay media are not the same thing, and it is likely to lead to confusion if we group their permissions together.

cpearce.