Intent to Implement and Ship: autoplay muted videos on Android

[Mounir Lamouri]

Contact emails

Eng: ava...@chromium.org, mlam...@chromium.org, zqz...@chromium.org

PM: renga...@chromium.org

 

Spec

No spec.

Some discussion happened here: https://github.com/whatwg/html/issues/976

It will likely followed by minor spec changes.

 

Summary

Platform will support autoplay of media elements without requiring a user gesture if they are muted (using the muted attribute/property). User gesture will be required to unmute the media element programmatically. If not, the media element will pause.

 

Media elements using autoplay attribute will only start when they are visible in the user viewport.

 

Platform will not support autoplay if Data Saver is enabled and / or when the user indicates to turn off autoplay using a specific setting.

 

Motivation

The web is increasingly media centric. Background videos as design elements, effective use of media as narrative pieces is very common. However, the mobile web today requires a user gesture for playback of media.

 

However, animating pixels on the Web isn’t very hard. Motivated websites have resorted to two common workarounds -

• Animated images - which are bandwidth inefficient (~10x) larger than equivalent video
  

• Decoding video in javascript - this is incredibly power inefficient
  

 

Enabling muted autoplay ensures the platform provides the right support for the intent. This also enables increasingly common usecases on the web such as background video as a design element.

 

Interoperability and Compatibility Risk

Firefox: No public signals (already ship with autoplay by default on mobile)

Edge: No public signals

Safari: Public support

Web developers: Strongly positive

 

Safari is supportive of the spec changes. It is the only other browser blocking autoplay on mobile.

 

With the recent spec change of Promise<> return for play(), we now have the full feedback mechanism to developers to indicate successful playback.

 

Ongoing technical constraints

None.

 

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

As far as Chrome is concerned, this will only ship on Android. However, the feature itself doesn’t have platform specific constraints.

 

OWP launch tracking bug

https://crbug.com/618000

 

Link to entry on the feature dashboard

https://www.chromestatus.com/features/4864052794753024

 

Requesting approval to ship?

Yes.

[Kenji Baheux]

Non owner +1,

The workarounds I've seen are extremely inefficient across at least one dimension (if not all of them):

• power
• performance
  
• battery
• bandwidth
• memory

Providing an efficient way to solve this use case is dearly needed.

[TAMURA, Kent]

This sounds a low-risk and a good feature, but I hesitate to approve this because of no spec.

--

TAMURA Kent
Software Engineer, Google

[Jochen Eisinger]

Will videos that are off screen also start playing regardless?

[Mounir Lamouri]

If a video has the autoplay attribute and is muted, we will wait for it
to be in viewport to play it. For videos that autoplay using the play
method, we will play as soon as asked to to not break developers
expectations. The reason for the latter behaviour is that we don't want
to introduce too much magic behaviour and there are valid use cases to
start a video while not visible. Our assumption is that most muted
videos are actually <video autoplay muted>. We are introducing metrics
to track this and we are open to make play() follow the autoplay
attribute if needed.

Regarding the spec, there are actually no real spec changes that are
required. The HTMLMediaElement already acknowledge the fact that a media
element playback might be rejected because of user gesture restrictions.
The restriction is exposed to the website via a rejected promise. The
spec leaves the decision regarding rejecting the play request to the UA
and doesn't specify it. What this feature will do is to no longer reject
in some situations. There is no spec changes with regards to this. In
some way, you can this this feature as an un-intervention :)

The only spec changes I think we might want to introduce is more of a
note that changing the muted state might have side effects (ie. pausing
in this case). Otherwise, I think the full plan is fine per spec.

Cheers,
-- Mounir

On Wed, 8 Jun 2016, at 06:59, Jochen Eisinger wrote:
> Will videos that are off screen also start playing regardless?
>
> TAMURA, Kent <tk...@chromium.org> schrieb am Mi., 8. Juni 2016, 07:49:
>
> > This sounds a low-risk and a good feature, but I hesitate to approve this
> > because of no spec.
> >
> >
> > On Wed, Jun 8, 2016 at 7:37 AM, Mounir Lamouri <mou...@lamouri.fr> wrote:
> >

> >> *Contact emails*
> >>
> >> Eng: *ava...@chromium.org* <ava...@chromium.org>,
> >> *mlam...@chromium.org* <mlam...@chromium.org>, *zqz...@chromium.org*
> >> <zqz...@chromium.org>
> >>
> >> PM: *renga...@chromium.org* <renga...@chromium.org>
> >>
> >>
> >> *Spec*

> >>
> >> No spec.
> >>
> >> Some discussion happened here:

> >> *https://github.com/whatwg/html/issues/976*

> >> <https://github.com/whatwg/html/issues/976>
> >>
> >> It will likely followed by minor spec changes.
> >>
> >>

> >> *Summary*

> >>
> >> Platform will support autoplay of media elements without requiring a user
> >> gesture if they are muted (using the muted attribute/property). User
> >> gesture will be required to unmute the media element programmatically. If
> >> not, the media element will pause.
> >>
> >>
> >> Media elements using autoplay attribute will only start when they are
> >> visible in the user viewport.
> >>
> >>
> >> Platform will not support autoplay if Data Saver is enabled and / or when
> >> the user indicates to turn off autoplay using a specific setting.
> >>
> >>

> >> *Motivation*

> >>
> >> The web is increasingly media centric. Background videos as design
> >> elements, effective use of media as narrative pieces is very common.
> >> However, the mobile web today requires a user gesture for playback of media.
> >>
> >>
> >> However, animating pixels on the Web isn’t very hard. Motivated websites
> >> have resorted to two common workarounds -
> >>
> >> -
> >>
> >> Animated images - which are bandwidth inefficient (~10x) larger than
> >> equivalent video

> >> -

> >>
> >> Decoding video in javascript - this is incredibly power inefficient
> >>
> >>
> >>
> >> Enabling muted autoplay ensures the platform provides the right support
> >> for the intent. This also enables increasingly common usecases on the web
> >> such as background video as a design element.
> >>
> >>

> >> *Interoperability and Compatibility Risk*

> >>
> >> Firefox: No public signals (already ship with autoplay by default on
> >> mobile)
> >>
> >> Edge: No public signals
> >>
> >> Safari: Public support
> >>
> >> Web developers: Strongly positive
> >>
> >>
> >> Safari is supportive of the spec changes. It is the only other browser
> >> blocking autoplay on mobile.
> >>
> >>
> >> With the recent spec change of Promise<> return for play(), we now have
> >> the full feedback mechanism to developers to indicate successful playback.
> >>
> >>

> >> *Ongoing technical constraints*
> >>
> >> None.
> >>
> >>
> >> *Will this feature be supported on all six Blink platforms (Windows, Mac,
> >> Linux, Chrome OS, Android, and Android WebView)?*

> >>
> >> As far as Chrome is concerned, this will only ship on Android. However,
> >> the feature itself doesn’t have platform specific constraints.
> >>
> >>

> >> *OWP launch tracking bug*
> >>
> >> *https://crbug.com/618000* <https://crbug.com/618000>
> >>
> >>
> >> *Link to entry on the **feature dashboard*
> >> <https://www.chromestatus.com/>
> >>
> >> *https://www.chromestatus.com/features/4864052794753024*
> >> <https://www.chromestatus.com/features/4864052794753024>
> >>
> >>
> >> *Requesting approval to ship?*

> >>
> >> Yes.
> >>
> >
> >
> >
> > --
> > TAMURA Kent
> > Software Engineer, Google
> >
> >
> >
>

> --
> You received this message because you are subscribed to the Google Groups
> "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+...@chromium.org.

[Dale Curtis]

What are your thoughts about video elements not in the DOM used to drive WebGL playbacks? See http://ro.me/

- dale

[TAMURA, Kent]

On Wed, Jun 8, 2016 at 8:09 PM, Mounir Lamouri <mou...@lamouri.fr> wrote:

If a video has the autoplay attribute and is muted, we will wait for it
to be in viewport to play it. For videos that autoplay using the play
method, we will play as soon as asked to to not break developers
expectations. The reason for the latter behaviour is that we don't want
to introduce too much magic behaviour and there are valid use cases to
start a video while not visible. Our assumption is that most muted
videos are actually <video autoplay muted>. We are introducing metrics
to track this and we are open to make play() follow the autoplay
attribute if needed.

Regarding the spec, there are actually no real spec changes that are
required. The HTMLMediaElement already acknowledge the fact that a media
element playback might be rejected because of user gesture restrictions.
The restriction is exposed to the website via a rejected promise. The
spec leaves the decision regarding rejecting the play request to the UA
and doesn't specify it. What this feature will do is to no longer reject
in some situations. There is no spec changes with regards to this. In
some way, you can this this feature as an un-intervention :)

The only spec changes I think we might want to introduce is more of a
note that changing the muted state might have side effects (ie. pausing
in this case). Otherwise, I think the full plan is fine per spec.

Ok, I understand.  LGTM1 to ship.

[Alexandre Elias]

On Tue, Jun 7, 2016 at 3:37 PM, Mounir Lamouri <mou...@lamouri.fr> wrote:

Contact emails

Eng: ava...@chromium.org, mlam...@chromium.org, zqz...@chromium.org

PM: renga...@chromium.org

 

Spec

No spec.

Some discussion happened here: https://github.com/whatwg/html/issues/976

It will likely followed by minor spec changes.

 

Summary

Platform will support autoplay of media elements without requiring a user gesture if they are muted (using the muted attribute/property). User gesture will be required to unmute the media element programmatically. If not, the media element will pause.

 

Media elements using autoplay attribute will only start when they are visible in the user viewport.

 

Platform will not support autoplay if Data Saver is enabled and / or when the user indicates to turn off autoplay using a specific setting.

Can this be fully decoupled from Data Saver mode?  A core promise of Data Saver from my POV is that bandwidth is saved, but otherwise absolutely nothing else changes in my browsing experience.  I believe this would be the first time that Data Saver changes/breaks something user-visible, and in a way that's quite surprising.

I think it would be better for the new specific setting you're introducing to control the autoplay behavior, full stop.

[Alexandre Elias]

On Wed, Jun 8, 2016 at 8:22 PM, Alexandre Elias <ael...@chromium.org> wrote:

>

> On Tue, Jun 7, 2016 at 3:37 PM, Mounir Lamouri <mou...@lamouri.fr> wrote:

>>

>> Contact emails
>>
>> Eng: ava...@chromium.org, mlam...@chromium.org, zqz...@chromium.org
>>
>> PM: renga...@chromium.org
>>
>>  
>>
>> Spec
>>
>> No spec.
>>
>> Some discussion happened here: https://github.com/whatwg/html/issues/976
>>
>> It will likely followed by minor spec changes.
>>
>>  
>>
>> Summary
>>
>> Platform will support autoplay of media elements without requiring a user gesture if they are muted (using the muted attribute/property). User gesture will be required to unmute the media element programmatically. If not, the media element will pause.
>>
>>  
>>
>> Media elements using autoplay attribute will only start when they are visible in the user viewport.
>>
>>  
>>
>> Platform will not support autoplay if Data Saver is enabled and / or when the user indicates to turn off autoplay using a specific setting.
>
>
> Can this be fully decoupled from Data Saver mode?  A core promise of Data Saver from my POV is that bandwidth is saved, but otherwise absolutely nothing else changes in my browsing experience.  I believe this would be the first time that Data Saver changes/breaks something user-visible, and in a way that's quite surprising.
>
> I think it would be better for the new specific setting you're introducing to control the autoplay behavior, full stop.

Alternatively, I'd suggest tying this to "Lofi mode" which is triggered when bandwidth is detected to be very low, and whose main effect today is to disable images.  That's the "OK to break the web" submode of Data Saver, with an infobar and opt-out button.
 

[Mounir Lamouri]

Dale, regarding playing videos off tree to render them with WebGL, I
would say that given the principles used to say that we should allow
autoplay muted video there is no reason why it shouldn't be allowed. I
guess if the video was set as muted, it would work. In the future, we
might not required the muted attribute if we can detect that there is no
audio track. Though it's not planned for the first step.

Alexandre, the main concern here is that we want to be cautious and not
waste users' bandwidth. It is very hard to say if this change will have
a positive or negative impact on bandwidth usage (and the impact might
be different for users and over time). We prefer to make sure it doesn't
impact users who clearly mark their intent to save bandwidth. In
addition of that, there will be a setting that will allow users to
disable the feature. I shared the same concerns regarding disabling
muted video autoplay as a side effect of Data Saver but both Chrome
Android leadership and the Data Saver team were very happy with this
solution. It's something that we can always revisit later if needed
though.

-- Mounir

On Thu, 9 Jun 2016, at 05:17, Alexandre Elias wrote:
> On Wed, Jun 8, 2016 at 8:22 PM, Alexandre Elias <ael...@chromium.org>
> wrote:
>
> >
>
> > On Tue, Jun 7, 2016 at 3:37 PM, Mounir Lamouri <mou...@lamouri.fr> wrote:
>

> *>>*
>
> *>> Contact emails*
> >>
> >> Eng:* ava...@chromium.org <ava...@chromium.org>*,*
> mlam...@chromium.org <mlam...@chromium.org>*,* zqz...@chromium.org
> <zqz...@chromium.org>*
> >>
> >> PM:* renga...@chromium.org <renga...@chromium.org>*
> >>
> >>
> *>>*
> *>> Spec*
> >>
> >> No spec.
> >>
> >> Some discussion happened here:*
> https://github.com/whatwg/html/issues/976
> <https://github.com/whatwg/html/issues/976>*

> >>
> >> It will likely followed by minor spec changes.
> >>
> >>

> *>>*
> *>> Summary*

> >>
> >> Platform will support autoplay of media elements without requiring a
> user gesture if they are muted (using the muted attribute/property). User
> gesture will be required to unmute the media element programmatically. If
> not, the media element will pause.
> >>
> >>
> >>
> >> Media elements using autoplay attribute will only start when they are
> visible in the user viewport.
> >>
> >>
> >>
> >> Platform will not support autoplay if Data Saver is enabled and / or
> when the user indicates to turn off autoplay using a specific setting.
> >
> >
> > Can this be fully decoupled from Data Saver mode? A core promise of Data
> Saver from my POV is that bandwidth is saved, but otherwise absolutely
> nothing else changes in my browsing experience. I believe this would be
> the first time that Data Saver changes/breaks something user-visible, and
> in a way that's quite surprising.
> >
> > I think it would be better for the new specific setting you're
> introducing to control the autoplay behavior, full stop.
>
> Alternatively, I'd suggest tying this to "Lofi mode" which is triggered
> when bandwidth is detected to be very low, and whose main effect today is
> to disable images. That's the "OK to break the web" submode of Data
> Saver,
> with an infobar and opt-out button.
>
>
>
> >
> >
>
> >>
>
> >>

> *>>*
> *>> Motivation*

> >>
> >> The web is increasingly media centric. Background videos as design
> elements, effective use of media as narrative pieces is very common.
> However, the mobile web today requires a user gesture for playback of
> media.
> >>
> >>
> >>
> >> However, animating pixels on the Web isn’t very hard. Motivated websites
> have resorted to two common workarounds -
> >>
> >> Animated images - which are bandwidth inefficient (~10x) larger than
> equivalent video
> >>
> >> Decoding video in javascript - this is incredibly power inefficient
> >>
> >>
> >>
> >> Enabling muted autoplay ensures the platform provides the right support
> for the intent. This also enables increasingly common usecases on the web
> such as background video as a design element.
> >>
> >>

> *>>*
> *>> Interoperability and Compatibility Risk*

> >>
> >> Firefox: No public signals (already ship with autoplay by default on
> mobile)
> >>
> >> Edge: No public signals
> >>
> >> Safari: Public support
> >>
> >> Web developers: Strongly positive
> >>
> >>
> >>
> >> Safari is supportive of the spec changes. It is the only other browser
> blocking autoplay on mobile.
> >>
> >>
> >>
> >> With the recent spec change of Promise<> return for play(), we now have
> the full feedback mechanism to developers to indicate successful
> playback.
> >>
> >>

> *>>*
> *>> Ongoing technical constraints*
> >>
> >> None.
> >>
> >>
> *>>*
> *>> Will this feature be supported on all six Blink platforms (Windows,
> Mac, Linux, Chrome OS, Android, and Android WebView)?*

> >>
> >> As far as Chrome is concerned, this will only ship on Android. However,
> the feature itself doesn’t have platform specific constraints.
> >>
> >>

> *>>*
> *>> OWP launch tracking bug*
> *>> <https://crbug.com/618000>*
> *>> https://crbug.com/618000 <https://crbug.com/618000>*
> >>
> >>
> *>>*
> *>> Link to entry on the** feature dashboard
> <https://www.chromestatus.com/>*
> *>> <https://www.chromestatus.com/features/4864052794753024>*
> *>> https://www.chromestatus.com/features/4864052794753024
> <https://www.chromestatus.com/features/4864052794753024>*
> >>
> >>
> *>>*
> *>> Requesting approval to ship?*
> >>
> >> Yes.
> >
> >
>
> --

[Philip Jägenstedt]

A spec change will also be required for the autoplay attribute. Currently, there is exactly one place that says "If the autoplaying flag is true ..." where the UA may play, and I think it's important that the timing of the second opportunity to autoplay ("when [media elements] are visible in the user viewport") is also well defined. There's enough room for variation on the details that I doubt we'd get interoperable implementations of the behavior without that.

(FWIW, I'm happy that delaying play() isn't part of the current plan, because that's something that a script should already be able to do.)

We should also change the muted setter in the spec to optionally pause, so that the order of the volumechange and paused attribute is defined.

Having this spec PR in place would be good before this ships, is that something you'd like to work on? I'll do my best to review in a timely manner :)

Philip

[Dale Curtis]

On Thu, Jun 9, 2016 at 3:17 AM, Mounir Lamouri <mou...@lamouri.fr> wrote:

Dale, regarding playing videos off tree to render them with WebGL, I
would say that given the principles used to say that we should allow
autoplay muted video there is no reason why it shouldn't be allowed. I
guess if the video was set as muted, it would work. In the future, we
might not required the muted attribute if we can detect that there is no
audio track. Though it's not planned for the first step.

My point was that these videos have no visibility information, which also means no controls and no way to unmute if they're also providing the audio track.

 

[Mounir Lamouri]

Before anything, we are not planning to fix all the use cases with this
change. This website on Chrome Android simply does not work today. If we
can reach a state where it can work in some ways, it is a win. We should
aim for the best web developers and users experience - finding a balance
is hard and we need to iterate.

This said, the visibility information isn't a requirement. We will only
autoplay visible videos if the autoplay attribute is set. We will follow
the request coming from the play() method. I believe such a complex use
case will not rely on the browser to autoplay the video.

I'm not sure why it should have no controls. It should be possible to
have buttons that interact with the off-tree video, right? An unmute
button for example. This said, the website has a "Begin" button which
could be used to "unlock" all the videos and require no user gesture
then.

-- Mounir

[chris.h...@gmail.com]

In terms of accessibility considerations, autoplay is a bad plan. Of course, GIFs do that now, but why not also show a frame and a play button? That way we can even save more bandwidth and avoid confusing all kind of users with different abilities.

https://www.w3.org/TR/2008/REC-WCAG20-20081211/#time-limits-pause

[Mounir Lamouri]

On Sat, 11 Jun 2016, at 00:58, chris.h...@gmail.com wrote:
> In terms of accessibility considerations, autoplay is a bad plan. Of
> course, GIFs do that now, but why not also show a frame and a play
> button?
> That way we can even save more bandwidth and avoid confusing all kind of
> users with different abilities.
>
> https://www.w3.org/TR/2008/REC-WCAG20-20081211/#time-limits-pause

With regards to autoplay and a11y, Marcy Sutton pointed to the same
issue on Twitter [1] and she seemed happy with the autoplay settings
that offers user to block autoplay entirely [2]. Hopefully, that
addresses your concerns too :)

[1] https://twitter.com/marcysutton/status/741506299825782784
[2] https://twitter.com/marcysutton/status/741516897355595778

-- Mounir

[Rick Byers]

LGTM2

IMHO this is relaxing an existing intervention to reflect that it was a misguided design that didn't account for how the ecosystem would react to the change (more use of animated gifs, JS video decoders, etc.).  This is a really valuable lesson to internalize IMHO - our attempts to improve the user experience by preventing autoplay ultimately harmed both the user and developer experience in the long run.  Thanks for pushing to fix that mistake ASAP Mounir!

There's obviously lots more to be done in this space (figure out audio obviously, and how exactly the specs should be updated to reflect the implementation of media interventions in practice).  But given the existing implementations and discussion in the community, I don't see any reason to delay making this change to our existing intervention.  In particular, I'm thrilled at the prospect that Mobile Safari may match this behavior.

Rick

[Chris Harrelson]

LGTM3

[Mounir Lamouri]

FWIW, Apple announced that Safari 10 will ship autoplay for muted videos
and videos without audio tracks. They will also enable autoplay for
"primary content".

More info: https://developer.apple.com/safari/

-- Mounir

On Mon, 13 Jun 2016, at 18:33, Chris Harrelson wrote:
> LGTM3
>
> On Mon, Jun 13, 2016 at 7:24 AM Rick Byers <rby...@chromium.org> wrote:
>
> > LGTM2
> > IMHO this is relaxing an existing intervention to reflect that it was a
> > misguided design that didn't account for how the ecosystem would react to
> > the change (more use of animated gifs, JS video decoders, etc.). This is a
> > really valuable lesson to internalize IMHO - our attempts to improve the
> > user experience by preventing autoplay ultimately harmed both the user and
> > developer experience in the long run. Thanks for pushing to fix that
> > mistake ASAP Mounir!
> >
> > There's obviously lots more to be done in this space (figure out audio
> > obviously, and how exactly the specs should be updated to reflect the
> > implementation of media interventions in practice). But given the existing

> > implementations and discussion <https://github.com/whatwg/html/issues/976>
> > in <https://github.com/WICG/interventions/issues/23> the community, I

> > don't see any reason to delay making this change to our existing
> > intervention. In particular, I'm thrilled at the prospect that Mobile
> > Safari may match this behavior

> > <https://github.com/whatwg/html/issues/976#issuecomment-210527388>.

[Darin Fisher]

Great to see. It seems like this will enable autoplay with sound in some cases. Is that right?

[Mounir Lamouri]

Yes, it is possible that it means that Safari will autoplay videos with
sound if they are considered primary content. It is something I would
like to test before making conclusions. Though, we were considering
enabling autoplay on top frame content which might be similar to what
they are doing in which case, we should reconsider this :)

-- Mounir

[Darin Fisher]

I wonder if it uses the same heuristic as their version of plugin power saver.

-Darin

[btk...@gmail.com]

Thanks to all working on this. 

What I simply do not understand is why this isn't being consider the same as other hardware access via the browser? We all love using web conferencing, this potentiates almost immoral access to hardware, but is mitigated successfully by far less draconian means. Simply make usage of the end user's media functionality accessible via white listing. The spec as it has been implemented greatly pushes the community to fracture, creating 'native' browsers like electron and react-native. Those projects are great, but they should not exist out of necessity.