PSA: Autoplay and Extensions policy changes in Chrome

[hu...@webrtc.org]

Google and other browser vendors continuously balance the need for powerful web platform capabilities, with the need for a safe and great user experience. Recent changes in media autoplay and extension installation are aimed to reduce misuse of browser capabilities and may affect some WebRTC services.

Screen capture

WebRTC services can capture screen content using the chooseDesktopMedia API, which is only available to Chrome extensions. After September 12th, extensions will no longer be able to be installed “inline” using the  chrome.webstore.install() API. Users can still install and use extensions, but will need to navigate to the extension details page in the Chrome Web Store to complete the install. Extensions that are already installed are not affected.

Google is in the process of implementing the standardized getDisplayMedia API for screen capture, which does not require a Chrome extension. However, this implementation is unlikely to be completed before the new Chrome Web Store changes take effect in September. We therefore recommend WebRTC services to update their extension installation method as outlined in this FAQ.

We recognize that this will cause some difficulty in the short term, but the end result of not needing a Chrome extension should lead to a significantly improved experience, and we are working to make that happen as soon as reasonably possible.

Autoplay

To combat the proliferation of auto-playing videos and ads, Chrome 66 introduced new policies that restrict usage of autoplay for <audio> and <video> elements. The policies will also be enabled for AudioContext (the Web Audio API) in Chrome 70, which will be released around October 16th. These changes may affect WebRTC-based services.

How it affects WebRTC services

MediaStreamTracks from PeerConnections or local devices that are played out in audio/video elements or in an AudioContext may not autoplay; per Chrome policy, a user gesture will be required before any attempt to play out audio.

General Recommendations

Developers should assume that media elements (<video> and <audio>) as well as Web Audio (AudioContext) may not be allowed to autoplay before a user gesture. Currently, such elements that have a MediaStream assigned via the srcObject attribute (as is the case in WebRTC applications) are allowed to autoplay, but this might change in the future. As such, WebRTC applications should ensure that a user gesture is received before trying to play MediaStreams. When autoplay is not allowed, this will result in a failed play() promise for media elemented and a suspended AudioContext for Web Audio. Developers should detect this, and enable audio (either by calling play() for media, or resume() for an AudioContext) after the next user gesture. More detailed recommendations, including debugging tips, can be found at https://developers.google.com/web/updates/2017/09/autoplay-policy-changes.

We will be evaluating the impact of this policy and possible improvements over the next few releases. While the intent is to allow autoplay in legitimate use cases, developers should assume that autoplay may require a user gesture in certain situations, including future policy adjustments or user settings.

Processing MediaStreams in Web Audio

WebRTC applications often use an AudioContext to implement a microphone volume meter to provide visual feedback to the user. As noted above, the current direction is that these uses of Web Audio will also require a user gesture (as of Chrome 70). While OfflineAudioContexts are exempted from the user gesture requirement, the use of MediaStreams with OfflineAudioContexts has been deprecated and therefore their use in the "volume meter" scenario is discouraged.

We recognize that Web Audio processing that results in no audio output is a legitimate use case and are evaluating various potential improvements for this scenario.

Playback in hidden muted video elements

A common pattern in WebRTC applications is to use a hidden, muted video element to process the video from a MediaStream using capture from element. In these cases the autoplay attribute should not be used as Chrome will not start playback until the element becomes visible. The recommendation is to directly invoke the play() method of the hidden muted video element to ensure playback. No user gesture is required to play a hidden muted element with the play() method.

[Huib Kleinhout]

An update on Autoplay:

In the past months various improvements in handling legacy sites and detecting legitimate autoplay cases have been implemented and in Chrome M71 (reaching stable early December) the autoplay policy will be re-enabled for Web Audio.

Updates to the autoplay documentation will come soon, but most notably for WebRTC services autoplay will be allowed when microphone or camera capture is active. 

The general policy still stands: web services are recommended to handle the possibility that autoplay is prevented when the user has not performed a user gesture yet.

Huib Kleinhout

Product Manager @ Google

[David Fridley]

Hi,

I don't know where to bring this up, but I found this post so I am starting here.  Please let me know if there is somewhere else to take it.  I  wish that WebRTC would define a standard for clients to ask for policy permission upfront, and asking if that permission can be persistent.   I make this requires because I am  building a  web app that runs on chrome and safari and mixes realtime media with recorded media - the purpose of the app is structured, large scale deliberative discussion (that's productive rather than polarizing).  The problem that is the inconsistent policy experiences between implementations (chrome and safari) and the poor user experience this creates.   I realize that you want to prevent abuse, and I am all for that because I hate it when sites play videos (with loud audio) unexpectedly.  

In chrome ( 71.0.3578.98 ), I have implemented a button to click on, that causes the user interaction that allows autoplay.  But there is still a separate popup asking for permission to use the camera.  On safari, (on iOS 12.1.2) it seems that I need to get permission each time I auto play (I auto play things in a sequence), also it seems there is 'lowered volumne' audio until there is another user interaction.

What I'm thinking is that my app (javascript running on the client) could request permissions at the beginning, permissions to access the camera, auto play video, audio, switch media sources, and what ever other policy questions we come up with.   Much like a native application does when you download it.  And that the user could choose if this was permitted one-time, for the day, for the page, or for the site. 

My thinking is, if you make it an explicit part of the API then it will be (fairly) consistent across applications, and then developers could provide a consistent user experience.

Where as, if everyone tries to guess, by trial and error, how abusers are going to take advantage, we will always be losing a cat and mouse game, and/or there will be so much inconsistent security policy that innovative new application of audio and video won't  be possible, or will be encumbered by bad user experiences. 

I am open to other ideas, or if there is a way to gather all the permissions at once that I am not seeing, please let me know.

Thanks

David

[hu...@webrtc.org]

On Tuesday, January 29, 2019 at 2:56:22 AM UTC-8, David Fridley wrote:

Hi,

I don't know where to bring this up, but I found this post so I am starting here.  Please let me know if there is somewhere else to take it.  I  wish that WebRTC would define a standard for clients to ask for policy permission upfront, and asking if that permission can be persistent.   I make this requires because I am  building a  web app that runs on chrome and safari and mixes realtime media with recorded media - the purpose of the app is structured, large scale deliberative discussion (that's productive rather than polarizing).  The problem that is the inconsistent policy experiences between implementations (chrome and safari) and the poor user experience this creates.   I realize that you want to prevent abuse, and I am all for that because I hate it when sites play videos (with loud audio) unexpectedly.  

In chrome ( 71.0.3578.98 ), I have implemented a button to click on, that causes the user interaction that allows autoplay.  But there is still a separate popup asking for permission to use the camera.  On safari, (on iOS 12.1.2) it seems that I need to get permission each time I auto play (I auto play things in a sequence), also it seems there is 'lowered volumne' audio until there is another user interaction.

Cross-browser consistency is certainly important, I'll forward your feedback to team in Google working on it. 

 

What I'm thinking is that my app (javascript running on the client) could request permissions at the beginning, permissions to access the camera, auto play video, audio, switch media sources, and what ever other policy questions we come up with.   Much like a native application does when you download it.  And that the user could choose if this was permitted one-time, for the day, for the page, or for the site. 

In old Android this was possible (though not time bound) and it got extensively overused unfortunately. Context specific and access specific permissions are more complex for developers, but provide much better privacy for end-users. Autoplay should generally not be blocked for legitimate use cases though, so not requiring an extra click. 

[pps]

Sorry to hijack, webrtc groups has the "Note: If it's your first time posting your message will be moderated and will not appear straight away, please be patient, it's not necessary to post/send emails several times.", should I wait till next new year? does anybody even check these new messages? It's been like weeks I'm checking for my posts and they aren't there.

Thanks

[Harald Alvestrand]

In standards space, access to query the permissions a page has is proposed using the permissions API (https://w3c.github.io/permissions/).

An API is presented to query permissions: https://w3c.github.io/permissions/#permissions-interface

The ability to request permissions using this API has been extremely controversial, and is at the moment not part of the API proposal - the section on requesting permission (https://w3c.github.io/permissions/#requesting-more-permission) is intended to be invoked from other specifications as appropriate.

The proper forum for discussing this API is the W3C Webappsec WG - the email list address is given in the document header.

There's also a "feature policy" specification (https://w3c.github.io/webappsec-feature-policy/#introspection) that describes how to use page headers to say that certain features are off limits to a page no matter what the user wants - this allows one to run iframes and subpages knowing that no matter what happens, it won't be able to use the given feature.

You'll have to check the status of these APIs in each browser in order to figure out what the status is.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/0ab995c2-46dc-4d8b-9e93-0b7e34dbf6a5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[david]

I really appreciate the referrals!

It sounds like all the attention is going into protecting users from bad apps. I need to go speak for the yet to be innovations that won't be if a balance is not maintained. 

Thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CAOqqYVENdc4PYzi6_K4Z86LP47CCWsNDfJrY8Nf7U2xmmjVOtQ%40mail.gmail.com.

[Harald Alvestrand]

If your messages to a W3C list don't appear, try reaching out to the chairs of the WG. They should be able to check if someone's forgotten to approve them. It shouldn't take more than a couple of working days.
I believe it's W3C staff that does the vetting of new submisssions; if this isn't happening, the WG may have a problem.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/0b7b2c58-e8d4-46a3-a1b2-dffe7985dea7%40googlegroups.com.