Sandoxed Iframe throws: "play() failed because the user didn't interact with the document first"

[Roberto Oneto]

Hi folk,

I'm developing a quite simple radio player extension.

I have a extension page with all player commands (play/stop, next/previous station etc) while the main audio element <AUDIO> is placed inside an sandboxed iframe of the same page.

I'm exchanging messages between page and iframe to act on playback.

I've chosen to isolate the two environments because in the sandboxed iframe I'm already using some remote code (external JS library).

Is important to state that I'm testing the extension with Chrome Dev channel (version 127.0.6485.0)
Everything was working fine until some day ago.
Now I've found this error in console:
NotAllowedError: "play() failed because the user didn't interact with the document first".
Only clicking on that iframe and trying back to initiate the sound reprodution (with "Play" button) the radio sound starts.

My question is why?
Some day ago everything worked well.
I installed the same extensin on different browsers and I found that Edge, Iron and SlimJet don't raise that error.
Edge version is 125.0.2535.67
Iron version is 123.0.6250.0.
while SlimJet version is 122.0.6261.39

Is a version issue or a browser issue?

Something that I can report as a bug...

...or is it a deliberate behavior (which sooner or later I will also find on Edge and Iron)?

TIA

[Roberto Oneto]

I installed the same extensin on different browsers and I found that Edge, Iron, SlimJet, Brave and Naver Whale don't raise that error.

Edge version is 125.0.2535.67
Iron version is 123.0.6250.0.

SlimJet version is 122.0.6261.39

Naver Whale version is 124

while Brave version is 125.0.6422..112

[woxxom]

It may be a bug, you can report it in https://crbug.com if it wasn't already reported. Meanwhile try playing the file in the extension page itself. If it needs data generated by external code you can send it via parent.postMessage. For more efficient communication you can also transfer a MessageChannel port via parent.postMessage and then use the port directly. There are probably many examples around, here's one I remember making for a somewhat similar setup: https://stackoverflow.com/a/68689866

[Jackie Han]

It may be a bug, But it is very likely not a bug, Chrome has limitations (policy) about autoplay. See these links:

• https://developer.chrome.com/blog/autoplay/
• https://developer.chrome.com/blog/play-returns-promise
• chrome://media-engagement/
• https://stackoverflow.com/q/49930680/1330598
• https://issues.chromium.org/issues/40831921 (someone already reported a bug)

Workaround:

try { await media.play(); } catch(e) { show_tip_for_user_click_play_button }

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/7882dacd-fe3d-48a9-9317-b375c979b3b2n%40chromium.org.

[woxxom]

Extension pages should be exempt from this restriction, otherwise it'd be meaningless to have reason: ['AUDIO_PLAYBACK'] in the offscreen document.

[woxxom]

Moreover, many radio extensions have been playing audio in their background pages for years.

[Roberto Oneto]

 @woxxom  If I move the AUDIO element out of the iframe it works. However, I had already verified this.

By clicking on the "Play" button, reproduction starts because in fact "I am initially interacting with the document".

If I put the AUDIO in the iframe instead there is no interaction apparently.

But it worked before and even now on all Chromium based browsers except Chrome it still works; which seems really strange to me.

@ Jackie Han  I imagine that every browser vendor has policies that are sometimes slightly different from their competitors, but we are always talking about Chromium...

Perhaps on these topics they should find a common front in order not to drive developers crazy to test and adapt their code.

I tryed a workaround like the one you suggest me:   try { await media.play(); } catch(e) { show_tip_for_user_THEN_click_play_button }
but I think it's a terrible solution. Thank anyway for the suggestion.

I also tried with allow="autoplay" inside the iframe tag but it doesn't work.

I saw the bug report about this kind of issue, so all I have to do is wait and...hope

Thanks everybody

[woxxom]

I see Chromium is working on sandboxed extension frames to be handled in a separate isolated process for proper security, not in the same extension process, so they don't inherit the audio permission, which is apparently granted per process. Although this commit is not yet merged, but maybe there was another similar change that wasn't yet adopted by other Chromium-based browsers. I would consider it a bug, given the fact that you explicitly allowed audio on the iframe element, so you should report it in https://crbug.com. Meanwhile you'll have to play the audio in the extension page and there should be no need for a click because extension pages have been able to play audio without user interaction e.g. on a schedule, like an alarm clock.

[Roberto Oneto]

I just found out that the problem is the sandbox itself!

I commented out that section in manifest.json and everything worked again.

The allow="autoplay" doesn't make any difference once the sandbox is removed

[Jackie Han]

I add a "Spec Clarification" issue on WECG: https://github.com/w3c/webextensions/issues/625

[woxxom]

To clarify the specification with the possibility of introducing a breaking change the necessary step is evaluation of the existing use cases as well as generally allowed ones, so I encourage you and others to list list such examples including the two I listed: radio and scheduled audio playback (alarm clock).

[Roberto Oneto]

Thanks everybody.

I read the Spec Clarification request on WECG and... could not have been written in a better way! Thaks @Jackie Han

@ woxxom   for audio palyback just like a  short duration alarm clock, I used an Offscreen document and I have not yet face this kind of issue with that kind of document.

In this case (OSD), there is no interaction with the user at all and the playback starts anyway with no problem!

[woxxom]

Yes, of course, it's because the offscreen document is an extension page, which currently are exempted from the autoplay restriction for the web. But, if the spec is clarified to introduce a new permission, the offscreen document won't be able to play audio by default in extensions without the permission. This is why I suggest adding the existing use cases as well as possible ones.

[Jackie Han]

I updated the issue with suggested use cases and more explanation.

I don't think this is a high priority issue. It involves other teams (multimedia team and policy team) in the browser.