How to initiate audio playback from Chromium

[Shlomi Ben Shimon]

Hi,

I'm new to Chromium and I'm exploring a path to build a small PoC for audio playback UI in Chromium.

The UI should be native and very minimal - just basic playback controls like play/pause/seek.

To start with, I have some audio bytes stream (received from our service) and I'm looking into feeding it into a media player.

We also need to be able to control the playback (play/pause, seek, etc...), and get notifications regarding the playback status (buffering, stopped, playing, error...) for updating a UI in the browser.

Is it possible to start an arbitrary audio playback from the browser process?

I've come across Blink's web_media_player, but I suspect it's too low level and I couldn't find out how it can be used directly from Chromium.

I've also seen this group message suggesting a possible workaround of creating an HTML <audio> element, and I wonder whether that's the recommended current approach and whether it supports our need?

If so, can you send me some relevant code pointers to look into?

Thanks!

Shlomi.

[guest271314]

You can build your UI around Web Audio API AudioContext.

If you are streaming audio you can't really seek. You can play and pause to an appreciable degree without using HTMLMediaElement.

--
You received this message because you are subscribed to the Google Groups "media-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to media-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/media-dev/CALBA9twwjbYTUnmtsMSG_Ro5McYc5KF3QBmmoxe7RoyLAJpR-w%40mail.gmail.com.

[Shlomi Ben Shimon]

Thanks!

Re-seeking, I will correct myself - our audio consists of multiple closed mp3 files that we wish to play one by one, so seeking is in the granularity of these files, or in the middle of one of the files (while we do have the entire file information). In that case I assume seeking should be supported?

Re Web Audio API AudioContext - looks like that's part of Blink as well?

Can you share a bit more on how to trigger that from Chromium (e.g from a mojo service)?

[guest271314]

I must have not posted the link to the SO answer https://stackoverflow.com/a/45343042. Where is the user expected to interact with the media player UI?

[Shlomi Ben Shimon]

Thanks!

To start with, the user is expected to interact with the UI in some dedicated player near the address bar or something similar (part of the browser controls).

Point is that the player is part of the browser but isn't part of the web page dom.

Does that make sense?

Regarding design - assuming you are referring to the UI design?

In that case, I'm imagining a play button that can toggle play/pause initially.

As a next step, I'll be looking into adding seek capabilities, speed control, and more advanced parameters (which are not necessarily tied to the playback but rather to some parameters of our backend).

Is there an way to build a custom UI that initiate the playback and listen to state changes events? 

[guest271314]

Yes, the requirement is possible. You can use an extension to create an offscreen document that plays the audio and use a popup window for controls. In the previously linked extension tab-pip-stream branch I use a PictureInPictureWindow and Chrome global media controls for playback. Here I stream arbitrary data from a ServiceWorker to the Web page using a window that can be positioned in any part of the viewport and include custom media controls https://github.com/guest271314/sw-transfer-stream; here I use WebRTC to stream arbitrary data to the Web page from an extension ServiceWorker using an offscreen document and WebRTC data channel. The offscreen document could be used to play the audio and you can create controls to play, pause, seek the audio playing in the offscreen document. mpv has existing JavaScript bindings to media playback controls that you could use as a template, see https://github.com/mpv-player/mpv/blob/1ffdb9128d87dffe869c1e9f4024a16bfd0022c4/DOCS/man/javascript.rst. Technically you can launch and control an instance of mpv from the browser.

[guest271314]

Here is the offscreen document WebRTC repository https://github.com/guest271314/offscreen-webrtc/blob/main/background.js. You'll notice I stream a 291MB WAV file to the browser which can be played back during the fetch request and transfer. Since the WebRTC connection is between an offscreen document and arbitrary Web page you can control the offscreen document from the Web page, including controlling media playback, so you can create a UI in the active window that is persistent and runs in parallel on all open tabs and windows. You can use a an extension popup and content script for controls. I think the requirement is possible at the front end, with the ability to fetch and stream, potentially indefinitively, specific media content from server side or client side.

[Shlomi Ben Shimon]

Thanks again for your elaborated answers!

IIUC, the path you are describing is by using Chromium extensions (written in JS), am I right?

If so, I think we are looking into a deeper integration in the Chromium core (c++) code.

Are you familiar with any way we can initiate a playback directly through a core chromium component, rather than through a JS extension?

Also, with regards to WebRTC, do we still need to use it in case we don't require any integration with the web page?