Design doc: Audio service

[Olga Sharonova]

Contacts:

ol...@chromium.org, maxm...@chromium.org

Summary:

Moving platform audio to a Mojo service and out of browser process.

Design doc:

https://docs.google.com/a/chromium.org/document/d/1s_Fd1WRDdpb5n6C2MSJjeC3fis6hULZwfKMeDd4K5tI/edit?usp=sharing

[Raymond Toy]

Awesome.  

How will this affect latency?  Will it be the same as today?

How regular will the audio callbacks be?  WebAudio is pretty sensitive to latency and callback frequency.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAHri2_QqdWDctWBXV5LMFYgK9o0B-0%2BuJtLQ3LvPir6cXWEg8w%40mail.gmail.com.

[Olga Sharonova]

Excellent question!

Short answer: latency and callback regularity will remain unchanged.

Long answer:

There is a preceding effort in progress to mojofy audio IPC. It will switch control path of renderer/browser audio stream interactions to mojo (mojom::AudioOutputStream is an example), but real-time rendering mechanisms will remain untouched (sync socket/shared memory). So no extra latency on rendering path is introduced at this step.

The next step is moving audio streams to be hosted in the audio service.

This will require 2 hops for a stream creation: renderer requests a stream from browser, browser checks permissions and requests the stream from the audio service/process. But all the further interactions with AudioOutputStream including real-time audio rendering are preformed directly between renderer and the audio service/process in the same way as they are performed between renderer and browser now[after step 1].

So from real-time audio rendering perspective moving the streams to another process is transparent and should not affect neither latency nor callback regularity.

[Raymond Toy]

Even more Awesome!

Thanks for the info. We look forward to the servification.

[Olga Sharonova]

The design doc has been updated with the details on servicification approach: 

How we are planning to transition to in-process and later out-of-process Audio Service.

(No fine details for experiments yet.)

Olga

[Peter Kasting]

I did a quick scan through the document and didn't see anything about TTS (text to speech).  Have the needs in this area been considered?

(For more context, see https://bugs.chromium.org/p/chromium/issues/detail?id=517317#c30 -- moving TTS capabilities to some kind of service is part of making TTS governed by the standard site muting controls.)

PK

[Olga Sharonova]

Regarding tab muting/tab audio indicator (also +miu@ on that):

We'll be refactoring the browser-side business logic layer which now tracks audio streams created by renderers and interacts with WebContents/AudioStreamMonitor.

media parts there are now quite mixed with //content, and we'll be separating them.

(1) media logic (physically muting groups of output streams) will be moved to the Audio service,

and (2) content part (grouping output streams by WebContents, reporting to AudioStreamMonitor and initiating mute/unmute) will remain in the browser.

My understanding is TTS uses TTS engine as a black box to render audio directly to the output device, no Chrome audio rendering infrastructure is involved - correct?

In this case Audio service cannot help with muting since audio bypasses it.

But we may share some portion of content logic (2) with TTS.

You can also take a look at AudioOutputStreamObserver mojo interface and implementation.

Makes sense?

Are there other needs in TTS area we should conciser?

Olga

[Peter Kasting]

On Wed, Dec 13, 2017 at 10:06 AM, Olga Sharonova <ol...@chromium.org> wrote:

My understanding is TTS uses TTS engine as a black box to render audio directly to the output device, no Chrome audio rendering infrastructure is involved - correct?

I don't know.  I don't work on TTS, it's more that I have an interest in seeing this bug fixed, and I know the TTS folks wanted that to be refactored to a separate service.  Since it sounded like that may interact with/be subsumed by this, I wanted to make sure people on each side were aware of each others' needs and talking to each other about the right overall architecture.

So my only request is that you read the comment I linked to, and if it sounds like in the future it might interact with your code, reach out to the couple folks in charge of that stuff.

PK 

[Jeremy Randolph]

Get Outlook for Android

---

From: pkas...@google.com <pkas...@google.com> on behalf of Peter Kasting <pkas...@chromium.org>
Sent: Wednesday, December 13, 2017 2:38:30 PM
To: Olga Sharonova
Cc: Yuri Wiitala; Chromium-dev; services-dev
Subject: Re: [chromium-dev] Design doc: Audio service

 

--

--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAAHOzFBH2FsmGcpaATB30aNk4D%3DP%2BM93jKsAL19VV47RHe7rKg%40mail.gmail.com.

[f...@bebo.com]

Hi,

sorry we're late to this discussion. We are working on capturing audio + video from capture cards inside chromium (nw.js) for capturing video games.

From what we see most capture cards expose a single DirectShow capture filter for both audio and video. And you can only have one per hardware instance.

If it would be possible to run both audio & video capture inside the same process, that would significantly simplify things.

best regards,
Florian

[Olga Sharonova]

​Audio service design doc is updated now.

The list of open questions can be found here.

Audio process experiment is described here.

Olga

​

On Fri, Jun 2, 2017 at 4:30 PM, Olga Sharonova <ol...@chromium.org> wrote: