Audio input default 8Khz breaks peerconnection factory

[ak]

Hi

Mac OS X Lion appears to default the built in audio device sampling to
8Khz and this breaks the peerconnection factory. Even if one manually
sets the sampling to a supported rate (eg: 44.1Khz) on restarting the
Mac the system resets it back to 8Khz.

There are no other applications accessing audio on System Restart.

Is there any reason 8KHz sampling rate is not included in your Input
Device Selection?
Without it webRTC is dysfunctional in OS X Lion.

Thanks.
A

[1457:-1397779776:15521853857370:VERBOSE3:webrtcvoiceengine.cc(321)]
WebRtcVoiceEngine::Init
[1457:-1397779776:15521853936044:VERBOSE2:webrtcvoiceengine.cc(341)]
Init() failed, err=10028
[1457:-1397779776:15521853955435:VERBOSE1:webrtcvoiceengine.cc(326)]
WebRtcVoiceEngine::Init failed
[1457:-1397779776:15521853967846:VERBOSE3:webrtcvoiceengine.cc(414)]
WebRtcVoiceEngine::Terminate
[1457:-1397779776:15521864488484:VERBOSE3:webrtcvideoengine.cc(383)]
WebRtcVideoEngine::~WebRtcVideoEngine
[1457:-1397779776:15521875309974:ERROR:media_stream_impl.cc(469)]
Could not create PeerConnection factory

[ak]

FYI Issue Tracker #437:

Currently we fail to initialize if either the playout or recording
side uses a non-supported sampling rate. For playout we support
{96000, 44100,48000} and for recording we support
{96000,48000,44100,32000,16000}.

[Henrik Andreasson]

I am a bit confused here. Are you saying that the format you set for the microphone is overwritten by the OS at each restart? Does the same thing happen for a USB headset?

WebRTC requires at least  16kHz sample rate to support e.g. iSAC which is a wideband (fs=16kHz) speech codec. Allowing 8kHz will reduce the performance substantially.

[Ivan Vučica]

If Lion really breaks WebRTC audio input (and I didn't yet test if it does), converting 8kHz into 16kHz is easy, isn't it? Just repeat twice the incoming data. 

I took a quick look at Mac Mini's input under Lion, and its Line-In is set to 44.100Hz after restart. It doesn't have a microphone so I can't check what's the situation with a microphone until I get to my Macbook.

Also, I did work a bit with Core Audio playback, but not with Core Audio input; that means I don't know if that's the case, but it doesn't sound unreasonable that one could tell the OS to feed the app 16kHz stream no matter what the defaults or user configuration is.

--
Ivan Vučica - iv...@vucica.net

[Henrik Andreasson]

I am trying to understand the background to the problem and how common it is, but yes, it is possible to support 8kHz as well even if it would degrade the overall user experience.

[ak]

Henrik

Yes exactly what is happening. Leaving the Audio Midi Setup utility
open and restarting with the "Reopen windows on login" option checked
one can actually see the sampling rate switch back (from say 44.1Khz
manually set) to 8Khz.

Have tested 5 different Mac OS X Lion machines (Macbooks ... Mac
Air ... OS X 10.6.8 ... 10.7.3) and all exhibiting the same behavior.

Bizarre.

Also forgive my limited understanding regarding Audio/Codecs however
my questions:

1. Surely webRTC (or any app) can request I/O at any supported rate?
Why is webRTC simply accepting the default and failing to INIT because
the default is not on the webRTC list when the device clearly does
support the higher standard sampling rates (16 32 4.11 48 88.2 96)?

2. On using narrowband speech (G711 @8Khz) the System Audio OUT shows
44.1Khz. What is the difference between the hardware and codec rate?
Where does the sampling rate conversion happen? Is that a function of
the device driver itself?

3. Surely such a "critical" INIT error should bubble up to the UI?
Does getUserMedia return the error? All the demo apps simply hang.

Lastly while 8Khz would degrade performance using G711 is at the lower
sampling spectrum anyway and right now there is no performance :)

A.

[Henrik Andreasson]

Thanks for the detailed input. I'll ensure that we handle this scenario in a better way than today.

It is a rather surprising behavior if I may say so.

See more detailed answer inlined below:

On Mon, Apr 23, 2012 at 7:46 PM, ak <star...@gmail.com> wrote:

Henrik

Yes exactly what is happening. Leaving the Audio Midi Setup utility
open and restarting with the "Reopen windows on login" option checked
one can actually see the sampling rate switch back (from say 44.1Khz
manually set) to 8Khz.

Have tested 5 different Mac OS X Lion machines (Macbooks ... Mac
Air ... OS X 10.6.8 ... 10.7.3) and all exhibiting the same behavior.

Bizarre.

Also forgive my limited understanding regarding Audio/Codecs however
my questions:

1. Surely webRTC (or any app) can request I/O at any supported rate?
Why is webRTC simply accepting the default and failing to INIT because
the default is not on the webRTC list when the device clearly does
support the higher standard sampling rates (16 32 4.11 48 88.2 96)?

The exact answer varies depending on the platform but in short; you can't set any possible sample rate (e.g. 192kHz) and expect it to work today.  In your list above, 88.2 is not supported.

 

2. On using narrowband speech (G711 @8Khz) the System Audio OUT shows
44.1Khz. What is the difference between the hardware and codec rate?
Where does the sampling rate conversion happen? Is that a function of
the device driver itself?

WebRTC does the resampling and adjust the sample rate after decoding to match the audio layer.

WebRTC queries the audio layer for its native rate and adjusts accordingly to match it. This statement holds for most native rates.

 

3. Surely such a "critical" INIT error should bubble up to the UI?
Does getUserMedia return the error? All the demo apps simply hang.

The current implementation does not feed this type of error up to the UI today, or to getUserMedia.

I will ensure that the demo application does not hang.

 

Lastly while 8Khz would degrade performance using G711 is at the lower
sampling spectrum anyway and right now there is no performance :)

The default codec selected in WebRTC is iSAC 16kHz.  There is a clear QoS improvement between iSAC and G.711. Your perceived QoS will be reduced substantially if 8kHz is used as capture rate.

[ak]

Thank you for your prompt response and audio tut.

What other option is there for G711 (8Khz) when interworking with the
PSTN? Practically no gateway out there suports broader band on the
interconnect and transcoding from iSAC still results in lower quality
audio since the limiting factor is on the legacy PSTN side.

Also what I am asking is why webRTC cannot simply attempt opening the
device at a supported rate? Surely 16Khz would be a good starting
point?

Would that be a simple fix versus supporting 8Khz directly and what
kind of expectation could we have for an ETA on resolving this? Our
application is stalled because clearly one cannot get users to mess
with Audio Driver Settings.

A.

[ak]

PS: we love webRTC

[Henrik Andreasson]

I am unable to dive deeper into the details on this topic at this stage but it is evident that the input audio issue on OS X 10.x are well known. See the links below for more details.

http://forums.macrumors.com/showthread.php?t=1312982

https://discussions.apple.com/thread/2634364?start=0&tstart=0 

With the current WebRTC design; it is up to the user to set a suitable, and supported, rate for input and output.

[Ivan Vučica]

ak,

can you take a look at /Library/Preferences/Audio and see if files in there are writable? Do they get updated when you edit the settings in Audio MIDI Setup? (These files are mentioned in the second link posted by Henrik).

Alternatively, try moving them somewhere else to see if the problems appear again.

The threads Henrik linked to make it sound as if the files might have wrong permissions after an upgrade from SL to Lion. Your computer might be always reverting to settings you last had under Snow Leopard.

[ak]

Henrik

Thank you for those validating references. I am sure you would agree
its a non starter to get users involved in Audio Sampling Rate issues.
Let alone that the INIT error in webRTC goes unreported.

Could we lean on your generosity and expertise to simply allow webRTC
to accept 8Khz Input.

That is what G711 reduces to anyway.

Parfavor

On Apr 24, 1:26 am, Henrik Andreasson <henr...@webrtc.org> wrote:
> I am unable to dive deeper into the details on this topic at this stage but
> it is evident that the input audio issue on OS X 10.x are well known. See
> the links below for more details.
>

> http://forums.macrumors.com/showthread.php?t=1312982https://discussions.apple.com/thread/2634364?start=0&tstart=0

[ak]

Ivan

Thanks for chiming in here. The first file does not exist. Removed the
second and third files to no effect. Lion simply recreates them and
still defaults to 8Khz. Seems absurd that such a small glitch is
tripping up such a magnanimous project.

Delete these three files:
/Library/Preferences/com.apple.soundpref.plist
/Library/Preferences/Audio/com.apple.audio.DeviceSettings.plist
/Library/Preferences/Audio/com.apple.audio.SystemSettings.plist

On Apr 24, 5:39 am, Ivan Vučica <ivuc...@gmail.com> wrote:
> ak,
>
> can you take a look at /Library/Preferences/Audio and see if files in there
> are writable? Do they get updated when you edit the settings in Audio MIDI
> Setup? (These files are mentioned in the second link posted by Henrik).
>
> Alternatively, try moving them somewhere else to see if the problems appear
> again.
>
> The threads Henrik linked to make it sound as if the files might have wrong
> permissions after an upgrade from SL to Lion. Your computer might be always
> reverting to settings you last had under Snow Leopard.
>
>
>
>
>
>
>
>
>
> On Tue, Apr 24, 2012 at 10:26, Henrik Andreasson <henr...@webrtc.org> wrote:
> > I am unable to dive deeper into the details on this topic at this stage
> > but it is evident that the input audio issue on OS X 10.x are well known.
> > See the links below for more details.
>
> >http://forums.macrumors.com/showthread.php?t=1312982
> >https://discussions.apple.com/thread/2634364?start=0&tstart=0
>
> > With the current WebRTC design; it is up to the user to set a suitable,
> > and supported, rate for input and output.
>

> Ivan Vučica - i...@vucica.net

[ak]

Henrik

Appreciate your commenting on the fact that webRTC

1. supports the G711 (8Khz) codec and yet
2. fails to support an Audio Input Device set to that same 8Khz
sampling?

> > > > > > > For playout we support
> > > > > > > {96000, 44100,48000} and for recording we support
> > > > > > > {96000,48000,44100,32000,16000}.

Is there any technical reason that precludes webRTC from supporting
8Khz recording?

A.

On Apr 24, 10:48 am, ak <starlo...@gmail.com> wrote:
> Henrik
>
> Thank you for those validating references. I am sure you would agree
> its a non starter to get users involved in Audio Sampling Rate issues.
> Let alone that the INIT error in webRTC goes unreported.
>
> Could we lean on your generosity and expertise to simply allow webRTC
> to accept8KhzInput.
>
> That is what G711 reduces to anyway.
>
> Parfavor
>
> On Apr 24, 1:26 am, Henrik Andreasson <henr...@webrtc.org> wrote:
>
>
>
>
>
>
>
> > I am unable to dive deeper into the details on this topic at this stage but
> > it is evident that the input audio issue on OS X 10.x are well known. See
> > the links below for more details.
>

> >http://forums.macrumors.com/showthread.php?t=1312982https://discussio...

>
> > With the current WebRTC design; it is up to the user to set a suitable, and
> > supported, rate for input and output.
>
> > On Tue, Apr 24, 2012 at 1:23 AM, ak <starlo...@gmail.com> wrote:
> > > Thank you for your prompt response and audio tut.
>
> > > What other option is there for G711 (8Khz) when interworking with the
> > > PSTN? Practically no gateway out there suports broader band on the
> > > interconnect and transcoding from iSAC still results in lower quality
> > > audio since the limiting factor is on the legacy PSTN side.
>
> > > Also what I am asking is why webRTC cannot simply attempt opening the
> > > device at a supported rate? Surely 16Khz would be a good starting
> > > point?
>

> > > Would that be a simple fix versus supporting8Khzdirectly and what

> > > > > Lastly while8Khzwould degrade performance using G711 is at the lower

> > > > > sampling spectrum anyway and right now there is no performance :)
>
> > > > The default codec selected in WebRTC is iSAC 16kHz.  There is a clear QoS
> > > > improvement between iSAC and G.711. Your perceived QoS will be

> > > > reduced substantially if8kHzis used as capture rate.

>
> > > > > A.
>
> > > > > On Apr 23, 12:03 am, Henrik Andreasson <henr...@webrtc.org> wrote:
> > > > > > I am a bit confused here. Are you saying that the format you set for
> > > the
> > > > > > microphone is overwritten by the OS at each restart? Does the same
> > > thing
> > > > > > happen for a USB headset?
>
> > > > > > WebRTC requires at least  16kHz sample rate to support e.g. iSAC
> > > which
> > > > > is a

> > > > > > wideband (fs=16kHz) speech codec. Allowing8kHzwill reduce the

> > > > > performance
> > > > > > substantially.
>
> > > > > > On Mon, Apr 23, 2012 at 12:21 AM, ak <starlo...@gmail.com> wrote:
> > > > > > > FYI Issue Tracker #437:
>
> > > > > > > Currently we fail to initialize if either the playout or recording
> > > > > > > side uses a non-supported sampling rate. For playout we support
> > > > > > > {96000, 44100,48000} and for recording we support
> > > > > > > {96000,48000,44100,32000,16000}.
>
> > > > > > > On Apr 22, 3:15 pm, ak <starlo...@gmail.com> wrote:
> > > > > > > > Hi
>
> > > > > > > > Mac OS X Lion appears to default the built in audio device
> > > sampling
> > > > > to

> > > > > > > >8Khzand this breaks the peerconnection factory. Even if one

> > > manually
> > > > > > > > sets the sampling to a supported rate (eg: 44.1Khz) on
> > > restarting the
> > > > > > > > Mac the system resets it back to8Khz.
>
> > > > > > > > There are no other applications accessing audio on System
> > > Restart.
>

> > > > > > > > Is there any reason8KHzsampling rate is not included in your

[Henrik Andreasson]

There is no strong technical reason not to support 8kHz input sampling rate and I will ensure that it is added when other more important tasks are finalized. However, it does "cripple" the performance since you exclude all usage of proper wideband codecs and it might also be difficult for a user to figure out that he/she is actually missing the chance of experiencing wideband. But yes, I do agree that things should not crash when 8kHz is used; that is not acceptable.

I have tried WebRTC in Chrome on 10.7 machines as well but was not able to find any that was locked down to 8kHz input rate.

Until a patch has landed, I would recommend that you try to resolve the issue on your side since it can't be an exepected behavior to ignore user settings. As a bonus, you will be able to enjoy much better audio than plain old G.711. 

[ak]

Henrik

Thank you for your prompt response.
Can you qualify: "As a bonus, you will be able to enjoy much better

audio than plain old G.711"

How exactly does one enjoy better audio when interconnecting with the
PSTN which is limited to 8Khz audio? WebRTC browser to browser is not
the only use case.

[ak]

On reading your comments again it seems you are saying it is in the
interest of the user to set a higher sampling rate to enjoy webRTC in
its broader context.

My experience is getting users to understand let alone modify such
technicalities is a non starter.

Surely WebRTC can attempt to address Audio at an acceptable rate
(44.1Khz) rather than simply accept the current rate set and take
remedial action (step down the list) if that fails which is highly
unlikely.

[Henrik Andreasson]

I meant that a typical WebRTC user who does not use G.711 will benefit from the higher bandwidth.

If PSTN is involved, there is not much we can do of course.

[Henrik Andreasson]

Windows (Vista and above) and Max OS X allows the user to set a preferred sample rate as "native" rate. Are you saying that we should simply ignore this setting and assume that the user did not know what he was doing when he/she selected a certain rate? If so, what other types of user settings should be ignored; mute, volume levels etc.? If a user has asked for 8kHz as input rate, then the best rate to work with is 8kHz. There is not much we can do about that and the user will have to live with the consequences. On some platforms, it is possible to use an alternative sample rate than the native rate but that has other negative effects.

Again, WebRTC shall work for 8kHz on all platforms but it will not function at its full capacity.

[ak]

The average user would not even know less care about a "sampling
rate". In all the years we (as a somewhat skilled user group) have
*never* set such native rate nor have we ever been asked to except
here.

"Welcome to WebRTC ... check your native input and output audio
sampling rate?"

Surely not :)

> ...
>
> read more »

[ak]

"If a user has asked for 8kHz as input rate, then the best rate to
work with is 8kHz."

We eagerly await supporting this view :)

> ...
>
> read more »

[ak]

Henrik

When last did you set Audio paramaters in Skype? Every other comms
software we have used addresses the Audio transparently as it requires
and especially as webRTC is native is should be able to do the same
without user intervention.

> ...
>
> read more »

[Don Park]

What users really want are results, not explanations or knowledge.

Why not just add a magical kick button that tumbles internal options?

- Don

[ak]

100

"make the tech invisible". who wants to train the masses.

> ...
>
> read more »

[Ivan Vučica]

Personally, if YOU believe that minimum optimal functionality for WebRTC is achieved with 16kHz, then the default MINIMUM should be 16kHz, and overridable somewhere deep, deep inside browser options. :-)

Unless the user specifically said something about 8kHz (and probably they did not; the OS or the sound card picked it for some reason), then the browser can probably know better than the user. Just a few days ago someone complained about not being able to persist the system setting to something more than 8kHz after restart on OS X, despite that person knowing what and where they had to change.

Just saying. :-)

[ak]

Ivan

Precisely. Henrik has agreed to ensure that 8khz is to be included in
the webRTC supported Audio sampling rates. Hope soonest so can move on
from this.

We encouraging our users to "upgrade to chrome" and challenging enough
getting users to enable the "media flag" (and soon to be extra peer
flag)

Any reason one cannot script link to "chrome://flags#MediaStream" so
user only has to click?

On May 5, 12:47 am, Ivan Vučica <ivuc...@gmail.com> wrote:
> Personally, if YOU believe that minimum optimal functionality for WebRTC is
> achieved with 16kHz, then the default MINIMUM should be 16kHz, and
> overridable somewhere deep, deep inside browser options. :-)
>
> Unless the user specifically said something about 8kHz (and probably they
> did not; the OS or the sound card picked it for some reason), then the
> browser can probably know better than the user. Just a few days ago someone
> complained about not being able to persist the system setting to something
> more than 8kHz after restart on OS X, despite that person knowing what and
> where they had to change.
>
> Just saying. :-)
>

> On Fri, May 4, 2012 at 9:19 PM, Henrik Andreasson <henr...@webrtc.org>wrote:
>
>
>
>
>
>
>
> > Windows (Vista and above) and Max OS X allows the user to set a preferred
> > sample rate as "native" rate. Are you saying that we should simply ignore
> > this setting and assume that the user did not know what he was doing when
> > he/she selected a certain rate? If so, what other types of user settings
> > should be ignored; mute, volume levels etc.? If a user has asked for 8kHz
> > as input rate, then the best rate to work with is 8kHz. There is not much
> > we can do about that and the user will have to live with the consequences.
> > On some platforms, it is possible to use an alternative sample rate than
> > the native rate but that has other negative effects.
>
> > Again, WebRTC shall work for 8kHz on all platforms but it will not
> > function at its full capacity.
>

> ...
>
> read more »

[PhistucK]

There are no anchors in about:flags, you can file a feature request for that at crbug.com.

Soon you would only need to enable one flag - the PeerConnection one, because MediaStream will be enabled by default (I think it will enabled by default pretty soon on the canary channel - see http://codereview.chromium.org/10379003/).

☆PhistucK

[ak]

thanks for the info ...

is there any reason flags need to be enabled at all given that permissions are required before access? 

is it simply to protect the browser from the "experiment" until stable?

also is there anyway to javascript link to "chrome:/flags" so user does not have to manually type that address?

[PhistucK]

These features are currently in development. That means they can be pretty crashy and may change significantly before getting into some kind of a beta phase.

They can also affect other, currently working, features badly since not all of the loose ends and connections are quite implemented, or may compromise security.

Features generally start out behind a compile time flag, then behind a run time flag and after working out the big kinks, they get to be enabled by default.

Using regular JavaScript, no, but you can do so using Chrome extensions (chrome.tabs.create({url:"chrome:flags"});), I believe.

☆PhistucK

[Henrik Andreasson]

Support for 8kHz input sample rate has now been added to Chrome. Should land in Canary in one or two days.

I still recommend that the core issue is resolved as well. It should be possible for a user to modify the input sample rate and make it stick after restart. Even if it is possible, it might not always be the best solution to override all user settings just to make things work.

Thanks for all the feedback on this topic.

And finally, please spin of a new thread since the current discussion does not match the title of this thread.

[ak]

Thank you Henrik.

And yes on resolving the non-stick issue. Have a tech support ticket with Apple engineering and they finding it challenging so ...

PS: notice that webRTC has quietly appeared in Chrome Stable channel (ver 18/+) ... any reason no blog about it?

On Monday, May 7, 2012 3:55:56 AM UTC-7, Henrik Andreassson wrote:

Support for 8kHz input sample rate has now been added to Chrome. Should land in Canary in one or two days.

I still recommend that the core issue is resolved as well. It should be possible for a user to modify the input sample rate and make it stick after restart. Even if it is possible, it might not always be the best solution to override all user settings just to make things work.

Thanks for all the feedback on this topic.

And finally, please spin of a new thread since the current discussion does not match the title of this thread.