iOS Bluetooth, 8KHz sample rate

[Nuno Jun]

Hi,

With the latest version of webrtc, I just found that sound gets distorted when BT devices (at 8KHz sample rate) are connected. Below are the related logs.

When Initializing PeerConnection:

(audio_device_ios.mm:101): VerifyAudioSession

(audio_device_ios.mm:530): AudioDeviceIOS::RegisterNotificationObservers

(audio_device_ios.mm:670): AudioDeviceIOS::SetupAudioBuffersForActiveAudioSession

(audio_device_ios.mm:673):  sample rate: 8000

(audio_device_ios.mm:674):  IO buffer duration: 0.008

(audio_device_ios.mm:675):  output channels: 1

(audio_device_ios.mm:676):  input channels: 1

(audio_device_ios.mm:677):  output latency: 0.01

(audio_device_ios.mm:678):  input latency: 0.01

(audio_device_ios.mm:685): Unable to set the preferred sample rate

(audio_device_ios.mm:699):  frames per I/O buffer: 64

(audio_device_ios.mm:701):  bytes per I/O buffer: 128

(audio_device_ios.mm:517): AudioDeviceIOS::UpdateAudioDevicebuffer

(audio_device_ios.mm:721):  required playout buffer size: 288

(audio_device_ios.mm:740): AudioDeviceIOS::SetupAndInitializeVoiceProcessingAudioUnit

(audio_device_ios.mm:907): Voice-Processing I/O unit is now initialized

(audio_device_ios.mm:390): AudioDeviceIOS::StartPlayout

(audio_device_ios.mm:402): Voice-Processing I/O audio unit is now started

When RTC communication starts:

webrtcvoiceengine.cc:630): ApplyOptions: AudioOptions {audio_jitter_buffer_max_packets: 50, audio_jitter_buffer_fast_accelerate: false, }

(webrtcvoiceengine.cc:647): Always disable AEC and AGC on iOS. Use built-in instead.

(audio_device_generic.cc:51): BuiltInAECIsAvailable: Not supported on this platform

(webrtcvoiceengine.cc:702): Echo control set to 0 with mode 2

(audio_device_generic.cc:66): BuiltInAGCIsAvailable: Not supported on this platform

(webrtcvoiceengine.cc:737): Auto gain set to 0 with mode 4

(webrtcvoiceengine.cc:807): NetEq capacity is 50

(webrtcvoiceengine.cc:815): NetEq fast mode? 0

(webrtcvoiceengine.cc:823): Typing detection is enabled? 0

(voe_audio_processing_impl.cc:956): SetTypingDetectionStatus: not supported

(webrtcvoiceengine.cc:827): SetTypingDetectionStatus(0) failed, err=8003

(webrtcvoiceengine.cc:851): Delay agnostic aec is enabled? 0

(webrtcvoiceengine.cc:860): Extended filter aec is enabled? 0

(webrtcvoiceengine.cc:869): Experimental ns is enabled? 0

(audio_device_ios.mm:374): AudioDeviceIOS::InitRecording

(audio_device_ios.mm:423): AudioDeviceIOS::StartRecording

(channel.cc:1415): Changing voice state, recv=1 send=1

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

(audio_device_ios.mm:1047): OnRecordedDataIsAvailable: AudioOutputUnitStart failed: -50

(audio_device_ios.mm:1036): in_number_frames (512) != 64

Is there anyone who has same issue as above?

[Henrik Andreasson]

Seems related to https://bugs.chromium.org/p/webrtc/issues/detail?id=5058.

I had marked it as fixed but feel free to add new details about your issue.

Please add information about device, OS version, BT model etc.

Also, explain the exact repro steps.

Thanks.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/de7c2026-e8c4-4eec-81ce-21145513fac6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nuno Jun]

Thanks for your quick response. I've refered to this issue (https://groups.google.com/forum/#!searchin/discuss-webrtc/iOS$20bluetooth/discuss-webrtc/sJzSe_yoF48/axfW0WODCAAJ) and tested with the bug fix version as you suggested but got exactly same result.

I've tested with iPad2, iPhone 6+ and iPhone 6s+ and tested with both iOS 9.1 and 9.2.

I guess it is related to the sample rate. Only difference is my BT devices are all 8KHz sample rate not 16KHz. BT devices that I tested are as below:

• Harman Kardon Esquire
• Sony SBH20

Above devices are not bluetooth headset but bluetooth speakers with microphone. Hope this is helpful to solve.

2015년 12월 11일 금요일 오후 5시 57분 49초 UTC+9, Henrik Andreassson 님의 말:

[Henrik Andreasson]

New type of BT devices; not yet tested. I will order devices and try to resolve the issue.

Please star https://bugs.chromium.org/p/webrtc/issues/detail?id=5058 if you want to follow the development.

And add any details in that issue if possible. Thanks.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/eced0ae9-5e07-4c69-b3bb-480198fe7f93%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Henrik Andreasson | Software Engineer | hen...@google.com | +46 70 376 91 74

[Henrik Andreasson]

Tried the Sony SBH20 device on iPhone 6s (iOS 9.2).

Using latest AppRTCDemo worked well and I was able to open up both sides at 16kHz.

I  did also verify that the microphone on the portable device was used.

Hence I am currently unable to reproduce this issue.

Do you switch device during a call? I connect the SBH20 device before starting the call in my test.

[Faraz Khan]

I'm experiencing something quite similar with M48 which has your patch Henrik. 

I'm using these: http://www.amazon.com/JayBird-BlueBuds-Sport-Bluetooth-Headphones/dp/B00AIRUOI8

Logs from audio_device_ios.mm: 

2015-12-11 19:42:15.110 (audio_device_generic.cc:51): virtual bool webrtc::AudioDeviceGeneric::BuiltInAECIsAvailable() const: Not supported on this platform

2015-12-11 19:42:15.117 (webrtcvoiceengine.cc:702): Echo control set to 0 with mode 2

2015-12-11 19:42:15.117 (audio_device_generic.cc:66): virtual bool webrtc::AudioDeviceGeneric::BuiltInAGCIsAvailable() const: Not supported on this platform

2015-12-11 19:42:15.117 (webrtcvoiceengine.cc:737): Auto gain set to 0 with mode 4

2015-12-11 19:42:15.117 (audio_device_generic.cc:76): virtual bool webrtc::AudioDeviceGeneric::BuiltInNSIsAvailable() const: Not supported on this platform

2015-12-11 19:42:15.117 (webrtcvoiceengine.cc:784): Noise suppression set to 1 with mode 5

2015-12-11 19:42:15.123 (webrtcvoiceengine.cc:790): High pass filter enabled? 1

2015-12-11 19:42:15.123 (webrtcvoiceengine.cc:807): NetEq capacity is 50

2015-12-11 19:42:15.123 (webrtcvoiceengine.cc:815): NetEq fast mode? 0

2015-12-11 19:42:15.123 (webrtcvoiceengine.cc:823): Typing detection is enabled? 0

2015-12-11 19:42:15.124 (webrtcvoiceengine.cc:1223): webrtc: (voe_audio_processing_impl.cc:959): virtual int webrtc::VoEAudioProcessingImpl::SetTypingDetectionStatus(bool): not supported

2015-12-11 19:42:15.127 (webrtcvoiceengine.cc:827): SetTypingDetectionStatus(0) failed, err=8003

2015-12-11 19:42:15.127 (webrtcvoiceengine.cc:851): Delay agnostic aec is enabled? 1

2015-12-11 19:42:15.127 (webrtcvoiceengine.cc:860): Extended filter aec is enabled? 0

2015-12-11 19:42:15.127 (webrtcvoiceengine.cc:869): Experimental ns is enabled? 0

2015-12-11 19:42:15.127 (audio_device_ios.mm:271): AudioDeviceIOS::InitRecording

2015-12-11 19:42:15.127 (audio_device_ios.mm:784): AudioDeviceIOS::InitPlayOrRecord

2015-12-11 19:42:15.128 (audio_device_ios.mm:84): ActivateAudioSession(1)

2015-12-11 19:42:17.039 (audio_device_ios.mm:423): AudioDeviceIOS::RegisterNotificationObservers

2015-12-11 19:42:17.039 (audio_device_ios.mm:563): AudioDeviceIOS::SetupAudioBuffersForActiveAudioSession

2015-12-11 19:42:17.040 (audio_device_ios.mm:566):  sample rate: 8000

2015-12-11 19:42:17.040 (audio_device_ios.mm:567):  IO buffer duration: 0.008

2015-12-11 19:42:17.040 (audio_device_ios.mm:568):  output channels: 1

2015-12-11 19:42:17.042 (audio_device_ios.mm:569):  input channels: 1

2015-12-11 19:42:17.043 (audio_device_ios.mm:570):  output latency: 0.01

2015-12-11 19:42:17.047 (audio_device_ios.mm:571):  input latency: 0.01

2015-12-11 19:42:17.047 (audio_device_ios.mm:578): Unable to set the preferred sample rate

2015-12-11 19:42:17.062 (audio_device_ios.mm:592):  frames per I/O buffer: 64

2015-12-11 19:42:17.062 (audio_device_ios.mm:594):  bytes per I/O buffer: 128

2015-12-11 19:42:17.063 (audio_device_ios.mm:410): AudioDeviceIOS::UpdateAudioDevicebuffer

2015-12-11 19:42:17.063 (audio_device_ios.mm:614):  required playout buffer size: 288

2015-12-11 19:42:17.063 (audio_device_ios.mm:633): AudioDeviceIOS::SetupAndInitializeVoiceProcessingAudioUnit

2015-12-11 19:42:17.071 (audio_device_ios.mm:169): LogABSD

2015-12-11 19:42:17.071 (audio_device_ios.mm:170):  sample rate: 8000

2015-12-11 19:42:17.074 (audio_device_ios.mm:171):  format ID: lpcm

2015-12-11 19:42:17.074 (audio_device_ios.mm:172):  format flags: c

2015-12-11 19:42:17.074 (audio_device_ios.mm:173):  bytes per packet: 2

2015-12-11 19:42:17.074 (audio_device_ios.mm:174):  frames per packet: 1

2015-12-11 19:42:17.074 (audio_device_ios.mm:175):  bytes per frame: 2

2015-12-11 19:42:17.075 (audio_device_ios.mm:176):  channels per packet: 1

2015-12-11 19:42:17.075 (audio_device_ios.mm:177):  bits per channel: 16

2015-12-11 19:42:17.078 (audio_device_ios.mm:178):  reserved: 0

2015-12-11 19:42:17.084 (audio_device_ios.mm:318): AudioDeviceIOS::StartRecording

2015-12-11 19:42:18.645 (channel.cc:1432): Changing voice state, recv=1 send=1

2015-12-11 19:42:18.706 webrtc: (audio_device_ios.mm:859): in_number_frames (512) != 64

2015-12-11 19:42:18.706 webrtc: (audio_device_ios.mm:870): OSStatus webrtc::AudioDeviceIOS::OnRecordedDataIsAvailable(AudioUnitRenderActionFlags *, const AudioTimeStamp *, UInt32, UInt32): AudioOutputUnitStart failed: -50

2015-12-11 19:42:18.770 webrtc: (audio_device_ios.mm:859): in_number_frames (512) != 64

2015-12-11 19:42:18.771 webrtc: (audio_device_ios.mm:870): OSStatus webrtc::AudioDeviceIOS::OnRecordedDataIsAvailable(AudioUnitRenderActionFlags *, const AudioTimeStamp *, UInt32, UInt32): AudioOutputUnitStart failed: -50

2015-12-11 19:42:18.774 webrtc: (webrtcvoiceengine.cc:1223): webrtc: GetRTPStatistics() failed to read RTP statistics from the RTP/RTCP module (error=9028)

2015-12-11 19:42:18.835 webrtc: (audio_device_ios.mm:859): in_number_frames (512) != 64

2015-12-11 19:42:18.835 webrtc: (audio_device_ios.mm:870): OSStatus webrtc::AudioDeviceIOS::OnRecordedDataIsAvailable(AudioUnitRenderActionFlags *, const AudioTimeStamp *, UInt32, UInt32): AudioOutputUnitStart failed: -50

2015-12-11 19:42:18.882 webrtc: (webrtcvoiceengine.cc:1223): webrtc: GetRTPStatistics() failed to read RTP statistics from the RTP/RTCP module (error=9028)

2015-12-11 19:42:18.899 webrtc: (audio_device_ios.mm:859): in_number_frames (512) != 64

2015-12-11 19:42:18.899 webrtc: (audio_device_ios.mm:870): OSStatus 

[Faraz Khan]

Also - maybe this is helpful. But after attaching such a device this happens when I try to shutdown the peerconnection:

2015-12-11 19:51:41.213 Slack[587:280332] 19:51:41.213 ERROR:    [pcfactoryworker] AVAudioSession.mm:697: -[AVAudioSession setActive:withOptions:error:]: Deactivating an audio session that has running I/O. All I/O should be stopped or paused prior to deactivating the audio session.

#

# Fatal error in ../../webrtc/modules/audio_device/ios/audio_device_ios.mm, line 92

# Check failed: CheckAndLogError(success, error)

# 

On Friday, December 11, 2015 at 7:09:46 AM UTC-8, Henrik Andreassson wrote:

[Faraz Khan]

Henrik, 

Some interesting findings! I made a comment under https://bugs.chromium.org/p/webrtc/issues/detail?id=5058

In short - It works fine if you 1. don't use AVAudioPlayer and 2. Never switch to speakerphone.

On Friday, December 11, 2015 at 7:09:46 AM UTC-8, Henrik Andreassson wrote:

[Nuno Jun]

I will test with the latest version of webrtc once again and get back to you. But it is little odd that Sony SBH20 worked at 16kHz. When i tested most BT devices worked at 8kHz and produced error that I reported earlier. Thanks for your quick response again.

2015년 12월 12일 토요일 오전 12시 9분 46초 UTC+9, Henrik Andreassson 님의 말:

[Nuno Jun]

Thanks for your suggestion. It turned out that this issue was exactly related to the issue 5058!! Thanks Henrik and Faraz!

2015년 12월 12일 토요일 오후 2시 16분 30초 UTC+9, Faraz Khan 님의 말:

...

[Henrik Andreasson]

We are doing some more long term work where the goal is to break out the audio session parts and ensure that the user (or app developer) gets more control. However, it is not a straightforward task due to the delicate balance between what's best for the app and what's best for the real-time media flow. Just FYI that changes are on its way in that area.

Regarding BT on iOS in the existing implementation, I'll try to extend the tests and buy more devices but I am a bit surprised that users end up at 8kHz on the recording side; something I fail to reproduce. IMHO, a device that falls back to 8kHz should not be used for VoIP but we should of course support it (and don't generate bad audio).

In any case, best way forward would be if those of you who still have issues try to reproduce using AppRTCDem or provide patches to it with additional parts (e.g. usage of AVAudioPlayer) required to trigger problems.

Thanks for helping out improving our iOS audio stack.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/19a69a76-8e50-4d17-b452-4e208adb4682%40googlegroups.com.

[faraz khan]

Henrik,
Thanks for looking into this! Isn't 8khz the only sample rate that's supported with the Bluetooth hfp (hands free profile) when both the speaker and the mic are active? Are there devices that do higher sample rates?

Also are there are instructions on getting apprtdemo deployed on the iPhone? I've tried with ninja/ios-deploy but its a mess (code signing/provisioning profile problems). How do you guys generally do this?

You received this message because you are subscribed to a topic in the Google Groups "discuss-webrtc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/discuss-webrtc/8xu1ghmDmQM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CAD2mqc7yj3kgkSZcbLxrUEeLWDpm_EyiyFsXPc0%3DKHRAKhJePQ%40mail.gmail.com.

[faraz khan]

I stand corrected by myself - apparently hfp v1.6 added 16khz sampling ability. I'll test with a v1.6 headset and let you know if the problem exists.

[Henrik Andreasson]

I have tried four different BT devices but never been able to provoke usage of 8kHz on the capture side.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CALBWJdXdTJmzTxqTUeJ1_cQb24Fr5AnVSXHUPOt3Af3vBDmqjg%40mail.gmail.com.