PSA: WebRTC M57 Release Notes

[Anatoli Davidson]

M57

WebRTC M57 branch (cut at r16123)

Summary

WebRTC M57, currently available in Chrome's beta channel and as native libraries for Android and iOS, contains over 40 new features and over 60 bug fixes, enhancements and stability/performance improvements. As with previous releases, we encourage all developers to run versions of Chrome on the Canary, Dev, and Beta channels frequently and quickly report any issues found. Please take a look at this page, for some pointers on how to file a good bug report. The help we have received has been invaluable!

The Chrome release schedule can be found here.

Important PSAs

Default RTCRtpMuxPolicy is now "require"

The default RTCRtpMuxPolicy was changed from “negotiate” to “require”, matching the standard. This means that by default, offer/answer negotiation with an endpoint that doesn't support RTCP multiplexing will fail. If your application doesn’t support RTCP multiplexing, you can get the old behavior by explicitly setting the policy to “negotiate”, though it may be deprecated in a later release. See the public PSA for more details.

Changes in error reporting in getUserMedia

getUserMedia will now report success/failure after access to the underlying audio capture device has completed, and not before (see issue 679210). This may result in slightly longer times for getUserMedia to complete but the fix addresses a situation where an audio track could be returned that represented an inactive or non-existing audio source.

Deprecating legacy VoE APIs

In September 2015 we announced that the low-level VoiceEngine APIs (VoEBase, VoENetwork, etc) were going to be deprecated. Most usage of these legacy APIs inside of the WebRTC native library has now been replaced.

M57 is the last release to feature the legacy VoE APIs and we will start to remove them shortly from ToT. We know there are a few users who may be affected by this change, but this is a step we need to take in order to maintain velocity in evolving other parts of the stack. As these APIs were never officially supported, there is unfortunately no exact replacement for them. To build an app that can take advantage of improvements to WebRTC without running into compatibility issues, the PeerConnection API (in the webrtc/api/ folder) should be used.

Features

Echo detection stats available in chrome://webrtc-internals

A new audio processing component was added that can detect echo, which can be used to detect cases where either the hardware or software AEC is not functioning properly. The echo likelihood is represented as a number between 0 (no echo) and 1 (echo highly likely), where values higher than 0.5 indicate echo problems. The value of the stat can be seen on chrome://webrtc-internals, and it can be queried using GetStats (googResidualEchoLikelihood for the instantaneous value, and googResidualEchoLikelihoodRecentMax for the maximum value of the previous 10 seconds). This feature has been merged to the M56 branch after the cut, so it is available there as well. See 6525 and 6797 for more details.

H.264 VideoToolbox hardware encoder on Mac

Native applications on macOS can now take advantage of hardware H.264 encode/decode support out-of-the-box.

Support for Bluetooth devices in AppRTCMobile

We have added support for BT audio in our example application allowing the user to enumerate connected BT devices and also select them as full-duplex audio devices. The test implementation supports the headset profile but not yet A2DP. See 6649 for more details.

DTMF supported for multiple rates

DTMF, a.k.a. "telephone-event" is now offered for timestamp rates matching the set of supported audio codecs.

Enable HW H.264 encoding by default in Chrome on Android

Chrome for Android now supports H.264 hardware encoding for WebRTC, in addition to the previously-released support for H.264 hardware decoding, for Qualcomm (Kitkat and later) and Exynos (Lollipop and later) chipsets. As there is no software encoder implementation, only the aforementioned chipsets are supported; a few particular devices have been blacklisted due to poor encoder performance. Note that H.264 still needs to be negotiated in SDP for this change to have any effect.

Deprecations

Platform	Issue	Description	Component
Chrome	6506	Remove voice_engine_configurations.h	Audio
Chrome	6796	Deprecating old CongestionController::Observer::OnNetworkChanged(...)	Network
Chrome	6983	Remove WebRtcVideoSendStream2::VideoSink inheritance.	Video
Chrome	6870	getStats: remove googViewLimitedResolution	Video
Chrome	653994	Remove gpu::SyncToken usage from video capture pipeline	Blink>GetUserMedia>Webcam
Chrome	579648	Remove //media/base/mac glue now that Chromium is 10.9+	Blink>GetUserMedia>Webcam
Mobile (Android)	664549	Consider removing Tango Chrome video capture support	Blink>GetUserMedia>Webcam
Chrome	6829	Remove base/dbus.h,.cc Remove base/libdbusglibsymboltable.h Remove base/linux.h Remove base/linuxfdwalk.h	PeerConnection

Features and Bugfixes

Chrome

Type	Issue	Description	Component
Feature	6525	Implement residual echo detector	Audio
Feature	6785	Move transport.h to webrtc/api and switch the dependencies to the new place.	Audio
Feature	6346	Rewrite of Audio Conference Mixer	Audio
Feature	6797	Tracking bug for improvements to echo detection stats	Audio
Feature	2795	telephone-event should be offered at 48K (and ideally all rates)	Audio
Feature	5006	Reduce WebRTC checkout size.	Build
Feature	6646	Support rotation in DirectX capturer	DesktopCapture
Feature	6513	Share differ logic between ScreenCapturer and WindowCapturer	DesktopCapture
Feature	6864	Don't allow ICE pool size to change after SetLocalDescription	Network
Feature	6840	Allow disabling TLS certificate checking for TURNS in PeerConnection API	PeerConnection
Feature	6933	No way to build without SCTP easily.	PeerConnection
Feature	6875	Improve threading of new GetStats API (expose TransportController::GetStats_n)	Stats
Feature	6578	Add histogram stats for H264 encoder QP	Video
Feature	6495	Move QualityScaler to ViEEncoder.	Video
Feature	6300	Expose frame logging from the native API	Video
Feature	6897	VP8 screenshare temporal layer strategy should throttle input fps	Video
Feature	6690	New overuse filter for delay based BWE	Video
Feature	6789	Refactor VideoCaptureModule	Video
Feature	676061	Support 420mpeg2 Y4M input in FileVideoCaptureDevice	Blink>GetUserMedia
Feature	314516	Implement desktop capture using DXGI 1.2 DuplicateOutput API	Blink>GetUserMedia>Desktop
Feature	648183	Move device enumeration and notifications from MediaStreamDispatcherHost to a Mojo-based MediaDevicesDispatcherHost.	Blink>MediaStream
Feature	658428	Expose RTCDataChannel and RTCDataChannelEvent (with constructor)	Blink>WebRTC
Feature	671153	Make Chromium use the new way of specifying parameters for the high-pass filter in the audio processing module	Blink>WebRTC>Audio
Feature	669510	Logging/stats for when using ALSA in a WebRTC call	Blink>WebRTC>Audio
Feature	659286	Rename RTCIceCandidateEvent to RTCPeerConnectionIceEvent and expose	Blink>WebRTC>PeerConnection
Feature	652923	Avoid using main render thread in WebRTC render path	Blink>WebRTC>Video
Bugfix	6622	The analog gain controller does not sufficiently reduce the gain for clipped echo	Audio
Bugfix	6793	WebRtcVoiceMediaChannel should support x-google-{min,start,max}-bitrate when bandwidth estimation is used.	Audio
Bugfix	5676	Remove dependency on Opus private APIs.	Audio
Bugfix	6508	RTT is not updated for audio only calls	Audio
Bugfix	6815	MediaContraints cannot carry UTF-8 string.	Audio
Bugfix	6909	Wrong packet loss rate by Audio Network Adaptor FEC controller	Audio
Bugfix	6932	If network enumeration fails/doesn't return any networks, webrtc just gives up and doesn't allocate any ports	Network
Bugfix	6737	TimeCallback for BoringSSL appears broken	Network
Bugfix	6763	DTLS Channel errors with SSL handshake failure when running cross compiled webrtc for ARM with node-webrtc on Raspberry PI	Network
Bugfix	6030	Change default RTCP mux policy to "require"	Network
Bugfix	6966	Errors caused by "rtcpMuxPolicy" of "require" produce a confusing error message.	PeerConnection
Bugfix	6972	Having a=crypto in an SDP offer causes exception in Canary	PeerConnection
Bugfix	6917	Implement currentLocalDescription/pendingLocalDescription/etc.	PeerConnection
Bugfix	6942	Java and Obj-C PeerConnection wrappers don't merge RTCConfiguration with RTCMediaConstraints	PeerConnection
Bugfix	6916	SetConfiguration does not return any error information.	PeerConnection
Bugfix	6862	SSRC missing from RtpEncodingParameters for audio	PeerConnection
Bugfix	6888	SSRC missing from RtpEncodingParameters for video RtpSender	PeerConnection
Bugfix	6903	SSRC missing from RtpEncodingParameters in Java/Obj-C wrappers	PeerConnection
Bugfix	6752	Use AudioOptions for Peerconnection JNI method of CreateAudioSource.	PeerConnection
Bugfix	6705	Stop using hardcoded payload types for VideoCodecs	PeerConnection
Bugfix	6714	setConfiguration doesn't implement "needs-ice-restart" logic.	PeerConnection
Bugfix	6795	WebRTC H.264 screen content does not scale up from initial resolution	Video
Bugfix	6957	webrtcvideoengine2.cc may mark internal encoders as having internal-source when switching from an external encoder	Video
Bugfix	6774	Sent bitrate stats are incorrect if FlexFEC is configured	Video
Bugfix	6692	Zero rtt may be reported to and used by RtcpBandwidthObserver.	Video
Bugfix	6825	CanReceiveFlexfec fails sometimes.	Video
Bugfix	6723	Only last spatial layer is accounted for in MediaOptimization/FrameDropper	Video
Bugfix	6603	webrtc::VideoCodec should not expose an union for codec-specific information	Video
Bugfix	6860	Regression in quality downscale stats.	Video
Bugfix	6402	cricket::WebRtcVideoEncoderFactory::VideoCodec does not contain enough information for H264	Video
Bugfix	6743	Decoder codec types throws away H264 profile information	Video
Bugfix	6677	RTX codecs for different H264 profiles will end up with the same payload type	Video
Bugfix	6995	MedianSlopeFilter doesn't pick up parameters from the field trial string	Video
Bugfix	6892	Precision loss in MedianSlopeFilter if clock initialized to a very large value	Video
Bugfix	6884	Precision loss in TrendlineFilter if clock initialized to a very large value	Video
Bugfix	6848	Refactor code in videoadapter.cc	Video
Bugfix	677347	Tab Capture video freezes if tab crashes, does not restore on navigation	Blink>GetUserMedia>Tab
Bugfix	601029	tabCapture cursor rendering of carret inaccurate	Blink>GetUserMedia>Tab
Bugfix	676687	TabCapture can't get the cursor position accurate on Linux and Win	Blink>GetUserMedia>Tab
Bugfix	670944	stuttering local video with logitech C920	Blink>GetUserMedia>Webcam
Bugfix	659183	Cursor missing in tabCapture	Blink>MediaStream
Bugfix	673734	Naked values in constraints are dependent on context for interpretation	Blink>MediaStream
Bugfix	669877	Invalid negative values in stats returned by new getStats	Blink>WebRTC
Bugfix	668704	Incorrect parameters for WebRTC echo detector	Blink>WebRTC
Bugfix	668632	WebRTC Echo likelihood stat does not propagate to chrome://webrtc-internals page	Blink>WebRTC
Bugfix	653873	RTCPeerConnection.getStats: RTCTransportStats	Blink>WebRTC
Bugfix	612294	PowerSaveBlock activated even though there are no active PeerConnections	Blink>WebRTC
Bugfix	671046	video_WebRtcCamera started failing in M57	Blink>WebRTC>Video
Bugfix	653434	samus: many decode errors on apprtc loopback with H264	Blink>WebRTC>Video
Bugfix	679215	Default maximum number of audio input streams, is too low.	Blink>WebRTC>Audio
Bugfix	679210	getUserMedia does not report an error for audio sources that fail to initialize	Blink>WebRTC>Audio
Bugfix	678697	Local AudioTracks do not become 'ended' if audio source initialization fails	Blink>WebRTC>Audio
Bugfix	677550	webrtc: setRemoteDescriptionOnFailure doesn't show up webrtc-internals if SDP parsing fails	Blink>WebRTC>PeerConnection
Bugfix	665884	m_hasDataChannels field not initialized correctly in blink::RTCPeerConnection	Blink>WebRTC>PeerConnection

Native Android/iOS

Type	Issue	Description	Component
Feature	6745	Add Full HD and 4K support to AppRTCMobile Android.	Mobile (Android)
Feature	6499	Allow custom metrics implementations on Android.	Mobile (Android)
Feature	6649	Add support for Bluetooth devices in AppRTCMobile	Mobile (Android)
Feature	6470	Split out common code from SurfaceViewRenderer to RendererCommon	Mobile (Android)
Feature	6753	Add API to crop video to different aspects on send-side	Mobile (iOS), Video
Feature	6654	Add maximum bitrate setting to AppRTCMobile for iOS.	Mobile (iOS)
Feature	6827	Expose audio_jitter_buffer_fast_accelerate to objc RTCConfiguration	Mobile (iOS)
Feature	5963	Update ice server provider response format in the iOS AppRTCMobile app	Mobile (iOS)
Feature	6722	iOS: Add SendSideAudioBwe field trial	Mobile (iOS)
Feature	6902	iOS: Add trendline filter field trial	Mobile (iOS)
Feature	6680	Split AVFoundationVideoCapturer and RTCAVFoundationVideoCapturerInternal in separate files	Mobile (iOS)
Feature	2578123002	Add QP stats to the statsview in AppRTCMobile for ios.	Mobile (iOS)
Feature	664652	Enable HW H264 encoding by default on Android	Blink>WebRTC (Android)
Feature	2544563002	Make SurfaceTextureHelper and I420Frame public in Java.	Mobile (Android)
Feature	6317	H264 VideoToolbox hardware encoder on Mac	Video (macOS)
Bugfix	6660	classreferenceholder.h: "rettype JNIEXPORT JNICALL" incorrect	Mobile (Android)
Bugfix	6684	Android AppRTC demo leaks every CallActivity created.	Mobile (Android)
Bugfix	6982	Android: Do not report camera disconnected event if camera is closing	Mobile (Android)
Bugfix	6858	AppRTCMobile encoder not working.	Mobile (iOS)
Bugfix	6034	Possible wrong size used in bitrate adjustment	Mobile (iOS)
Bugfix	6925	ios video rotate	Mobile (iOS)
Bugfix	6768	Remote picture is rendered upside down in Chrome on macOS in an H264 call with Android	SampleApps (Android)
Bugfix	6546	Remove NALU parser implementation from MediaCodecVideoEncoder and use existing one	Cleanup (Android)
Bugfix	6907	Remove outdated hardware name -> device model name table	Cleanup (iOS)
Bugfix	630266	Video Quality very poor using Samsung S5	Blink>WebRTC>Video (Android)
Bugfix	669573	Handle H264 rotation switches when rendering frames in WebMediaPlayerMS	Internals>Media>Codecs (Android)

Regards

Anatoli

[Joshua Xiong]

Hi, 

I am new in video coding. I have a, maybe naiive, question here:

"Enable HW H.264 encoding by default in Chrome on Android"

If I remeber right, in M48 release note says VP9: "deliver the same high quality at 40% lower bitrate (compared to VP8 or H.264), at the cost of 15% additional CPU". So I am wondering why H264 is the default encoder for Chrome on Android?

Thanks a lot in advance.

[Alexandre GOUAILLARD]

H.264 is not the default codec, but when you choose to use H.264 it is Hardware encoded by default.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrtc+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/33204e50-b75f-487b-a904-909c2e58fb0e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Alex. Gouaillard, PhD, PhD, MBA

------------------------------------------------------------------------------------

President - CoSMo Software Consulting, Singapore

------------------------------------------------------------------------------------

sg.linkedin.com/agouaillard

• 
  

[PhistucK]

> googResidualEchoLikelihood for the instantaneous value, and googResidualEchoLikelihoodRecentMax

Were new goog-prefixed statistics added to Chrome as part of this work?

If so, will they exist in both of the getStats variants (promise and callback based)?

Obviously, I really hope the answer is "no".

☆PhistucK

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CAHgZEq6z2g_LMNwzRa6F%3DJKFN-GcKAAMCY%2Ba8HPHfQtgE-023g%40mail.gmail.com.

[Philipp Hancke]

2017-02-23 14:47 GMT+01:00 PhistucK <phis...@gmail.com>:

> googResidualEchoLikelihood for the instantaneous value, and googResidualEchoLikelihoodRecentMax

Were new goog-prefixed statistics added to Chrome as part of this work?

Yes, to the legacy variant that is full of undocumented proprietary stats. This one is documented at least.

 

If so, will they exist in both of the getStats variants (promise and callback based)?

There will be something in the spec-stats, see discussion at https://github.com/w3c/webrtc-stats/issues/153

[PhistucK]

Adding new prefixed features to an already deprecated API does not make sense (none of the parts makes sense, really)...

☆PhistucK

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrtc+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CADxkKi%2BPUpy7nkbdyj68Jo_Y8c37PukS39j0X0LDrRVWR_eLFQ%40mail.gmail.com.

[Philip Jägenstedt]

[previous reply bounced, trying again after joining group]

Thanks for pointing this out, PhistucK.

+Henrik Boström who is implementing the new stats API and who commented on https://codereview.webrtc.org/2629563003

Is there some relevant background reading to understand this property in particular and goog-prefixed ones in general? My understanding is that the same code is used for stats for all webrtc embedders (not just chromium) and that it's not always even intentional/necessary to expose some added stats to the web.

Now that the promise-vending getStats is on a path to stable in M58, is it a safe assumption that nothing (prefixed or not) will ever be added to the callback-based getStats again? If so, could the set of stats exposed to the web via the callback-based getStats be frozen to precisely the current set? If not, I'm curious to understand :)

On Thu, Feb 23, 2017 at 9:33 PM PhistucK <phis...@gmail.com> wrote:

Adding new prefixed features to an already deprecated API does not make sense (none of the parts makes sense, really)...

☆PhistucK

On Thu, Feb 23, 2017 at 4:25 PM, 'Philipp Hancke' via discuss-webrtc <discuss...@googlegroups.com> wrote:

2017-02-23 14:47 GMT+01:00 PhistucK <phis...@gmail.com>:

> googResidualEchoLikelihood for the instantaneous value, and googResidualEchoLikelihoodRecentMax

Were new goog-prefixed statistics added to Chrome as part of this work?

Yes, to the legacy variant that is full of undocumented proprietary stats. This one is documented at least.

 

If so, will they exist in both of the getStats variants (promise and callback based)?

There will be something in the spec-stats, see discussion at https://github.com/w3c/webrtc-stats/issues/153

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

[PhistucK]

Henrik, any word?

☆PhistucK

On Mon, Feb 27, 2017 at 3:05 PM, Philip Jägenstedt <foo...@google.com> wrote:

[previous reply bounced, trying again after joining group]

Thanks for pointing this out, PhistucK.

+Henrik Boström who is implementing the new stats API and who commented on https://codereview.webrtc.org/2629563003

Is there some relevant background reading to understand this property in particular and goog-prefixed ones in general? My understanding is that the same code is used for stats for all webrtc embedders (not just chromium) and that it's not always even intentional/necessary to expose some added stats to the web.

Now that the promise-vending getStats is on a path to stable in M58, is it a safe assumption that nothing (prefixed or not) will ever be added to the callback-based getStats again? If so, could the set of stats exposed to the web via the callback-based getStats be frozen to precisely the current set? If not, I'm curious to understand :)

On Thu, Feb 23, 2017 at 9:33 PM PhistucK <phis...@gmail.com> wrote:

Adding new prefixed features to an already deprecated API does not make sense (none of the parts makes sense, really)...

☆PhistucK

On Thu, Feb 23, 2017 at 4:25 PM, 'Philipp Hancke' via discuss-webrtc <discuss-webrtc@googlegroups.com> wrote:

2017-02-23 14:47 GMT+01:00 PhistucK <phis...@gmail.com>:

> googResidualEchoLikelihood for the instantaneous value, and googResidualEchoLikelihoodRecentMax

Were new goog-prefixed statistics added to Chrome as part of this work?

Yes, to the legacy variant that is full of undocumented proprietary stats. This one is documented at least.

 

If so, will they exist in both of the getStats variants (promise and callback based)?

There will be something in the spec-stats, see discussion at https://github.com/w3c/webrtc-stats/issues/153

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrtc+unsubscribe@googlegroups.com.

[samfast]

Is there any documentation for Android? Where I can see how the functions should be used? and what the functions looks like? Just Java Documentation.

[hb...@webrtc.org]

Sorry I missed this. Ideally the callback-based getStats would be completely frozen in favor of the new stats but because existing products still rely on the callback-based getStats and it has a few stats that are not in the new getStats we need to come up with a plan for migration before we can safely say that old getStats is deprecated and frozen. We shouldn't add any stats to the callback-based one that we don't plan to standardize though, but we need to figure out what to do about experimental stats.