PSA: WebRTC M66 Release Notes

[Anatoli Davidson]

WebRTC M66 Release Notes

WebRTC M66 branch (cut at r22215)

Summary

WebRTC M66, currently available in Chrome's beta channel and as native libraries for Android and iOS, contains over 10 new features and over 40 bug fixes, enhancements and stability/performance improvements. As with previous releases, we encourage all developers to run versions of Chrome on the Canary, Dev, and Beta channels frequently and quickly report any issues found. Please take a look at this page, for some pointers on how to file a good bug report. The help we have received has been invaluable!

The Chrome release schedule can be found here.

PSAs

Fix for long delays when Opus was used with DTX

A bug was found in the audio jitter buffer (NetEq) that caused the buffer to apply too long delay when a codec with internal DTX/CNG (discontinuous transmission mode with comfort noise) was used. The only codec supporting this as open-source is Opus, and it only hits when DTX is actually used. The bug would manifest itself as excessive delay for some time after each period of silence (when the codec would send CNG). The added delay got higher with bad network conditions, and the elevated delay lasted in the order of seconds. The most apparent effect is poor audio/video sync with the audio lagging after the video from time to time during a call. The bug was fixed behind a flag in the M64 branch, and is now permanently fixed in the M66 branch. Note that Chrome users will get the fix from M64, since the flag is now default enabled from that version.

Changes to RTCDTMFSender

Chrome’s previous implementation of DTMF support in WebRTC involved using the createDTMFSender() call on an RTCPeerConnection. This approach has been replaced in the spec with a “dtmf” attribute on an RTCRtpSender. See details in the original PSA.

MediaStreamTrack.getCapabilities()

A new method on the MediaStreamTrack interface called getCapabilities() returns a MediaTrackCapabilities object, which specifies the values or range of values supported for each constrianable property. Capabilities vary by device.

Deprecations

Issue	Description	Component
8768	Get rid of packet loss stuff from videoprocessor	Video
8677	Remove custom MD5 / SHA-1 implementations	Cleanup, Internals
8852	Remove deprecated RTCAVFoundationVideoSource.	Mobile
8456	Remove ThreadUtils.waitUninterruptibly	Mobile

Features and Bugfixes

Type	Issue	Description	Component
Feature	8640	Add RelayPortFactoryInterface that allows for custom relay (e.g turn) ports	Network>ICE
Feature	8906	Send application-specific bytes from RtpSender through Transport	Network>RTP
Feature	8789	Make RTCP interval configurable.	Network>RTP
Feature	8653	RTCRtpEncodingParamers.active for each simulcast stream for RTPSender	PeerConnection
Feature	8813	Verify signaling state in CreateAnswer	PeerConnection
Feature	8764	Generate stats with PeerConnection in Unified Plan semantics	PeerConnection
Feature	8800	Implement PeerConnection::GetRemoteAudioSSLCertificate	PeerConnection
Feature	8355	Additional metrics for dropped video frames	Stats, Video
Feature	8656	Make current fec controller plug-able	Video
Feature	8799	Enable PeerConnectionFactory uses external fec controller	Video
Feature	4690	VoE API Refactoring Tracking Bug	Audio
Feature	54180 54100	Add configurable STUN binding request interval and collect stats of STUN binding requests	Network>ICE
Feature	34622 34660	Structured ICE logging via RtcEventLog	Network>ICE
Bug	8488	NetEq introduces delay when codec-internal DTX is used	Audio
Bug	8862	During audio pipeline issues, delay estimation realignment causes echo leakage when AEC3 is used	Audio
Bug	8940	The WebRTC echo canceller 3 would benefit from being informed that the setup used has clock drift	Audio
Bug	8879	The AEC3 delay estimator sometime is slow at tracking audio path changes	Audio
Bug	8889	The AEC3 look window is too short to cover the very long audio buffer delays seen on some platforms	Audio
Bug	804267	The look window in AEC3 for the nonlinear mode was shortened too much	Blink>WebRTC>Audio
Bug	812524	The AEC3 look window is too short to cover the very long audio buffer delays seen on some platforms	Blink>WebRTC>Audio
Bug	811658	The AEC3 delay estimator sometime is slow at tracking audio path changes	Blink>WebRTC>Audio
Bug	805815	The new fast delay AEC3 alignment may trigger a reset cycle	Blink>WebRTC>Audio
Bug	810371	During audio pipeline issues, delay estimation realignment causes echo leakage when AEC3 is used	Blink>WebRTC>Audio
Bug	804270	AEC3 sometimes fails to cancel echoes during the first seconds in a call	Blink>WebRTC>Audio
Bug	804873	On systems with frequent audio path glitches AEC3 leaks more echoes than necessary	Blink>WebRTC>Audio
Bug	637558	Missing audio data with getUserMedia	Blink>WebRTC>Audio
Bug	8021	Software fallback doesn't trigger for wrapped media::VideoFrames	HardwareCodec
Bug	809201	Stop using unsupported POSIX APIs under OS_FUCHSIA	Internals>PlatformIntegration
Bug	8847	Adds voice concealment periods reporting to neteq_rtpplayback	Audio
Bug	8346	Make EchoController an injectable module in Audio Processing Module	Audio
Bug	8908	InsertDTMF() doesn't return an error when the channel is not connected	Audio
Bug	8784	AEC3 sometimes fails to cancel echoes during the first seconds in a call	Audio
Bug	8788	On systems with frequent audio path glitches AEC3 leaks more echoes than necessary	Audio
Bug	8783	The look window in AEC3 for the nonlinear mode was shortened too much	Audio
Bug	8798	The new fast delay AEC3 alignment may trigger a reset cycle	Audio
Bug	8782	Echo saturations makes the echo alignment less accurate in AEC3	Audio
Bug	8887	AEC3 should report when it detects issues in the external audio pipeline	Audio
Bug	8883	Before the render and capture signals have been aligned AEC3 may leak echo	Audio
Bug	8891	Native AEC code on Windows is not being exercised	Audio
Bug	8598	Analog gain not updated	Audio, Mic
Bug	804263	Echo saturations makes the echo alignment less accurate in AEC3	Blink>WebRTC>Audio
Bug	805759	networkType in RTCIceCandidateStats doesn't identify VPN interfaces	Blink>WebRTC>Network
Bug	8933	Lots of work done in rtc::LogMessage for dropped log messages	Internals
Bug	8920	Make RTCCertificateStats work for certificate chains	Network>DTLS, Stats
Bug	8826	Handle lifetime shorter than 2 minutes for turn allocations	Network>ICE
Bug	8805	Fix RTT calculation when incoming receiver report has both report block with and without fields for rtt.	Network>RTP
Bug	8766	Packets may become unretransmittable during very high bitrate usage	Network>RTP
Bug	8808	H264 codecs "match" even if packetization-mode is different.	PeerConnection
Bug	8772	RtpSenderInterface::SetParameters should fail with RTCError, not bool	PeerConnection
Bug	7600	RTCRtpTransceiver API	SpecConformance
Bug	8673	Stats are not generated before connection	Stats
Bug	7949	Reported sent bitrate higher than target bitrate	Stats
Bug	8767	RTP header can be incorrectly copied to RED packets containing FEC, when PlayoutDelay RTP header extension is used	Video
Bug	8917	Jitter buffer drops around 3% of frames even in ideal network conditions.	Video
Bug	8770	nanosleep(0) is very slow on some Android devices after app was minimizing	Video
Bug	8773	Artifacts Observed When Alpha Channel Is On	Video
Bug	8921	Multiplex Codec should consider the necessary Padding when using H264.	Video
Bug	8275	WebRTC cannot decode an SPS with scaling lists	Video
Bug	5749	googFrameWidthSent stats are incorrect for VP9	Video
Bug	812587	WebRTC RTPSender should support "dtmf" attribute	Blink>WebRTC>PeerConnection

[Dag-Inge Aas]

How will autoplay affect WebRTC usage in M66? According to
https://www.chromium.org/audio-video/autoplay autoplay will be
enforced from M66, yet there is no information about this in the
release notes (as promised).

The bug I was asked to follow
https://bugs.chromium.org/p/chromium/issues/detail?id=804091 is marked
as fixed, but it doesn't seem like this is included in the notes here.
The related code change
https://chromium.googlesource.com/chromium/src.git/+/ef28e2a3c5bceaeff46da187529966bdeb194394
is also not included as far as I can see.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "discuss-webrtc" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to discuss-webrt...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/discuss-webrtc/CACxSdkGMtgQN30RsCVtYc2%3DjUP7PYRpAJ8MA19LCUDGME3QOKQ%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.



--
Dag-Inge Aas
CTO Confrere
https://meet.confrere.com/daginge

[Huib Kleinhout]

On Wednesday, March 21, 2018 at 3:55:07 PM UTC+1, Dag-Inge Aas wrote:

How will autoplay affect WebRTC usage in M66? According to
https://www.chromium.org/audio-video/autoplay autoplay will be
enforced from M66, yet there is no information about this in the
release notes (as promised).

There is no special treatment for WebRTC based media streams so the same restrictions apply. 

We encourage you to test the new policy in M66 and file bugs if there are any further problems with the implementation.  

 

The bug I was asked to follow
https://bugs.chromium.org/p/chromium/issues/detail?id=804091 is marked
as fixed, but it doesn't seem like this is included in the notes here.
The related code change
https://chromium.googlesource.com/chromium/src.git/+/ef28e2a3c5bceaeff46da187529966bdeb194394
is also not included as far as I can see.

This bug has been fixed in Chromium code that is not owned by the WebRTC team. Those type of changes are therefore not tracked in our release notes.

Huib 

> email to discuss-webrtc+unsubscribe@googlegroups.com.

[Philipp Hancke]

2018-03-23 8:26 GMT+01:00 'Huib Kleinhout' via discuss-webrtc <discuss...@googlegroups.com>:

On Wednesday, March 21, 2018 at 3:55:07 PM UTC+1, Dag-Inge Aas wrote:

How will autoplay affect WebRTC usage in M66? According to
https://www.chromium.org/audio-video/autoplay autoplay will be
enforced from M66, yet there is no information about this in the
release notes (as promised).

There is no special treatment for WebRTC based media streams so the same restrictions apply. 

Back in November I heard you were working on a set of recommendations for autoplay. Did I miss those?

I found out about another change in behaviour broke quite some of my tests:

   https://bugs.chromium.org/p/chromium/issues/detail?id=824866

In a nutshell, if you have a muted video element (which is quite common for local media) with a MediaStream attached and you hide it, it will get paused. Now pausing WebRTC video streams does not make much sense since they are realtime...

Now I also hear that audiocontext which is used quite some sites to measure the mic level is affected by the autoplay stuff as well.

There is a nice console warning:

    The AudioContext was not allowed to start. It must be resume (or created) after a user gesture on the page. https://goo.gl/7K7WLu

and documentation even. WebRTC is not mentioned on that page. Why?

We encourage you to test the new policy in M66 and file bugs if there are any further problems with the implementation.  

In order to test  one has to know there are changes.  By not mentioning the autoplay changes in the release notes you are making this a game of blind mans buff.

I'll give you an example for how I would in the release notes:

    "There is a new autoplay policy in Chrome 66. If you run into issues, check if they are related to the autoplay policy changes by running chrome with --autoplay-policy=no-user-gesture-required and file bugs"

The lack of devrel shown is once again harmful.

[Dag-Inge Aas]

On Mon, Apr 23, 2018 at 10:13 AM, 'Philipp Hancke' via discuss-webrtc <discuss...@googlegroups.com> wrote:

2018-03-23 8:26 GMT+01:00 'Huib Kleinhout' via discuss-webrtc <discuss-webrtc@googlegroups.com>:

On Wednesday, March 21, 2018 at 3:55:07 PM UTC+1, Dag-Inge Aas wrote:

How will autoplay affect WebRTC usage in M66? According to
https://www.chromium.org/audio-video/autoplay autoplay will be
enforced from M66, yet there is no information about this in the
release notes (as promised).

There is no special treatment for WebRTC based media streams so the same restrictions apply. 

Now I also hear that audiocontext which is used quite some sites to measure the mic level is affected by the autoplay stuff as well.

There is a nice console warning:

    The AudioContext was not allowed to start. It must be resume (or created) after a user gesture on the page. https://goo.gl/7K7WLu

and documentation even. WebRTC is not mentioned on that page. Why?

We (Confrere) were also quite surprised by this. Luckily we have been testing quite extensively on Safari so we managed to rewrite our MediaTester (https://confrere.com/test) to use a user action. However, Safari recently fixed this issue if the page is capturing: https://bugs.webkit.org/show_bug.cgi?id=180680

It's not clear to me if this is a bug we should file or not, but it certainly is a huge change to the way we are used to working with WebRTC and MediaStreams, and it would be nice if these things were made a PSA, and addressed in the official documentation for autoplay policy changes.

(Non-obvious changes such as this is exactly why I posted my original question, why I reached out on Twitter, and why I still want a guide to what specific changes the new autoplay policy has on WebRTC and MediaStreams)