How to enable Opus in-band FEC?

[Mikhail Fludkov]

Good day, everybody.

I'm experimenting with inband FEC in Opus right now. My goal is to prove that Chrome is able to recover audio packets using FEC and send packets with FEC as well. Looking at Chrome's SDP offer I see that neither Chrome 39 nor 37 advertise "useinbandfec" property. What I see is:

a=rtpmap:111 opus/48000/2

a=fmtp:111 minptime=10

Therefore it seems Chrome is unable to receive Opus FEC. At the same time according to this discussion https://code.google.com/p/webrtc/issues/detail?id=2419, it is able to send it. So what I do is advertise "useinbandfec" in the SDP answer back to Chrome:

a=rtpmap:111 opus/48000/2

a=fmtp:111 useinbandfec=1;

I'm using libopus for decoding. To check that FEC information is actually present in the Opus bitstream I'm monitoring if there are any audio frames containing LBRR frames. According to this discussion http://lists.xiph.org/pipermail/opus/2013-January/001904.html FEC information consists of LBRR frames. So I expect to see audio frames with LBRR flag = 1. But unfortunately it is always zero. To provoke Chrome to send audio stream with FEC information I tried to apply packet loss from chrome to call server and verified that chrome receives valid data about packet loss in RTCP. This didn't help either.

So far I'm unable to prove that Chrome supports Opus in-band FEC and wondering if I'm doing something wrong. Am I?

[Justin Uberti]

Try running apprtc.appspot.com with the opusfec=true parameter. This will force the use of useinbandfec=1, and you should see FEC frames if packet loss occurs.

[SProgrammer]

@Justin Uberti: I tried to simulate this and i do not see the FEC frames even when packet loss occurs (used 3G to laptop tethering hotspot and made weak/strong signals noticed packet loss in video/audio but no-where it was showing if it recovered)

[Vikas]

Hi,

What version of chrome you tested this with?

/Vikas

[SProgrammer]

@Vikas: I have tested with Google Chrome and Canary and AppRTC with that flag. 

[Vikas]

Hi,

I think since its inband FEC you won't see separate FEC packets as its included in opus payload. I tried testing by applying packetloss and then doing a listening test with 'useinbandfec =1' in sdp and without it. You can turn off inband fec for opus in apprtc by setting opusfec = false in url. I also looked at the logs and i could see it printing ' Attempt to enable codec internal fec on channel' when 'useinbandfec=1' was used. But if you find an issue, please file a bug.

/Vikas

[Justin Uberti]

It will be difficult to observe the FEC in the wire packets, since the FEC is inband, and the packets are encrypted. We will add stats to indicate when FEC is actually used in the near future (please file a bug at http://bugs.webrtc.org if you would like updates on progress).

[Mikhail Fludkov]

Hi Justin,

Sorry for late reply. For some reason I didn't receive email updates about replies to my question. As far as I can recollect I could prove that chrome able to send inband FEC information in Opus packets. But it was unstable 38.x, don't remember the version exactly. I'll play with apprtc.appspot.com and if I will find something weird I'll file a bug. BTW the FEC information (both for video and audio) in webrtc-internals would be very very use full. Looking forward to see it.

[Ugur]

This is a quite old topic but when useinbandfec is enabled, does opus include fec frames all the time or only when packet loss reaches a certain point? And what would be the advantage/disadvantage of ulpfec vs inbandfec? 

Thanks

[Mikhail Fludkov]

Yes, if there is no packet loss FEC will not be used. In my opinion, they are complementary and if you can you should use them together. Here is my take on differences between the two.

- Turning on/off usage of Opus inband FEC is very simple. You feed the packet loss stats to libopus and it does the magic for you. But just like every magic, you can't see what is happening behind the scenes ;) Here is what I mean. Opus FEC data is encapsulated within encoded frame, it does not come out of libopus as a separate chunk of encoded data. Thus there is no easy way to check whether the FEC data is actually exists in encoded frame without sophisticated decoding process. It is not only that what makes it "magical", sometimes you will not get FEC data at all. So you can imagine someone spending some time bending their architecture to provide the packet loss feedback to libopus. It works in the beginning, but then the code around it evolves and it stops producing FEC data at all. The reason why it happened is, as I found out latter, that opus can operate in SILK, CELT and some hybrid mode. Inband FEC is supported only when libopus brain selects SILK encoding mode. I don't remember exactly, but I think it is depending on the encoding settings: sample rate, frame size and bitrate. But if you manage to squeeze out the FEC data out of libopus. It provides very good protection against random packet loss. In case of busty loss, unfortunately, you will be able to hear robotic audio artifacts, as libopus audio packet loss concealment is not the best.

- with ULPFEC on the other hand there is no magic at all. You can grab a pcap and find FEC packets in with wireshark. It is media agnostic, meaning if you support it - you can use it for audio and video. But there is no library which everybody can use. There are other nasty things in rfc5109, but I will not highlight it here.

[Philipp Hancke]

2014-10-11 3:42 GMT+02:00 Justin Uberti <jub...@google.com>:

It will be difficult to observe the FEC in the wire packets, since the FEC is inband, and the packets are encrypted. We will add stats to indicate when FEC is actually used in the near future (please file a bug at http://bugs.webrtc.org if you would like updates on progress).

Did those stats ever make it somewhere? It would be good to determine from getStats if a call used FEC.

This should be known at the sender but I don't see it exposed on the ssrc report (and it is not obvious in the stats spec either)