NACK, PLI, FIR

[Nitesh Bansal]

Hello,

I have built a video service with WebRTC and asterisk.

I am having issues with video quality, so i decided to investigate RTCP feedback messages.
Following RTCP feedback messages seem very interesting, can you please clarify me their exact purpose and when should they be used.

NACK
FIR
PIL

It would be also nice to know the difference between PLI and FIR, these messages look very similar to me, as per my understanding, both PLI and FIR are used to request a fresh image for the video stream.
Any responses or references/links are welcome.

Regards,
Nitesh

[Matthew Fredrickson]

Replies inline.

On Tuesday, December 16, 2014 8:53:28 AM UTC-6, Nitesh Bansal wrote:

Hello,

I have built a video service with WebRTC and asterisk.

I am having issues with video quality, so i decided to investigate RTCP feedback messages.
Following RTCP feedback messages seem very interesting, can you please clarify me their exact purpose and when should they be used.

NACK 

FIR
PIL

It would be also nice to know the difference between PLI and FIR, these messages look very similar to me, as per my understanding, both PLI and FIR are used to request a fresh image for the video stream.

As I understand it, when dealing with packet loss related to picture frames, a PLI should be used, and that an FIR should be used ONLY in cases where an absolutely new refresh point is needed - so for example in a selective forwarding conferencing unit, where the speaker stream is changed to a new speaker, a new full picture frame would be requested from the new speaker via an FIR.

Here's a good paragraph in RFC5104 that seems to describe it (Section 4.3.1.2):

   FIR SHALL NOT be sent as a reaction to picture losses -- it is
   RECOMMENDED to use PLI [RFC4585] instead.  FIR SHOULD be used only in
   situations where not sending a decoder refresh point would render the
   video unusable for the users.

   A typical example where sending FIR is appropriate is when, in a
   multipoint conference, a new user joins the session and no regular
   decoder refresh point interval is established.  Another example would
   be a video switching MCU that changes streams.  Here, normally, the
   MCU issues a FIR to the new sender so to force it to emit a decoder
   refresh point.  The decoder refresh point normally includes a Freeze
   Picture Release (defined outside this specification), which re-starts
   the rendering process of the receivers.  Both techniques mentioned
   are commonly used in MCU-based multipoint conferences.

Matthew Fredrickson

Respoke.io

[Nitesh Bansal]

Hi Matthew,

Thanks for the response.

Do you have any idea if Chrome is using FIR or PLI to request image refresh? Basically i need to decide the priority of development for these FB messages, so which of these messages is more widely used in WebRTC world.

Thanks,
Nitesh

[BobK]

NACK will likely give you the best quality improvement.  This is used to handle packet loss.  It allows each side to request retransmission of packets not received by the decoder.  Enabling NACK also disables the VP8 error concealment (or so it once was reported).  The concealment is not really worth enabling.

[Nitesh Bansal]

Hi Bob,

I have a feeling that in a congested network, NACK is going to add to the congestion? i may be wrong, please correct me.
Moreover, since we are just a video gateway between Web browser and SIP phones, so we are forwarding the messages between two parties, are there lot of SIP phones/clients which support these features?

Nitesh

On Tuesday, December 16, 2014 3:53:28 PM UTC+1, Nitesh Bansal wrote:

[Sergio Garcia Murillo]

If NACK is going to add congestion, imagine what PLI/FIR would do! ;)

The correct way of handling it is with NACK+REMB/TMBR with lower bitrate to reduce congestion. Better if you have the congestion algorithm adapt before congestion happens.

BTW, IMHO pure media forwarding won't work at all (well, at least you require DTLS to SRTP), check this draft for RTCP handling in B2BUA apps:

https://tools.ietf.org/html/draft-miniero-straw-b2bua-rtcp-00

BR
Sergio

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Lorenzo Miniero]

Actually the latest version of the draft is this one :-)

https://tools.ietf.org/html/draft-ietf-straw-b2bua-rtcp-02

For the rest, as Sergio anticipated, just answering to NACKs by retransmitting is likely to add to the congestion, yes. In principle, when receiving several NACKs peers should realize there's something wrong and reduce the bitrate as well as part of the process (check

https://tools.ietf.org/html/rfc4588#section-7 for some better wording on this). Browsers also do this with the aid of REMB. While REMB messages can just be forwarded, though (using the approach described in the STRAW draft) NACK probably can't, for several reasons which include SRTP replays since you have a B2BUA in the path, which means you'll have to handle them directly.

L.

[Nitesh Bansal]

Thanks guys for your thoughts.

I have a question regarding REMB, the easy solution proposed is to forward the REMB message,
But is it going to work in the scenario when calling Browser has limited upload speed, but very download speed, will the Web Browser automatically adjust its sending bitrate?
In my use case, we don't antiicipate any bandwidth constraints on the gateway, we are observing that bandwidth constraint is normally on the side of browser (residential users with limited upload bandwidth)?


Regards,

Nitesh

On Tuesday, December 16, 2014 3:53:28 PM UTC+1, Nitesh Bansal wrote:

[Sergio Garcia Murillo]

Yes it will work as long as both ends supports REMB, which won't happen if both sides are not chrome.

In that case you will have to end RTCP and calculate the REMB by yourself to report it back to chrome.

BR
Sergio

[Lorenzo Miniero]

If you relay REMB, the behaviour is going to be the same as p2p browser calls. If it works there, it will for you as well.

L.

[Nitesh Bansal]

Actually in my setup, i have a Web Browser as calling party, our gateway being in the middle and on the termination side, we have a SIP phone and i think SIP phone won't support REMB.

So i am not really sure how it will work?

Regards,
Nitesh

On Tuesday, December 16, 2014 3:53:28 PM UTC+1, Nitesh Bansal wrote:

[Sergio Garcia Murillo]

You will have to implement in the gateway your congestion control algorithm and send the calculated REMB values back to chrome.

Note that will only handle congestion between chrome and gateway, you will have to handle congestion between gateway and phone in another way.

Best regards
Sergio

[BobK]

All of these feedback methods will add to congestion.  Only the ones like REMB will attempt to reduce the bitrate and thus reduce the number of packets per video frame.  However, is congestion a real issue for you?

FIR is a means of requesting a complete key frame.  These have the most packets per frame.

NACK is a mechanism to request packets not received by the decoder.  These are packets lost on the network (less common) and in the browser itself (more common) due to buffer limitations.

Let's say you lose a couple of packets from a key frame.  Would you rather have all the packets retransmitted (a new key frame is sent so not actually retransmitted) or just the ones which were lost?

[Nitesh Bansal]

Well some clarify ref Chrome now.
I can integrate REMB algorithm for Chrome.
Can you guys give me an idea is Firefox using any kind of congestion control?

Regards,
Nitesh Bansal

On Tuesday, December 16, 2014 3:53:28 PM UTC+1, Nitesh Bansal wrote:

[Nitesh Bansal]

Hi folks,

Another question on Congestion control between Web browser and out gateway,
i learnt that VP8 is adaptive codec, won't it adjust the bitrate if it finds out that network connectivity (specifically upload speed) isn't good enough.
In this case, do we still need the mechanisms like REMB?

Nitesh

On Tuesday, December 16, 2014 3:53:28 PM UTC+1, Nitesh Bansal wrote:

[Sergio Garcia Murillo]

How is the vp8 encoder supposed to know about the network conditions?
Spoiler: REMB.

Bes regards
Sergio

[Oscar Divorra Escoda]

VP8 does not adapt magically. It requires a mechanism of end-to-end rate control to adapt. REMB (together with RR) is part of that end-to-end rate control.

Best,

   Oscar

[Nitesh Bansal]

Thanks for erasing the stupid doubt in my mind.

As per my understanding then, REMB is implemented only in Chrome, how about other web browsers like Firefox?

Nitesh

On Tuesday, December 16, 2014 3:53:28 PM UTC+1, Nitesh Bansal wrote:

[Raj Kumaradass]

Doesn't "a=rtcp=fb:100 goog-remb" specific to chrome?(https://tools.ietf.org/html/draft-alvestrand-rmcat-remb-03) It appears firefox has not included in it as i do see only below RTCP FB attributes supported in firefox.

a=rtcp-fb:120 nack

a=rtcp-fb:120 nack pli

a=rtcp-fb:120 ccm fir

a=rtcp-fb:126 nack

- Raj.

[Vikas]

Hi,

Yes the goog-remb is Chrome specific param.

/Vikas

To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrtc+unsubscribe@googlegroups.com.

[Nitesh Bansal]

And since it is specific to Google only, it makes me wonder again and again, is there any mechanism in Firefox to do congestion/flow contorl, i had a look at SDP generated by FF and i don't see any anything related to congestion control in FF.

Nitesh

On Tuesday, December 16, 2014 3:53:28 PM UTC+1, Nitesh Bansal wrote:

[Randell Jesup]

On 12/24/2014 4:15 AM, Nitesh Bansal wrote:

And since it is specific to Google only, it makes me wonder again and again, is there any mechanism in Firefox to do congestion/flow contorl, i had a look at SDP generated by FF and i don't see any anything related to congestion control in FF.

Note that Firefox includes options like this:

a=rtcp-fb:120 nack
a=rtcp-fb:120 nack pli
a=rtcp-fb:120 ccm fir

NACK is primarily useful in low-RTT (or large jitter buffer) cases, where a NACK can be sent and arrive before the frame is pulled from the jitter buffer for decode.  *When* it's useful, there's a large advantage to it in terms of bandwidth and video quality compare to a keyframe/IDR.  Also note that the sender does not have to do retransmission in response to a NACK, but can instead send a keyframe or IDR if it prefers (or nothing).

FIR is Full Intraframe Refresh request , PLI is Packet Loss Indicator.  For loss recovery, normally you want PLI.  See https://svn.tools.ietf.org/html/draft-ietf-avt-avpf-ccm-10#section-4.3.1.2 for discussion of when you should use FIR instead.

Nitesh

On Tuesday, December 16, 2014 3:53:28 PM UTC+1, Nitesh Bansal wrote:

Hello,

I have built a video service with WebRTC and asterisk.

I am having issues with video quality, so i decided to investigate RTCP feedback messages.
Following RTCP feedback messages seem very interesting, can you please clarify me their exact purpose and when should they be used.

NACK
FIR
PIL

It would be also nice to know the difference between PLI and FIR, these messages look very similar to me, as per my understanding, both PLI and FIR are used to request a fresh image for the video stream.

-- 
Randell Jesup -- rjesup a t mozilla d o t com