forcing a keyframe

[Bryan Donnovan]

I have webrtc running in a client-server scenario, forwarding one stream to multiple people.

When someone joins the chat, there can be quite a lag before a keyframe shows up to give them video of the current speaker.  The one reliable method of producing a keyframe that I have found is to cover the camera briefly then uncover it again.

I would like to be able to force a keyframe when a new person joins, or to be able to specify a keyframe interval to set an upper bound on the video sync delay.

I've read that ways to set video codec parameters like bitrate and aspect ratio are forthcoming.  Will max keyframe interval be one of them, or is there some other way to handle my situation?

Thanks

[Mamadou]

You can send a RTCP FIR to request key frame.

[Bryan Donnovan]

Do you know for certain that this is implemented in WebRTC?  

Unlike H.261 http://www.networksorcery.com/enp/rfc/rfc2032.txt the VP8 payload format does not mention RTCP_FIR

http://tools.ietf.org/html/draft-ietf-payload-vp8-04 

On Wednesday, May 30, 2012 4:49:02 PM UTC-7, Mamadou wrote:

You can send a RTCP FIR to request key frame.

[Justin Uberti]

You want RFC 5104 FIR (http://tools.ietf.org/html/rfc5104)

[Mamadou]

@bryan
yes
you're not looking at the right place

On May 31, 3:21 am, Justin Uberti <jube...@google.com> wrote:
> You want RFC 5104 FIR (http://tools.ietf.org/html/rfc5104)
>

> On Wed, May 30, 2012 at 8:36 PM, Bryan Donnovan <bryandonno...@gmail.com>wrote:
>
>
>
>
>
>
>
> > Do you know for certain that this is implemented in WebRTC?
>

> > Unlike H.261http://www.networksorcery.com/enp/rfc/rfc2032.txtthe VP8

[Bryan Donnovan]

Thanks.  I've had a look at rfc 5104 and modified my offer SDP to include the following:

a=rtpmap:100 VP8/90000

a=rtcp-fb:100 ccm fir

However,  "a=rtcp-fb:100 ccm fir" is not present in the answer.  

Should it be present to indicate support of this feedback message ?

On Wednesday, May 30, 2012 6:24:13 PM UTC-7, Mamadou wrote:

@bryan
yes
you're not looking at the right place

[Ronghua Wu]

On Thu, May 31, 2012 at 8:23 AM, Bryan Donnovan <bryand...@gmail.com> wrote:

Thanks.  I've had a look at rfc 5104 and modified my offer SDP to include the following:

a=rtpmap:100 VP8/90000

a=rtcp-fb:100 ccm fir

However,  "a=rtcp-fb:100 ccm fir" is not present in the answer.  

We didn't implemented the rtcp-fb attribue in our SDP parser. PLI should have been enabled by default for the key frame request. Not sure why it didn't work for you. Can you give more information about your set up? 

[Bryan Donnovan]

I was not saying that it did not work, only that "a=rtcp-fb" was not present in the answer SDP.  I have not actually tried sending that RTCP packet yet.  

Even though we have the availability of RTCP FIR, I still hope that we have some way to establish a maximum keyframe interval at call setup time.  I know that WebRTC is currently focused on P2P, but regular keyframes would make many things go smoother in a client-server scenario.  For example, server side recording of the stream.

On Thursday, May 31, 2012 8:33:39 AM UTC-7, Ronghua Wu wrote:

[Mamadou]

RTP/SAVPF should be enough if chrome developers are logic. e.g. NACK
is not signaled but is sent.
PLI should not be used to request keyframe because of congestion
control.
In short a basic algorithm should be something like this:
if(packet_lost(n)){
if(send_nack(n) == ok){ // pkt retransmitted
// do nothing
}
else{ // the remote cannot retransmit the pkt
if(pkt_type(n) == I || lost_chunck_is_config()){ // Intra frame or
lost chunck contains config (e.g. quant tables for VP8, SPS/PPS for H.
264 etc...)
send_fir();
}
else{ // predicted (P or B for H.264 or Gold for VP8 ...)
if(decode(n) == -1){ // fails to decode/subjective quality is
below a threshold
send_fir();
}
else{
if(subjective < threshold){
send_pli(); // let the remote party send FIR or whatever
based on the bandwidth mgt
}
pkt_display(n);
}
}
}
}

On May 31, 5:33 pm, Ronghua Wu <w...@webrtc.org> wrote:

> On Thu, May 31, 2012 at 8:23 AM, Bryan Donnovan <bryandonno...@gmail.com>wrote:
>
> > Thanks.  I've had a look at rfc 5104 and modified my offer SDP to include
> > the following:
>
> > a=rtpmap:100 VP8/90000
> > a=rtcp-fb:100 ccm fir
>
> > However,  "a=rtcp-fb:100 ccm fir" is not present in the answer.
>
> We didn't implemented the rtcp-fb attribue in our SDP parser. PLI should
> have been enabled by default for the key frame request. Not sure why it
> didn't work for you. Can you give more information about your set up?
>
>
>
>
>
>
>
>
>
> > Should it be present to indicate support of this feedback message ?
>
> > On Wednesday, May 30, 2012 6:24:13 PM UTC-7, Mamadou wrote:
>
> >> @bryan
> >> yes
> >> you're not looking at the right place
>
> >> On May 31, 3:21 am, Justin Uberti wrote:

> >> > You want RFC 5104 FIR (http://tools.ietf.org/html/**rfc5104<http://tools.ietf.org/html/rfc5104>)

>
> >> > On Wed, May 30, 2012 at 8:36 PM, Bryan Donnovan wrote:
>
> >> > > Do you know for certain that this is implemented in WebRTC?
>

> >> > > Unlike H.261http://www.**networksorcery.com/enp/rfc/**rfc2032.txtthe<http://www.networksorcery.com/enp/rfc/rfc2032.txtthe>VP8

> >> > > payload format does not mention RTCP_FIR

> >> > >http://tools.ietf.org/html/**draft-ietf-payload-vp8-04<http://tools.ietf.org/html/draft-ietf-payload-vp8-04>

[Justin Uberti]

You don't want to send keyframes at a regular interval. Keyframes are large and therefore hurt coding efficiency/video quality.

Using NACK and FIR as needed is a much smarter way of handling this.

Regarding a=rtcp-fb, as Ronghua says, that's simply an omission in our SDP. FIR is supported by the underlying media code.

[Randell Jesup]

On 5/31/2012 12:05 PM, Mamadou wrote:
> RTP/SAVPF should be enough if chrome developers are logic. e.g. NACK
> is not signaled but is sent.
> PLI should not be used to request keyframe because of congestion
> control.

Not true, though response to a PLI must be congestion controlled, and
"smart" responses to PLI may avoid some of the issues with classic
"send-a-big-iframe" methods. PLIs over RTCP are limited by RTCP rules
(AVPF in this case, which makes them useful).

> In short a basic algorithm should be something like this:
> if(packet_lost(n)){
> if(send_nack(n) == ok){ // pkt retransmitted

Um, how do you know the packet was retransmitted? And retransmissions
in interactive communication are close-to worthless. Or is this running
on the sender side, and "send nack" is really "retransmit packet"? In
any case, not useful.

> // do nothing
> }
> else{ // the remote cannot retransmit the pkt
> if(pkt_type(n) == I || lost_chunck_is_config()){ // Intra frame or
> lost chunck contains config (e.g. quant tables for VP8, SPS/PPS for H.
> 264 etc...)
> send_fir();

Probably send_idr() (sender-side) is what you mean. The receiver
often can't tell what type of packet was lost (sometimes it can, but not
always). Having the receiver send a nack/pli lets the sender decide the
proper recovery method.

> }
> else{ // predicted (P or B for H.264 or Gold for VP8 ...)
> if(decode(n) == -1){ // fails to decode/subjective quality is
> below a threshold
> send_fir();
> }
> else{
> if(subjective< threshold){
> send_pli(); // let the remote party send FIR or whatever
> based on the bandwidth mgt
> }
> pkt_display(n);
> }
> }
> }
> }

Sorry, there's just too much here I disagree with.

My algorithm
Receiver:
if (packet_loss(n...)) {
send_pli(n...);
}

Sender:
if (is_pli()) {
if (fast_recovery_possible(n...))
send_fast_recovery(n...);
else
send_idr();
}

Where fast_recovery uses any non-IDR techniques available, which vary by
codec (long term reference frames, p-frame recovery, etc). Yes, you can
make it more complex, compare different encodings, etc, but in most
cases it isn't worth it. Fast recovery is generally better than
better-but-later recovery (and IDRs are often slower AND worse and cause
congestion or require dropping frames following, or running higher
compression levels).

--
Randell Jesup
randel...@jesup.org