RTCP Packet Loss

[Andrew Higginson]

Hi all :)

First a big thanks, I'm working with Janus as part of my dissertation and its made everything a lot easier :D

I'm currently trying to perform rate control (via scaling video quality and framerate) using Janus and so I figured the best way to do this is to monitor packet loss via incoming_rtcp, so I tested this by modifying the Streaming plugin as below:

void janus_streaming_incoming_rtcp(janus_plugin_session *handle, int video, char *buf, int len) {
if (handle == NULL || handle->stopped || g_atomic_int_get(&stopping) || !g_atomic_int_get(&initialized)) return;

rtcp_context ctx;
janus_rtcp_parse(&ctx, buf, len);
uint32_t lost = janus_rtcp_context_get_lost(&ctx);
uint32_t lostp = janus_rtcp_context_get_lost_fraction(&ctx);

printf("Packet loss = %lu = %lu percent\n", (unsigned long)lost, (unsigned long)lostp);
}

However when I do so, I seem to be receiving garbage values, which don't seem to match up with the values in chrome://webrtc-internals, they can be seen here:

http://pastebin.com/w0ndAEmA (no packet loss over loopback)

http://pastebin.com/vzHJm4fX (~2% packet loss simulated over loopback)

I feel as though I must be doing something wrong? 

Many thanks,

Andrew Higginson

[Lorenzo Miniero]

Plugins do not get the whole RTCP stuff: most RTCP, including RR/SR, are terminated by the core, which then only passes limited RTCP messages to the core (e.g., REMB, FIR, etc.). If you want to have statistics info on a call, you should use the admin API instead, which summarizes what is exchanged via RTCP too in its stats: http://www.meetecho.com/blog/understanding-the-janus-admin-api/

To get triggers on insufficient bandwidth on the plugin side, you can make use on the slow_link callback: the core will notify plugins using that callback when there are problems on either uplink or downlink for a media connection the plugin is handling, something it usually inferes from the amount of NACKs sent or received.

L.

[Andrew Higginson]

Hey :)

On Saturday, 4 June 2016 13:17:12 UTC+1, Lorenzo Miniero wrote:

Plugins do not get the whole RTCP stuff: most RTCP, including RR/SR, are terminated by the core, which then only passes limited RTCP messages to the core (e.g., REMB, FIR, etc.). If you want to have statistics info on a call, you should use the admin API instead, which summarizes what is exchanged via RTCP too in its stats: http://www.meetecho.com/blog/understanding-the-janus-admin-api/

Not sure if I understood correctly, but when I do use the admin api to take a look at packet loss (through admin.html in demos/) the packet loss again doesnt match up. It shows zero (http://pastebin.com/4RkAwC6y) when I can see via webrtc-internals and by the stuttering of the video it definitely is not zero :D

In any case, as you say below, its really the plugin side I want this info to be exposed.

 

To get triggers on insufficient bandwidth on the plugin side, you can make use on the slow_link callback: the core will notify plugins using that callback when there are problems on either uplink or downlink for a media connection the plugin is handling, something it usually inferes from the amount of NACKs sent or received.

Is there a reason why packet loss isn't exposed to plugins? I don't believe the flexibility/fidelity I desire in implementing a rate control algorithm will be given by relying on the slow_link callback?

Thanks for the quick reply :D

Andrew Higginson

 

[Lorenzo Miniero]

Il giorno sabato 4 giugno 2016 14:51:11 UTC+2, Andrew Higginson ha scritto:

Hey :)

On Saturday, 4 June 2016 13:17:12 UTC+1, Lorenzo Miniero wrote:

Plugins do not get the whole RTCP stuff: most RTCP, including RR/SR, are terminated by the core, which then only passes limited RTCP messages to the core (e.g., REMB, FIR, etc.). If you want to have statistics info on a call, you should use the admin API instead, which summarizes what is exchanged via RTCP too in its stats: http://www.meetecho.com/blog/understanding-the-janus-admin-api/

Not sure if I understood correctly, but when I do use the admin api to take a look at packet loss (through admin.html in demos/) the packet loss again doesnt match up. It shows zero (http://pastebin.com/4RkAwC6y) when I can see via webrtc-internals and by the stuttering of the video it definitely is not zero :D

It also shows a ton of video nacks, though:

"video_nacks": 1046,

which definitely justify the stuttering video.

 

In any case, as you say below, its really the plugin side I want this info to be exposed.

 

To get triggers on insufficient bandwidth on the plugin side, you can make use on the slow_link callback: the core will notify plugins using that callback when there are problems on either uplink or downlink for a media connection the plugin is handling, something it usually inferes from the amount of NACKs sent or received.

Is there a reason why packet loss isn't exposed to plugins? I don't believe the flexibility/fidelity I desire in implementing a rate control algorithm will be given by relying on the slow_link callback?

The reason is that it makes more sense to just implement it once in the core, which already handles the termination of the other media layers, rather than force each plugin to do the exact same thing over and over. Plugin developers should only worry about getting media and the possibility to generate/route some of their own, without having to worry about much else. If you need to be aware of problems in the network in order to adapt, the slow_link callback I mentioned allows you to do that, as it's triggered whenever NACKs in either direction exceed some threshold (which means the streaming plugin has been notified in your example above). Different plugins can then handle this feedback in different ways. The EchoTest, for instance, reacts by cutting down the cap bitrate, and telling the user about this, do that banwidth consumption is reduced automatically. Other plugins like VideoRoom and Record&Play just notify the user, and leave decisions up to the application level, e.g., to also reduce the bitrate, to close some streams, or other: different applications using the same plugin may want to do different things, which is why an autmated logic is not always a good idea.

Not sure what kind of rate control you'd like to do, and if it is in a new plugin or the Streaming one, but whatever it is, the slow_link callback is probably the way to go. If you believe it doesn't provide enough info as it is, feel free to play with the core or provide suggestions on how to improve that.

As a side note, we're also working on a new mechanism called the Events API in a separate branch: https://github.com/meetecho/janus-gateway/pull/536

That would allow you to receive info from multiple sources (core, plugins) on the whole lifecycle of a session, which might be of interest to you as well. Still WIP and incomplete, but feedback there would be even more welcome.

L.

[Andrew Higginson]

Ok sounds good, i think I'll explore both slow_link but also seeing about getting packetloss from the core (or possibly sending it via getstats back to the plugin)

Are you able to give a pointer of where rtcp packets getting packet loss are 'terminated' by the core? Are i correct in thinking it just doesn't pass these packets on, or does terminated mean something else?

Thanks,

Andrew Higginson

--
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Andrew Higginson

[Lorenzo Miniero]

Il giorno sabato 4 giugno 2016 16:48:40 UTC+2, Andrew Higginson ha scritto:

Ok sounds good, i think I'll explore both slow_link but also seeing about getting packetloss from the core (or possibly sending it via getstats back to the plugin)

About that, notice that the loss you see in the Admin API is not the whole loss that was there: actually I think it only refers to the last loss since the last SR, but I'm not sure. Thinking about it, an indication on the total loss since the beginning of the call would be a good idea (retty much as we do for the NACKs in the stats).

 

Are you able to give a pointer of where rtcp packets getting packet loss are 'terminated' by the core? Are i correct in thinking it just doesn't pass these packets on, or does terminated mean something else?

Yes, that's what it means. Check ice.c for where RTCP packets are received: they're all handled there, sometimes invoking methods from rtcp.c, before they're passed to plugins. Apart from RTCP packets that plugins might want to send, the core originated RTCP messages in each user's send thread instead, still in ice.c.

L.

 

Thanks,

Andrew Higginson

On Sat, 4 Jun 2016 at 14:15 Lorenzo Miniero <lmin...@gmail.com> wrote:

Il giorno sabato 4 giugno 2016 14:51:11 UTC+2, Andrew Higginson ha scritto:

Hey :)

On Saturday, 4 June 2016 13:17:12 UTC+1, Lorenzo Miniero wrote:

Plugins do not get the whole RTCP stuff: most RTCP, including RR/SR, are terminated by the core, which then only passes limited RTCP messages to the core (e.g., REMB, FIR, etc.). If you want to have statistics info on a call, you should use the admin API instead, which summarizes what is exchanged via RTCP too in its stats: http://www.meetecho.com/blog/understanding-the-janus-admin-api/

Not sure if I understood correctly, but when I do use the admin api to take a look at packet loss (through admin.html in demos/) the packet loss again doesnt match up. It shows zero (http://pastebin.com/4RkAwC6y) when I can see via webrtc-internals and by the stuttering of the video it definitely is not zero :D

It also shows a ton of video nacks, though:

"video_nacks": 1046,

which definitely justify the stuttering video.

 

In any case, as you say below, its really the plugin side I want this info to be exposed.

 

To get triggers on insufficient bandwidth on the plugin side, you can make use on the slow_link callback: the core will notify plugins using that callback when there are problems on either uplink or downlink for a media connection the plugin is handling, something it usually inferes from the amount of NACKs sent or received.

Is there a reason why packet loss isn't exposed to plugins? I don't believe the flexibility/fidelity I desire in implementing a rate control algorithm will be given by relying on the slow_link callback?

The reason is that it makes more sense to just implement it once in the core, which already handles the termination of the other media layers, rather than force each plugin to do the exact same thing over and over. Plugin developers should only worry about getting media and the possibility to generate/route some of their own, without having to worry about much else. If you need to be aware of problems in the network in order to adapt, the slow_link callback I mentioned allows you to do that, as it's triggered whenever NACKs in either direction exceed some threshold (which means the streaming plugin has been notified in your example above). Different plugins can then handle this feedback in different ways. The EchoTest, for instance, reacts by cutting down the cap bitrate, and telling the user about this, do that banwidth consumption is reduced automatically. Other plugins like VideoRoom and Record&Play just notify the user, and leave decisions up to the application level, e.g., to also reduce the bitrate, to close some streams, or other: different applications using the same plugin may want to do different things, which is why an autmated logic is not always a good idea.

Not sure what kind of rate control you'd like to do, and if it is in a new plugin or the Streaming one, but whatever it is, the slow_link callback is probably the way to go. If you believe it doesn't provide enough info as it is, feel free to play with the core or provide suggestions on how to improve that.

As a side note, we're also working on a new mechanism called the Events API in a separate branch: https://github.com/meetecho/janus-gateway/pull/536

That would allow you to receive info from multiple sources (core, plugins) on the whole lifecycle of a session, which might be of interest to you as well. Still WIP and incomplete, but feedback there would be even more welcome.

L.

 

Thanks for the quick reply :D

Andrew Higginson

 

--
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.

To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janus+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Andrew Higginson

[Andrew Higginson]

Thanks I'll take a look :)

 

Thanks,

Andrew Higginson

To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Andrew Higginson

--

You received this message because you are subscribed to the Google Groups "meetecho-janus" group.

To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Andrew Higginson

[Andrew Higginson]

Hey,

having some trouble here, I'm trying to track down when an RTCP packet is being received by Janus' core, which has correct packet loss information in it, which as Lorenzo& said isn't being passed onto plugins.

The way I'm trying to do this is this insert some code into the start of janus_ice_cb_nice_recv as so:

line 1667, ice.c

[...]

if (janus_is_rtcp(buf)) {

    rtcp_context ctx;
    janus_rtcp_process_incoming_rtp(&ctx, buf, len);
    uint32_t lost = janus_rtcp_context_get_lost(&ctx);

    printf("Lost= %d\n", lost);
}

[...]


To be clear, so it produces this http://pastebin.com/j7vxA6BE, however this gives me garbage values still. I also tried using the stream->video_rtcp_ctx instead of a new rtcp_context however this also didn't work, I think I'm misunderstanding something

Thanks,

Andrew Higginson

--

Andrew Higginson

[Lorenzo Miniero]

That code already processes incoming RTCP packets and updates the info accordingly, no need to do that again. You may want to look into how the messages are processed instead (rtcp.c), and check why it's not giving the results you expect.

L.

 

Thanks,

Andrew Higginson

To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janus+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Andrew Higginson

--
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.

To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janus+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Andrew Higginson

--

Andrew Higginson

[Andrew Higginson]

Ok so I believe I've worked it all out now, just posting in case someone needs a similar thing.

I was getting confused thinking rtcp_context was the same as an RTCP packet (which it isn't!)

The core does seem to pass through the RTCP packets I need (RR) via the incoming_rtcp plugin method and so the packet can be accessed like so (code adapted from code in rtcp.c). In the example, I log the cumulative packet loss and fraction of packet loss in the RR RTCP packets (http://tools.ietf.org/html/rfc3550#section-6.4.1)

void janus_streaming_incoming_rtcp(janus_plugin_session *handle, int video, char *buf, int len) {
     if (handle == NULL || handle->stopped || g_atomic_int_get(&stopping) || !g_atomic_int_get(&initialized))
        return;

 

     rtcp_header *rtcp = (rtcp_header*) buf;
     int total = len;
 

     while(rtcp) {
         if (rtcp->type == RTCP_RR) {

             rtcp_rr *rr = (rtcp_rr*)rtcp;
             uint32_t fractionlost = ntohl(rr->rb[0].flcnpl) >> 24;
             uint32_t packetloss = ntohl(rr->rb[0].flcnpl) & 0x00FFFFFF;
 

             JANUS_LOG(LOG_INFO, " Fraction Lost: %d, CumPacket: %d\n", fractionlost, packetloss);
             break;
         }
 

         /* If this is a compound packet, move onto next one */
         int length = ntohs(rtcp->length);
         total -= length*4+4;
 

         if (length == 0 || total <= 0) {
             // No more left to process
             break;
         }
         rtcp = (rtcp_header *)((uint32_t*)rtcp + length + 1);
     }
 }

 

Thanks,

Andrew Higginson

To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Andrew Higginson

--
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.

To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Andrew Higginson

--

Andrew Higginson

[Lorenzo Miniero]

Il giorno lunedì 6 giugno 2016 16:16:55 UTC+2, Andrew Higginson ha scritto:

Ok so I believe I've worked it all out now, just posting in case someone needs a similar thing.

I was getting confused thinking rtcp_context was the same as an RTCP packet (which it isn't!)

Yep, it's just a structure where we keep track of what's happening, and that gets updated when we receive some messages.

I think we're only parsing SR right now, while we ignore RR. We should probably update the context for incoming RR as well.

 

The core does seem to pass through the RTCP packets I need (RR) via the incoming_rtcp plugin method and so the packet can be accessed like so (code adapted from code in rtcp.c). In the example, I log the cumulative packet loss and fraction of packet loss in the RR RTCP packets (http://tools.ietf.org/html/rfc3550#section-6.4.1)

You're right, I was incorrect last time: it's *outgoing* packets that are discarded. That is, if your plugin generates a RR/SR, then the core dumps it, because it's going to generate them on its own. Sorry for the confusion...

L.

[Lorenzo Miniero]

I think we're only parsing SR right now, while we ignore RR. We should probably update the context for incoming RR as well.

Just implemented and pushed. This should also make the info in the Admin API more coherent (e.g., by checking lost-by-remote to see how many packets the peer says he lost).

L.

[Andrew Higginson]

Cool :)