Limiting bitrate of video streams

[vivaldi-va]

I'm working on limiting the bandwidth consumption of video streams for my video conferencing application,
though I haven't been able to reduce the bitrate below an avarage of around 600kbps.

I've set the room config to a bitrate of 128kbps when creating a room via Janus' API. I've also set the getUserMedia constraints to 320x240 and 10 fps.
Media constraints are all working, fps is definately low.

I also tried adding `b=AS:256` and `b=TIAS:256` into the offer SDP under the `a=mid` line, such as:

a=mid:video
b=TIAS:256

Codec is also the default used by Janus.

Is there any way to lower the bitrate? Quite new to WebRTC in general so I'm not quite sure where to go from here, but 600kbit (for 320x240@10fps) sounds a bit high.

[Oscar Vadillo]

I don't understand, When I config a room to a bitrate of 128kbps I never send streams with that diference of bitrate. Are you sure you are configuring correctly the room?

[vivaldi-va]

Request to create a room is done by:

{
  request: 'create',
  room: roomId,
  publishers: 12,
  bitrate: 128000,
}

I can see a difference in visual quality, though no change in actual bit rate oddly.

[Lorenzo Miniero]

Is this in Chrome or Firefox? How do you show the bitrate in use? Do you see any difference when playing with the bandwidth controls in the echo test?

https://janus.conf.meetecho.com/echotest

If so, then it should be working in VideoRoom, as we do the exact same thing there (REMB manipulation).

L.

[Mike]

Which browser are you using to test this?

Firefox does not currently support bandwidth control using B=AS, only Chromium browsers do.

After following this bug; https://bugzilla.mozilla.org/show_bug.cgi?id=976521 for 2 years it's finally planned to be added in V49 which should be released around September (B=TIAS, not =AS)

It seems silly to completely trust the client not to send loads of bandwidth though, I wonder if this can be limited on server-side easily

[Lorenzo Miniero]

Not without transcoding, and we're not going to do that. As long as browsers honor REMB feedback, you should be fine. We don't rely on the bandwidth attribute in the SDP typically.

L.

[Mike]

Judging by the comments in this thread Firefox also does not support bandwidth limiting with REMB

https://bugzilla.mozilla.org/show_bug.cgi?id=976521#c4

So users can publish and just push 2Mbps to everyone, which is definitely unwanted in poublic use cases

If you don't rely on the bandwidth attribute in the SDP, and browsers don't support REMB what would you use to limit this?

Is my only other option to destory handles that go over the bandwidth limit?

Thanks

Op vrijdag 10 juni 2016 09:49:31 UTC+2 schreef Lorenzo Miniero:

[Lorenzo Miniero]

In our experience Firefox seems to honor incoming REMB feedback though. Just try the echo test with Firefox and lower the bandwidth cap, and you'll see it go down.

L.

[vivaldi-va]

I'm displaying the bitrate by calling

remoteFeed.getBitrate();

which shows around 550-650kbit Also chrome webrtc-internals shows the same, and I've only been testing on chrome. I'll take a look at the echotest when i'm home, but it worked fine from what I remember. Just a question on how the echotest alters bitrate using pluginHandle.send({"message": { "bitrate": bitrate }}); Does that override what is specified in the room configuration (e.g. `bitrate: 128000`)? Or is it a separate mechanism?

[Lorenzo Miniero]

Il giorno venerdì 10 giugno 2016 13:52:27 UTC+2, vivaldi-va ha scritto:

I'm displaying the bitrate by calling

remoteFeed.getBitrate();

which shows around 550-650kbit Also chrome webrtc-internals shows the same, and I've only been testing on chrome. I'll take a look at the echotest when i'm home, but it worked fine from what I remember. Just a question on how the echotest alters bitrate using pluginHandle.send({"message": { "bitrate": bitrate }}); Does that override what is specified in the room configuration (e.g. `bitrate: 128000`)? Or is it a separate mechanism?

It does the same than a configure request to the videoroom plugin with a bitrate value would do, to override the bitrate cap for that specific user.

L.

[vivaldi-va]

I've tried using echotest on the same Janus server i'm testing my own project on at home, and the bitrate limitations work as expected.
Also tried setting the bandwidth via a configuration message on the receive only handle with no change either.

Here's a dump of the handle data from a receiver running from the videoroomtest on a room I created using my own application if it helps:
http://pastebin.com/ZG5b9aG0

Also the result from calling 'list' on the videoroom plugin

{
    room: 1002392081,
    description: 'Room 1002392081',
    max_publishers: 12,
    bitrate: 256000,
    fir_freq: 0,
    audiocodec: 'opus',
    videocodec: 'vp8',
    record: 'false',
    num_participants: 1
}

[Lorenzo Miniero]

You can only change the bitrate on the publisher's side, not the viewers.

L.

[vivaldi-va]

Tried sending the bitrate configure message in the success callback when creating an offer:

var publish = {
  request: 'configure',
  audio: true,
  video: true,
  bitrate: 128 * 1000
};

pluginHandle.send({ message: publish, jsep: jsep });

Still the same, no change in bitrate on a remote. In any case, shouldn't the bitrate in the room configuration be limiting publisher bitrate across the board?

[Lorenzo Miniero]

Il giorno sabato 11 giugno 2016 03:39:52 UTC+2, vivaldi-va ha scritto:

Tried sending the bitrate configure message in the success callback when creating an offer:

var publish = {
  request: 'configure',
  audio: true,
  video: true,
  bitrate: 128 * 1000
};

pluginHandle.send({ message: publish, jsep: jsep });

Still the same, no change in bitrate on a remote. In any case, shouldn't the bitrate in the room configuration be limiting publisher bitrate across the board?

Yes, which is why you send it for the publisher, and not for viewers as you said you tried to do.

[vivaldi-va]

Yep, I added the bitrate property to the send-only plugin handler, like what is configured in the videoroom demo.

Interestingly, with some more testing, I've noticed thatsometimes the bandwidth limitations will work and i'll get a ~128kbit/s rate for that publisher.
chrome webrtc-internals graph:
https://i.gyazo.com/4947de46cb800e4f0bb40c8c8cec0ffe.png

Howerver, in other cases there will be no limit and the googAvailableSendBandwidth graph will skyrocket, I'm assuming to my LAN bandwidth:
https://i.gyazo.com/411c40cff9bc33b69fdc425715e1b2bd.png

I cant figureout exactly why this is happening, seems to be 50/50.

Heres the local description of the unlimited publisher:
http://pastebin.com/F6MYuPXT

Here's the local description of a limited publisher (at 256kbit in this case):
http://pastebin.com/fZrzCxF0

[Lorenzo Miniero]

Have you tried sending a new configure (wthout jsep) *after* the PeerConnection has been established?

L.

[vivaldi-va]

Tried sending it using the webrtcState callback on a true value from the publisher handler, no effect.

[vivaldi-va]

Also looked at the janus log output (debug lvl.7) for a stream that isn't being limited according to webrtc-internals, and it looks like REMB is added to the stream:
http://pastebin.com/anu00hR4

though Chrome doesn't seem to use it.

On Saturday, 11 June 2016 14:49:44 UTC+1, Lorenzo Miniero wrote:

[vivaldi-va]

Bit more testing, seems that setting the log level to 5 or more causes the bitrate limiting to work pretty much all the time. Sending the configure request only in the webrtcState callback.

Could it be that the increased latency from more log output is giving things more time to happen?

On Saturday, 11 June 2016 14:49:44 UTC+1, Lorenzo Miniero wrote:

[Lorenzo Miniero]

Or maybe some method that is called in a logger function, and so only activates when the log is high enough (unlikely though).

If you're using the latest Janus version, try going back some revisions until you find one that works. This might help us figure out if any recent change caused a regression.

L.

[vivaldi-va]

Went back through the history, d49a834e4bd62c26a7ed65a9093b5af59c4fbffd in PR  #467 seems to be the culprit. Commits prior to it have working bitrate limiting. Master also works up until the merge.

[Lorenzo Miniero]

So exactly when we started doing RTCP termination... it's not a problem of missing packets, as we fixed that a few days ago (we were not using the internal methods and some packets originated from the core were dropped). I was just able to reproduce this as well in a VideoRoom with two people: one of them was capped at 128kbps, the other wasn't. Anyway, for the user that wasn't, sending a new configure during the call:

sfutest.send({message: {request: "configure", bitrate: 128000}});

fixed that, which makes me think what's missing or not working are the REMB messages as soon as the media session starts. I'll try and see if this can be fixed in an easy way.

Lorenzo 

[Lorenzo Miniero]

I think I found out the cause of the issue, let me know if this fixes it for you:

https://github.com/meetecho/janus-gateway/commit/2185b377714294ce362034448df6814d27f18703

The problem was basically that we were not zeroing the buffer for the automated REMB messages (that is, those the VideoRoom plugin sends by itself every tot seconds), which sometimes caused a padding bit to be incorrectly set, and as a consequence to an incorrect RTCP packet from the recipient's perspective. I fixed this in rtcp.c and it looks like it's working for me now. Let me know if that's not the case.

Lorenzo

[vivaldi-va]

Seems to be working for me, nice one!

[Lorenzo Miniero]

Thanks to you for spotting that out, it was a nasty bug that has been lingering there in the shadows for too long!

L.

[Mike]

Hi Lorenzo,

I've been testing the bandwidth limiting with every Major Firefox for quite some time, I've just tested it again with EchoTest on Firefox V47.0 and it did not limit the bitrate (it did on Chrome)

These 2 comments seem to confirm that Firefox only allows bandwidth limiting by changing the resolution and framerate;

https://bugzilla.mozilla.org/show_bug.cgi?id=976521#c9

https://bugzilla.mozilla.org/show_bug.cgi?id=1159489#c9

And here one by a Janus user implying it might be limited by changing a value in about:config some version ago, but I was not able to replicate this and it'd still leave it up to the user

https://groups.google.com/d/msg/meetecho-janus/n2c6yLY8jqM/a_jY2oyxBAAJ

Users coming into big videorooms where 64Kbps is the norm for Chrome users, starting to broadcast anywhere from 200kbps to 2Mbps can be quite annoying.
Any help on how to better limit this on the server-side is greatly appreciated

Thanks,

Mike

[Lorenzo Miniero]

Well, with a 1280x960 stream it's hard to stay low :-)

Anyway, not sure why, but my Firefox tests always seemed to honor the bandwidth caps. I just tried the echotest page of our demos with Firefox 47:

https://janus.conf.meetecho.com/echotest

it starts high (~900kbps), then hitting 128kbps brings it back down to ~200kbps. Whatever bandwidth I select, Firefox 47 seems to adapt, increasing to that value when I go up, and decreasing to what I ask for when I go down. That's why I always thought that, while Firefox seems not to generate REMB feedback, it at least seems to be able to parse it and adapt the encoded bitrate settings.

In your case this might not be working because of the too high resolution. Have you checked if it does indeed work if you use, let's say, 640x480?

That said, Firefox 49 should now support the b=TIAS attribute, which is something we might add to the plugins in the future.

Lorenzo

[Mike]

Thanks for the quick reply!

I just tried with a resolution of 640x360 and am indeed able to limit the bitrate a fair bit now, from 1500kbps down to 200kbps 

Setting anything lower than 200kbps doesn't seem to make a difference, 64000 will still be 200kbps, even after I've changed media.peerconnection.video.min_bitrate to '1' in about:config

echotest.send({"message": { "bitrate": 64000 }});

Dropping the framerate locally seems to help, at 10 FPS the bitrate drops to 80kbps, at 7 or 8 FPS it's around 50-70kbps which was the target bitrate I set.

After digging a bit it seems to be this pieice of code setting the min and max bitrates based on resolution, then limiting that some further with the framerate

https://hg.mozilla.org/integration/mozilla-inbound/rev/340c59ea962d#l1.66

This all relies on the getUserMedia request having the constraints and the about:config change, which can always be forced higher by the client.

Would there be a proper server-sided way of limiting the bitrate, whether it's bottlenecking the amount of data client can send, or monitoring their bitrate directly in the plugin.

I'm not quite comfortable enough with C to get something like this going without some examples and I'm not sure what to use to best limit this server sided, as relying on browser versions will also become a problem when trying to block older Firefox versions which will not support B=TIAS

To work around that I decided to create a nodejs application to poll the Admin API and check "video_bytes_lastsec" to kill the handle when it exceeds the maximum bitrate, but looping through sessions and handles to find the publishing handles to get that info each time seems like a long-winded way of doing it

I appreciate any input on how to deal with something like this in a more robust way for public purposes

Thanks, Mike

[Lorenzo Miniero]

We already do the best we can in limiting the bitrate on the server side. Anything else would require transcoding, which we don't and won't do.

As to polling the handles via Admin API, have a look at the Events API branch on github: https://github.com/meetecho/janus-gateway/pull/536

The existing sample plugin allows you to receive asynchronous events on several things via HTTP, including the statistics you mentioned. That would be definitely easier for you to handle, as you wouldn't have to loop anything, the info would fall in your lap. That branch is experimental, so if you play with it let us know how well it performs and if it suits your needs: the sooner we know it does its job well, the sooner we can merge.

L.