RTP to WebRTC scaled down resolution

[Max Dörfler]

Hey,

I'm running a setup where I stream RTP (generated by gstreamer and a videotestsrc) to a WebRTC Endpoint. The video stream I feed in is 1280x720 but the quality of the video in the browser is pretty bad. In Firefox I displayed the "video statistics" and the displayed resolution was smaller than the input and then scaled to my video player size. I cannot recall the exact resolution and currently I cannot reproduce this since there is this Firefox bug where it won't work with Kurento at all. I don't know how to display the resolution of the stream in chrome, does anybody know?
Anyway, I think somewhere in Kurento my Video is scaled down. Am I right? Can I prevent this from happening?

Some Info:
System: Ubuntu 14.04
Kurento Server: 5.1.3
Node Application with kurento-client 5.1.0

I create the RTP Enpoint by passing the following SDP:
v=0
o=- 1188340656180883 1 IN IP4 127.0.0.1
c=IN IP4 127.0.0.1
t=0 0
m=video 50000 RTP/AVP 101
a=rtpmap:101 H264/90000
m=audio 50010 RTP/AVP 98
a=rtpmap:98 OPUS/48000/2

This is the pipeline I'm currently sending. I send to the ports returned by the RTP Endpoint.
gst-launch-1.0 -v rtpbin latency=1000 name=rtpbin audiotestsrc is-live=true ! audio/x-raw,channels=2,rate=48000 ! audioconvert ! audioresample ! opusenc ! rtpopuspay pt=98 ! rtpbin.send_rtp_sink_1 rtpbin.send_rtp_src_1 ! udpsink host=127.0.0.1 port=54658 videotestsrc is-live=true ! video/x-raw,framerate=25/1,width=1280,height=720 ! x264enc bitrate=1000 byte-stream=false tune=zerolatency sliced-threads=true ! h264parse config-interval=1 ! rtph264pay pt=101 ! rtpbin.send_rtp_sink_0 rtpbin.send_rtp_src_0 ! udpsink host=127.0.0.1 port=38324

I then create a Webrtc Enpoint via client JS and connect the RtpEndpoint to the WebRTCEndpoint
kurentoUtils.WebRtcPeer.startRecvOnly(videoOutput, onOffer, onError);

I hope this is clear enough. Thanks in advance.
Max

[Ivan Gracia]

Are you using kurento-utils-js? In that wrapper for RTCPeerConnection, we have default constraints. Just create your own, and pass that to the webrtcPeer object during construction

mediaConstraints = { 
                     audio : true, 
                     video : {  
                              mandatory : {  
                                           maxWidth : 320,  
                                           maxFrameRate : 15,  
                                            minFrameRate : 15 
                                          }  
                              }  
                   };

The issue with firefox has also been fixed in KMS 5 & 6, but you’ll have to use the nightly builds PPA.

Ivan Gracia

--
You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

​

[Max Dörfler]

Thanks,
yes I'm using kurento-utils-js. I've already tried the following:

var constraints = {
        audio : true,
        video : {
            mandatory : {
                maxWidth : 1280,
                maxHeight: 720,
                minWidth : 1280,
                minHeight: 720,
            }
        },
    };
    webRtcPeer = kurentoUtils.WebRtcPeer.startRecvOnly(videoOutput, onOffer, onError, constraints);

and didn't see any difference.
There are no i386 builds for 5.1.4, right? :(

[Ivan Gracia]

Sorry, no 32 builds!

KMS is not doing anything special with video sizes. We are streaming full hd video with a similar setup (only the other way around: webrtc -> rtp) and it works fine. Did anything change in your webrtc-internals after you changed the constraints?

About the video quality being shown in the browser, the bitrate of two endpoints connected inside KMS is, by default, 300kpbs. That means that you won't see the same quality in the webrtc endpoint. There is a way to set the output bitrate, have a look at this post.

Ivan Gracia

[Ivan Gracia]

Sorry, no 32 builds!

KMS is not doing anything special with video sizes. We are streaming full hd video with a similar setup (only the other way around: webrtc -> rtp) and it works fine. Did anything change in your webrtc-internals after you changed the constraints?

About the video quality being shown in the browser, the bitrate of two endpoints connected inside KMS is, by default, 300kpbs. That means that you won't see the same quality in the webrtc endpoint. There is a way to set the output bitrate, have a look at this post.

Ivan Gracia

[Max Dörfler]

Thnak you so far.

Okay I got a 64bit Ubuntu running now (it was time anyway :) ) and it is now working on firefox again with the nightly builds. The displayed resolution is always jumping between different values but never 1280x720 (see screenshots).

If I understand the other post correctly setting the bitrate is a pretty new feature and is not yet possible via the nodejs library but only the java library. Did I get this right?

[Jose Antonio Santos Cadenas]

The resolutions changes you are experiencing are made by the codec, Vp8 changes the size if it considers that the reduced size fits better in the target bitrate.

[Max Dörfler]

Okay cool, thanks. So how do I adjust the bitrate? Which Versions of Kurento/Kurento-JS are needed and do you maybe have a code snippet? Sorry I don't want to appear lazy but I can neither find anything in the docs nor in the code (only shallow searches).

[Ivan Gracia]

You'll need KMS6. Clients are in the master branch in Github.

Ivan Gracia

[jesuscarb...@gmail.com]

Anyone find a solution for this problem? I don't find anything about bitrate :(

We suffered a high change to low resolution in 5 secs.

[Ivan Gracia]

When did this happen? Could you provide some more info, along with your KMS version?

Ivan Gracia

On Fri, Jun 26, 2015 at 4:07 AM, <jesuscarb...@gmail.com> wrote:

Anyone find a solution for this problem? I don't find anything about bitrate :(

We suffered a high change to low resolution in 5 secs.

[jesuscarb...@gmail.com]

Hello Ivan,

We are using KMS 5 with Js. Our intention is convert a video format H.264 to VP8, but we don't know how configure the end point resolution. We are testing in a local network.

Constraints of code of controller.js:

var constraints = {

        audio : true,

        video : {

            mandatory : {

                maxWidth : 1280,

                maxHeight: 720,

                minWidth : 1280,

                minHeight: 720,

                maxFrameRate : 30,  

                minFrameRate : 30 

            }

        },

    };

And sdp_patter (we don't know exactly how configure it :) we don't find any documentation)

v=0

o=- 0 0 IN IP4 0.0.0.0

s=Kurento Media Server

c=IN IP4 0.0.0.0

t=0 0

m=audio 0 RTP/AVP 98 99 0

a=rtpmap:98 OPUS/48000/2

a=rtpmap:99 AMR/8000/1

a=rtpmap:0 PCMU/8000

m=video 0 RTP/AVP 96 97 100 101

a=rtpmap:96 H263-1998/90000

a=rtpmap:97 VP8/90000

a=rtpmap:100 MP4V-ES/90000

a=rtpmap:101 H264/90000

a=fmtp:97 profile-level-id=15; mode=AAC-hbr;config=1408; SizeLength=13;IndexLength=3;IndexDeltaLength=3; Profile=1; bitrate=180000;

[jesuscarb...@gmail.com]

Hello,

We cleaned sdp_pattern now:

v=0

o=- 0 0 IN IP4 127.0.0.1

s=Kurento Media Server

c=IN IP4 127.0.0.1

t=0 0

m=audio 0 RTP/AVP 98 99 0

a=rtpmap:98 OPUS/48000/2

a=rtpmap:99 AMR/8000/1

a=rtpmap:0 PCMU/8000

m=video 0 RTP/SAVPF 120

a=rtpmap:120 VP8/90000

Our kurento.conf.json is default, and here is our javascript code (in AngularJs):

'use strict';

var constraints = {

        audio : true,

        video : {

            mandatory : {

                maxWidth : 1980,

                maxHeight: 1080 ,

                minWidth : 1980,

                minHeight: 1080 ,

                maxFrameRate : 60,  

                minFrameRate : 60 

            }

        },

    };

angular.module('clinicaonlineApp')

    .controller('VideoconferenceController', function ($scope, $translate) {

     $scope.ws_uri = "ws://x.x.x.x:8888/kurento";

     $scope.file_uri = "rtsp://aaaa:bb...@x.x.x.x/media.amp";

    

     $scope.videoOuput = document.getElementById('videoOutput');

    

     $scope.onError = function (error) {

     console.error(error);

     }

    

     $scope.loadAxis = function(){

     console.log('holaaa');

     console.info($scope.videoOuput);

     };

    

     $scope.loadAxis();    

     $scope.webRtcPeer = kurentoUtils.WebRtcPeer.startRecvOnly(

     $scope.videoOuput, 

function(sdpOffer){

     console.info("$scope.onOffer");

     kurentoClient($scope.ws_uri, function(error, kurentoClient) {

         console.info("kurentoClient");

     if(error) return $scope.onError(error);

     kurentoClient.create("MediaPipeline", function(error, pipeline) {

     if(error) return $scope.onError(error);

     pipeline.create('PlayerEndpoint', {uri: $scope.file_uri, useEncodedMedia: true}, function(error, player) {

     if(error) return $scope.onError(error);

     player.on('EndOfStream', function(event) {

     pipeline.release();

     videoInput.src = '';

     if ($scope.webRtcPeer) {

     $scope.webRtcPeer.dispose();

     }

     });

     pipeline.create("WebRtcEndpoint", function(error, webRtc){

     if(error) return $scope.onError(error);

     webRtc.processOffer(sdpOffer, function(error, sdpAnswer){

     if(error) return $scope.onError(error);

     $scope.webRtcPeer.processSdpAnswer(sdpAnswer);

     console.log("Processed SDP: " + sdpOffer);

     });

     player.connect(webRtc, function(error){

     if(error) return $scope.onError(error);

     console.log("webRtc connects to player");

     });

     });

     player.play(function(error){

     if(error) return $scope.onError(error);

     console.log("Playing");

     });

     });

     });

     });

     }, $scope.onError, constraints);

});

And this is our sdpOffer:

v=0
o=mozilla...THIS_IS_SDPARTA-38.0.5 4294967295 0 IN IP4 0.0.0.0
s=-
t=0 0
a=sendrecv
a=fingerprint:sha-256 B0:55:15:E4:F5:73:F2:97:1F:FC:F2:45:54:9D:93:DC:8E:37:69:02:6D:D4:66:93:9F:7A:86:D6:69:ED:CC:87
a=group:BUNDLE sdparta_0 sdparta_1
a=ice-options:trickle
a=msid-semantic:WMS *
m=audio 57827 RTP/SAVPF 109 9 0 8
c=IN IP4 10.8.72.38
a=candidate:0 1 UDP 2128609535 10.8.72.38 57827 typ host
a=candidate:0 2 UDP 2128609534 10.8.72.38 57828 typ host
a=recvonly
a=end-of-candidates
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=ice-pwd:13f01b2a7542bd0a581000d82f244131
a=ice-ufrag:24869769
a=mid:sdparta_0
a=rtcp-mux
a=rtpmap:109 opus/48000/2
a=rtpmap:9 G722/8000/1
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=setup:actpass
m=video 57685 RTP/SAVPF 120 126 97
c=IN IP4 10.8.72.38
a=candidate:0 1 UDP 2128609535 10.8.72.38 57685 typ host
a=candidate:0 2 UDP 2128609534 10.8.72.38 57686 typ host
a=recvonly
a=end-of-candidates
a=fmtp:120 max-fs=12288;max-fr=60
a=fmtp:126 profile-level-id=42e01f;level-asymmetry-allowed=1;packetization-mode=1
a=fmtp:97 profile-level-id=42e01f;level-asymmetry-allowed=1
a=ice-pwd:13f01b2a7542bd0a581000d82f244131
a=ice-ufrag:24869769
a=mid:sdparta_1
a=rtcp-fb:120 nack
a=rtcp-fb:120 nack pli
a=rtcp-fb:120 ccm fir
a=rtcp-fb:126 nack
a=rtcp-fb:126 nack pli
a=rtcp-fb:126 ccm fir
a=rtcp-fb:97 nack
a=rtcp-fb:97 nack pli
a=rtcp-fb:97 ccm fir
a=rtcp-mux
a=rtpmap:120 VP8/90000
a=rtpmap:126 H264/90000
a=rtpmap:97 H264/90000 

a=setup:actpass 

But we can't solve this issue, can anyone help me please? In 5 secs, video scaled down resolution dramaticaly... :(

We are using KMS 5.

 

[Ivan Gracia]

With KMS5, you won't be able to change the bitrate between endpoints. I'm afraid you´ll have to upgrade to KMS6.

Ivan Gracia

--

[Sandeep M R]

Hello Ivan,

I am using kurento 6.1. And I dont see any difference which I try to downgrade the video size and bit rate. Below is a screenshot of my configuration.


Best Regards,
Sandeep

[Ivan Gracia]

I'm not really sure what you want to achieve. Could you be a little bit more specific?

Ivan Gracia

[Sandeep M R]

Hi Ivan,

My intention is to scale down the video size and bit rate on need basis. In my previous reply, I have attached the screenshot of the code. When I bring down the numbers, I don't see any change in the quality of the video. Is it expected?

Best Regards,
Sandeep

[Ivan Gracia]

Could you check what comes out of the adapter? Kurento-utils was doing some unexpected magic, and maybe your constraints are being overwritten. You can use this page to test the different constraints.

By the way, those constraints are also not according to spec. Have a look at the default constraints used in the latest version of kurento-utils-js.

Ivan Gracia