Loss of quality and delay gap between audio & video when using modules like FaceOverlay or PlateDetector

[pro...@linux-power.de]

Hello,

after running the examples we noticed a strong drop of the streams quality as soon as we use any type of filtering.

When using the advanced one2one example and removing the faceoverlay, (so only having a videoconference including recording), the quality of the video was high, and audio and video were synchroneous.

The same test with activated faceoverlay (hat example) made the quality of the video far worse, showing high pixeling, and there started being a delay between audio and video that went up to 5 seconds or even more.

I understand there's obvious processing involved, but this drop of quality seems quite extreme. Is there a way to tweak the system to get past this problem?

We're testing on a this machine:
16 core AMD 2,3GHz
16 GB RAM
SSD drives
Ubuntu 14.04.2 LTS
Bandwidth can't be the issue either.

Is there i.e. a way to tell the system not to run the filtering on every frame (which I assume it does), but only say once or twice per second (for the platedetector and similar OCR detections)? Would this lessen the load and improve the quality?

Best regards

Markus

[Ivan Gracia]

Hi Markus,

When one media element is connected to another, the one that is acting as source will push media at 300kbps as long as there is a transcodification. This means that two WebRTC endpoints will exchange media normally, but if you go through a filter, for instance, the filter will get video at this lower quality. In the development version, media elements have a method setOutputBitrate to control this feature.

As for your second question, you can instantiate a generic GSreamerFilter that holds a videorate filter. With this filter, you reduce the frame ratio to the desired number. this would certainly reduce the CPU load, which might have an impact in video quality, depending on the number of clients connected, type of server and so on.

Ivan Gracia

--
You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[pro...@linux-power.de]

Hi Ivan,

thanks for your answer. I assume that I can forget about the second part, since the problem is noch CPU processing, but the incoming lower quality when using a filter.

You mentioned the method "setOutputBitrate" to control this feature. Could you elaborate here please? Do you mean I can correct the problem and get a higher video quality (or in best case the same quality as when not using a filter) when I use setOutputBitrate?

Furthermore, could you point me to some example where and how to use this method? I'm a little lost and couldn't find any further info to it. A code snippet would be of really great help here.

Thanks a lot!

Markus

[Ivan Gracia]

Hi Markus,

This feature has been introduced in the development version of KMS one week ago, so there are no examples for this. However, if you use the Java API, it should be already available in any MediaElement, as you can see in the module descriptor. You can invoke that method in any media element that you have instantiated. For instance, webrtcEp.setOutputBitrate(1_500_000) will give you 1.5MB from that endpoint to whatever you connect on the other side. The only issue is that you will have to compile your own version of kurento-client, since SNAPSHOTS are not published in central.

Cheers,

Ivan Gracia

[pro...@linux-power.de]

Hi Ivan,

thanks for you answer. We now installed KMS 6.0, and I adapted the code the way I understood it was to be used in CallMediaPipeline.java:

    public CallMediaPipeline(KurentoClient kurento, String from, String to) {
       
        // Media pipeline
        pipeline = kurento.createMediaPipeline();
       
        // Media Elements (WebRtcEndpoint, RecorderEndpoint, FaceOverlayFilter)
        webRtcCaller = new WebRtcEndpoint.Builder(pipeline).build();
        webRtcCallee = new WebRtcEndpoint.Builder(pipeline).build();

        // bitrate (your suggestion)
        webRtcCallee.setOutputBitrate(1_500_000);    // 1,5mbs
       
        recorderCaller =
                new RecorderEndpoint.Builder(pipeline, RECORDING_PATH + from
                        + RECORDING_EXT).build();
...

But when compiling I get this errormessage:

ERROR] /kurento-tutorial-java/kurento-one2one-call-advanced/src/main/java/org/kurento/tutorial/one2onecalladv/CallMediaPipeline.java:[60,29] cannot find symbol
  symbol:   method setOutputBitrate(int)
  location: variable webRtcCallee of type org.kurento.client.WebRtcEndpoint

What am I doing wrong? Is this not how it's supposed to be used?

Best regards

Markus

[Ivan Gracia]

What version of the tutorials and Java libraries are you using?

Ivan Gracia

[pro...@linux-power.de]

We have just now installed the KMS 6.0, updated from 5.0 install 2 weeks ago.

That is where the tutorials came from, so they probably are 5.0, same with the java libraries, if they do not auto-update when updating to 6.0.

I don’t know what exactly to tell you, we’re just getting started with java because of kurento, so very rookie here.

Where exactly can I find the infos you are asking for?

Markus

You received this message because you are subscribed to a topic in the Google Groups "kurento" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kurento/LsEhwHzdNFk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kurento+u...@googlegroups.com.

[Ivan Gracia]

In your pom.xml file, there should be a versionId. Most likely pointing to 5.1.0... You could checkout the master branch from github, and you'll get the tutorials updated for free :-)

What programming languages are you used to? We also have JavaScript and NodeJS tutorials, in case that helps.

Ivan Gracia