Mixer example

[Durgesh Mishra]

As per the documentation available on internet, here is the definition of Mixer.

"A Hub that allows routing of video between arbitrary port pairs and mixing of audio among several ports"

But the API to connect and disconnect is similar to Dispatcher with an additional 'MediaType'.

My question is, how do I make it mix audio of all ports and send to one ? because apparently it seems this connect api will just connect one source to one sink.

Please throw some light.

Regards

Durgesh O Mishra

[Durgesh Mishra]

I tried to use Mixer but it crashes the KMS.

Regards

Durgesh O Mishra

[Ivan Gracia]

Hi Durgesh,

What version of KMS are you using and how didi it crash? Do you get any errors in there?

Ivan Gracia

--
You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Durgesh Mishra]

Hi Ivan,

I am using following version of KMS on ubuntu,

Version: 5.1.3

Found modules:

Module: 'core' version '5.1.4'

Module: 'elements' version '5.1.3'

Module: 'filters' version '5.1.1'

2015-06-24 17:28:18.694087 14948 [139635864753920]   debug KurentoWebSocketTransport WebSocketTransport.cpp:478 processMessage() Message: >{"id":12,"jsonrpc":"2.0","method":"invoke","params":{"object":"ad6f6438-b516-4973-9b36-eb97cafc06e9_MediaPipeline/e9d735ea-aa9b-48c2-9759-7ef9914e972b_Mixer","operation":"connect","operationParams":{"media":"VIDEO","sink":"ad6f6438-b516-4973-9b36-eb97cafc06e9_MediaPipeline/ec95fc01-f6e1-413c-9c21-ad207a8d9ef2_HubPort","source":"ad6f6438-b516-4973-9b36-eb97cafc06e9_MediaPipeline/4c1f16ed-8d37-4152-9720-97ae8fa6d562_HubPort"},"sessionId":"c8b7572a-a0cd-4a85-9406-4aac50df073f"}}

<

*** Error in `/usr/bin/kurento-media-server': free(): invalid pointer: 0x00007eff6004ffe8 ***

Regards

Durgesh O Mishra

[Durgesh Mishra]

Hi Ivan,

I also tested it against and it still crashes.

Version: 5.1.3

Found modules:

Module: 'core' version '5.1.5'

Module: 'elements' version '5.1.4'

Module: 'filters' version '5.1.1'

It crashes when I call following,

this._mixer.connect('VIDEO',sourceHubPort,sinkHubPort, callback);

Regards

Durgesh O Mishra

[Durgesh Mishra]

Please give some feedback on this. Am I doing mistake in using connect API ?

Regards

Durgesh O Mishra

On Monday, June 22, 2015 at 8:13:22 PM UTC+5:30, Durgesh Mishra wrote:

[Ivan Gracia]

Hi Durguesh,

The mixer hasn't been receiving attention for some time. Our main focus it's right now on webrtc and performance. I'd suggest you try first with KMS6, and if you get the same error we can debug it form there, as it will be easier, and also bugfixes will come more often :-)

Ivan Gracia

--

[Durgesh Mishra]

Hi,

I updated the code for KMS 6 and tried to use Mixer the same way. And it still crashed at the same point.

When I replace Mixer by Dispatcher, everything works fine.. But I wanted to do mixing of audio and video of just one participant.

Here is the last thing I see in the media server.log

0:04:33.807714898 ^[[336m14219^[[00m 0x7f4a40026060 ^[[37mDEBUG  ^[[00m ^[[00mKurentoWebSocketTransport WebSocketTransport.cpp:482:processMessage:^[[00m Message: >{"id":24,"jsonrpc":"2.0","method":"invoke","params":{"object":"e9b7c07e-d55c-40a4-a2e0-fad2a00f8ab5_kurento.MediaPipeline/7b1bd1d9-4188-4a3c-b31d-eadf24324ea3_kurento.Mixer","operation":"connect","operationParams":{"media":"VIDEO","sink":"e9b7c07e-d55c-40a4-a2e0-fad2a00f8ab5_kurento.MediaPipeline/28c01042-0771-4326-af4e-9cb4830fcd72_kurento.HubPort","source":"e9b7c07e-d55c-40a4-a2e0-fad2a00f8ab5_kurento.MediaPipeline/2b1a9e0f-cb68-4367-a515-df1ec4332a07_kurento.HubPort"},"sessionId":"a8db867e-85cd-4d27-aeb8-f0c50c142e52"}}

Please help.

Regards

Durgesh O Mishra

On Monday, June 22, 2015 at 8:13:22 PM UTC+5:30, Durgesh Mishra wrote:

[Jesse Rusak]

I’m having the same problem with Mixer. Here’s my sample code; kms crashes while handling the RPC for the the last line.

I’m trying to get many-to-one audio. Is there a supported solution for that case? 

- Jesse

WebRtcEndpoint endpointA = new WebRtcEndpoint.Builder(pipeline).build();

WebRtcEndpoint endpointB = new WebRtcEndpoint.Builder(pipeline).build();

WebRtcEndpoint endpointOut = new WebRtcEndpoint.Builder(pipeline).build();

Mixer mixer = new Mixer.Builder(pipeline).build();

HubPort a = new HubPort.Builder(mixer).build();

HubPort b = new HubPort.Builder(mixer).build();

HubPort out = new HubPort.Builder(mixer).build();

endpointA.connect(a, MediaType.AUDIO);

endpointB.connect(b, MediaType.AUDIO);

out.connect(endpointOut, MediaType.AUDIO);

mixer.connect(MediaType.AUDIO, a, out);

[Jesse Rusak]

For anyone else’s reference, I was able to get this to work with KMS 6 and the “Composite” hub.

[Alberto Pierini]

Could you post the code please?

[Sanket Mehta]

Hello,

Its good that your grid code is workign great. I am working on composite since last week but not able to generate results.

Can you please share your code of working composite?

It will be so helpful.

[Jon Ruddell]

Hi Sanket,

For Composite code, please see the tests which show a working example. 

https://github.com/Kurento/kurento-java/blob/master/kurento-integration-tests/kurento-test/src/test/java/org/kurento/test/functional/composite/CompositeWebRtcTest.java

Best,

Jon

[Sanket Mehta]

Hey Jon,

Thank you for reply.

I have already checked that file, but i am not able to execute that project in any way.

Its always saying,

Could not resolve dependencies for project org.kurento:kurento-test:jar:6.6.3-SNAPSHOT and some others.

Do you have any idea?

[Micael Gallego]

Please use an stable version like 6.6.2:

https://github.com/Kurento/kurento-java/blob/6.6.2/kurento-integration-tests/kurento-test/src/test/java/org/kurento/test/functional/composite/CompositeWebRtcTest.java

Or configure SNAPSHOT Maven repositories:

http://doc-kurento.readthedocs.io/en/stable/mastering/kurento_development.html#kurento-java-client

Best regards

Micael Gallego

Kurento / OpenVidu Project Lead

To unsubscribe from this group and stop receiving emails from it, send an email to kurento+unsubscribe@googlegroups.com.
To post to this group, send email to kur...@googlegroups.com.
Visit this group at https://groups.google.com/group/kurento.

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/2aafa012-e8b2-4ab4-bfca-6753506deebc%40googlegroups.com.

[Sanket Mehta]

Thank you for response Micael.

I am now in folder /kurento-java/kurento-integration-tests/kurento-test/

When i execute mvn compile exec:java it gives me following error :


Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project kurento-test: The parameters 'mainClass' for goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java are missing or invalid -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project kurento-test: The parameters 'mainClass' for goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java are missing or invalid
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginParameterException: The parameters 'mainClass' for goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java are missing or invalid
    at org.apache.maven.plugin.internal.DefaultMavenPluginManager.populatePluginFields(DefaultMavenPluginManager.java:641)
    at org.apache.maven.plugin.internal.DefaultMavenPluginManager.getConfiguredMojo(DefaultMavenPluginManager.java:594)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:121)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
    ... 20 more


I am using 6.6.2 version of kurento test

To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.

To post to this group, send email to kur...@googlegroups.com.
Visit this group at https://groups.google.com/group/kurento.

[Jon Ruddell]

Sanket,

Sorry I should be more clear, I have not tried to run the integration tests but the code in the test, specifically these lines, show the correct flow for creating a composite with four sources and one viewer.

Best,

Jon

[Jesse Rusak]

Sorry, I don't have this sample handy anymore. If I recall correctly it really was just like the code below with the Mixer, but using Composite instead. The only difference might be that the "out" hubport was created first in my code, but I kind of doubt that is the problem. (To be clear, this was audio-only, not a grid.)

Hope that helps!

- Jesse

[Sanket Mehta]

Hi, Thanx for reply.

I have checked that code and implemented in my system. I am trying to make grid of 4 videos. All are copied from same file, so there wont be issue of resolution and time.

It is giving me black screen in output.

    MediaPipeline pipeline = kurento.createMediaPipeline();

    WebRtcEndpoint webRtcEndpoint1 = new WebRtcEndpoint.Builder(pipeline).build();
    WebRtcEndpoint webRtcEndpoint2 = new WebRtcEndpoint.Builder(pipeline).build();
    WebRtcEndpoint webRtcEndpoint3 = new WebRtcEndpoint.Builder(pipeline).build();
    WebRtcEndpoint webRtcEndpoint4 = new WebRtcEndpoint.Builder(pipeline).build();

    PlayerEndpoint playerEndpoint1 = new PlayerEndpoint.Builder(pipeline, "file:///tmp/Record1.webm").build();
    PlayerEndpoint playerEndpoint2 = new PlayerEndpoint.Builder(pipeline, "file:///tmp/Record2.webm").build();
    PlayerEndpoint playerEndpoint3 = new PlayerEndpoint.Builder(pipeline, "file:///tmp/Record3.webm").build();
    PlayerEndpoint playerEndpoint4 = new PlayerEndpoint.Builder(pipeline, "file:///tmp/Record4.webm").build();

    playerEndpoint1.connect(webRtcEndpoint1);
    playerEndpoint2.connect(webRtcEndpoint2);
    playerEndpoint3.connect(webRtcEndpoint3);
    playerEndpoint4.connect(webRtcEndpoint4);

    Composite composite = new Composite.Builder(pipeline).build();

    HubPort hubPort1 = new HubPort.Builder(composite).build();
    HubPort hubPort2 = new HubPort.Builder(composite).build();
    HubPort hubPort3 = new HubPort.Builder(composite).build();
    HubPort hubPort4 = new HubPort.Builder(composite).build();
    HubPort hubPort5 = new HubPort.Builder(composite).build();

    webRtcEndpoint1.connect(hubPort1);
    webRtcEndpoint2.connect(hubPort2);
    webRtcEndpoint3.connect(hubPort3);
    webRtcEndpoint4.connect(hubPort4);

    WebRtcEndpoint webRtcEndpointOut = new WebRtcEndpoint.Builder(pipeline).build();
    hubPort5.connect(webRtcEndpointOut);

    String sdpOffer = jsonMessage.get("sdpOffer").getAsString();
    String sdpAnswer = webRtcEndpointOut.processOffer(sdpOffer);

    JsonObject response = new JsonObject();
    response.addProperty("id", "startResponse");
    response.addProperty("sdpAnswer", sdpAnswer);

    playerEndpoint1.play();
    playerEndpoint2.play();
    playerEndpoint3.play();
    playerEndpoint4.play();

    synchronized (session) {
        session.sendMessage(new TextMessage(response.toString()));
    }

    webRtcEndpointOut.gatherCandidates();

Also i have tried combining two streams from camera, but its saying me that media pipeline is not allowed to share.

[Jon Ruddell]

Hi Sanket,

You should connect the PlayerEndpoints directly to the HubPorts, instead of through a WebRtcEndpoint.

Best,

Jon

[Sanket Mehta]

Hi Jon,

Now it is working like a charm, and playing all 4 files in a grid. Thank you very much for your support.

Now I am converting this code to work with direct camera input from the user. 

[Sanket Mehta]

Hello,

I am trying to access stream from kurento to ffmpeg but dont have luck. I am using following code to generate offer. Can anyone please help?

MediaPipeline pipeline = kurento.createMediaPipeline();
WebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build();
webRtcEndpoint.connect(webRtcEndpoint);

RtpEndpoint rtpEndpoint = new RtpEndpoint.Builder(pipeline).build();

String rtcSdpOffer = "v=0\n" +
        "o=- 0 0 IN IP4 127.0.0.1\n" +
        "s=Kurento\n" +
        "c=IN IP4 127.0.0.1\n" +
        "t=0 0\n" +
        "m=video 46784 RTP/AVP 100\n" +
    "a=rtpmap:100 H264/90000\n" +
        "a=recvonly";
String rtcSdpAnswer = rtpEndpoint.processOffer(rtcSdpOffer);

webRtcEndpoint.connect(rtpEndpoint);

console.log(rtcSdpAnswer);

In output of rtcEndPoint output i get following.

v=0
o=- 3711506820 3711506820 IN IP4 192.168.0.199
s=Kurento Media Server
c=IN IP4 192.168.0.199
t=0 0
m=video 2730 RTP/AVP 100
a=rtpmap:100 H264/90000
a=sendonly
a=ssrc:1614090344 cname:user1105350959@host-50c074d0

I saved that in file and opened with ffmpeg and also with vlc. But i cant get stream.

Thank you.