RtpEndpoint to WebRTC

[ignacio...@gmail.com]

Hi everyone,

I am trying to catch an RTP stream, do some image processing and to send it to a browser. Kurento seems perfect for this!

I have a raspberry pi and I can successfully send its camera stream as RTP using gstreamer, with the following command:

$ raspivid -n -t 0 -fps 30 -hf -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay ! udpsink host=10.0.0.98 port=9011

I know it works because on the machine where KMS is installed, I can read the stream successfully, in real time, using the following command:

$ gst-launch-1.0 udpsrc port=9011 ! application/x-rtp ! rtph264depay ! avdec_h264 ! fpsdisplaysink

Now the challenge is to use Kurento to capture the stream, so I can apply some filter. I've already built the filters, I just cannot capture the stream!

Could anyone provide an example? I am a newbie to Kurento, so far I have successfully ran the tutorials only, and made a couple of filters which work successfully when I plug them in the code of the tutorials.

Some related posts:

In this one, there is no real solution presented, just implications (if I get my code running, I'll post the solution!)

In this other one, they are dealing with the inverse case (WebRTC to RTP), which serves as example. There is some code provided, but where should I add it? Should I start by modifying the HelloWorld tutorial? Where?

Anyhow, I hope this is not too much of a newbie question. I wish there was a tutorial with other sources other than WebRTC only, I guess those will follow with time.

Thank you in advance,

—N

[Ivan Gracia]

Hi Ignacio,

Nice setup you’ve got there! If you want your KMS to be able to get the RTP stream, you’ll need to tell the endpoint where the media will be accessible. You can do so by building your own SDP offer, as if it was created by your R-Pi, and invoke in the server’s endpoint the processOffer method. I guess it would be something like this, assuming the port that the R-Pi is sending the video is 9011.

String offer = "v=0 o=- 0 0 IN IP4 <rpi-addr> s= c=IN IP4 <rpi-addr> t=0 0 m=video 9011 RTP/AVP 100 a=rtpmap:100 H264/90000 a=recvonly";
rtpEp.processOffer(offer);

Cheers,
Ivan Gracia

--
You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

​

[ignacio...@gmail.com]

Hi Ivan!

Thanks for the quick reply.

I'm not sure where to start coding actually; in which part of the code does the SDP offer is built. Do you recommend that I start a project from scratch, or that I start modifying a tutorial such as the HelloWorld or MagicMirror?

Do you have any example projects on the web where I could start my code?

Thanks again for the offer string! I owe you a beer

—N

[ignacio...@gmail.com]

Hi Ivan!

So I went ahead and started to modify the HelloWorld example ... it looks like this:

@RestController
public class HelloWorldController {

 @Autowired
 private KurentoClient kurento;

        String rpiaddr = "192.168.1.105";
       
 @RequestMapping(value = "/helloworld", method = RequestMethod.POST)
 private String processRequest(@RequestBody String sdpOffer) {


 // Media Logic
 MediaPipeline pipeline = kurento.createMediaPipeline();
                RtpEndpoint rtpEp = new RtpEndpoint.Builder(pipeline).build();
               
                WebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build();
 webRtcEndpoint.connect(rtpEp);
                rtpEp.connect(webRtcEndpoint);
               
                String offer = "v=0 o=- 0 0 IN IP4 " + rpiaddr + " s= c=IN IP4 " + rpiaddr + " t=0 0 m=video 9011 RTP/AVP 100 a=rtpmap:100 H264/90000 a=recvonly";
                String responseSdp = rtpEp.processOffer(offer);
               
                return responseSdp;
 }


}

When I launch the application, apparently there is an error when processing the offer (Code: 40205, Type:null).

Any ideas on why this might be happening?

Here's the stack trace from the terminal where I ran the modified HelloWorld tutorial

13:51:16.652 [HelloWorldApp.main()] INFO  - o.k.tutorial.helloworld.HelloWorldApp    - Starting HelloWorldApp on ignacioavellino with PID 8138 (/home/ignacioavellino/kurento-tutorial-java/kurento-hello-world2/target/classes started by ignacioavellino in /home/ignacioavellino/kurento-tutorial-java/kurento-hello-world2)

13:51:16.669 [HelloWorldApp.main()] DEBUG - o.k.tutorial.helloworld.HelloWorldApp    - Running with Spring Boot v1.2.0.RELEASE, Spring v4.1.3.RELEASE

13:51:22.494 [HelloWorldApp.main()] INFO  - o.h.validator.internal.util.Version      - HV000001: Hibernate Validator 5.1.3.Final

13:51:27.468 [HelloWorldApp.main()] DEBUG - org.kurento.client.KurentoClient         - Connecting to kms in uri ws://localhost:8888/kurento

13:51:28.929 [HelloWorldApp.main()] INFO  - o.k.tutorial.helloworld.HelloWorldApp    - Started HelloWorldApp in 13.196 seconds (JVM running for 39.304)

13:51:40.696 [http-nio-8080-exec-8] DEBUG - o.k.j.client.JsonRpcClientWebSocket      - Req-> {"id":1,"method":"create","params":{"type":"MediaPipeline","constructorParams":{}},"jsonrpc":"2.0"}

13:51:40.716 [http-nio-8080-exec-8] DEBUG - o.k.j.client.JsonRpcClientWebSocket      - <-Res {"id":1,"result":{"value":"0f9aa671-51cd-49fe-961f-d70f86ef09b4_MediaPipeline","sessionId":"e20576ee-d625-4167-8ebe-9ff074d7db7e"},"jsonrpc":"2.0"}

13:51:41.296 [http-nio-8080-exec-8] DEBUG - o.k.j.client.JsonRpcClientWebSocket      - Req-> {"id":2,"method":"create","params":{"type":"RtpEndpoint","constructorParams":{"mediaPipeline":"0f9aa671-51cd-49fe-961f-d70f86ef09b4_MediaPipeline"},"sessionId":"e20576ee-d625-4167-8ebe-9ff074d7db7e"},"jsonrpc":"2.0"}

13:51:41.582 [http-nio-8080-exec-8] DEBUG - o.k.j.client.JsonRpcClientWebSocket      - <-Res {"id":2,"result":{"value":"0f9aa671-51cd-49fe-961f-d70f86ef09b4_MediaPipeline/644c2019-2996-4d6b-a3e7-098f41f4c6f3_RtpEndpoint","sessionId":"e20576ee-d625-4167-8ebe-9ff074d7db7e"},"jsonrpc":"2.0"}

13:51:41.738 [http-nio-8080-exec-8] DEBUG - o.k.j.client.JsonRpcClientWebSocket      - Req-> {"id":3,"method":"create","params":{"type":"WebRtcEndpoint","constructorParams":{"mediaPipeline":"0f9aa671-51cd-49fe-961f-d70f86ef09b4_MediaPipeline"},"sessionId":"e20576ee-d625-4167-8ebe-9ff074d7db7e"},"jsonrpc":"2.0"}

13:51:41.918 [http-nio-8080-exec-8] DEBUG - o.k.j.client.JsonRpcClientWebSocket      - <-Res {"id":3,"result":{"value":"0f9aa671-51cd-49fe-961f-d70f86ef09b4_MediaPipeline/502d6885-9dcb-4130-b505-352d2f9d0c80_WebRtcEndpoint","sessionId":"e20576ee-d625-4167-8ebe-9ff074d7db7e"},"jsonrpc":"2.0"}

13:51:42.067 [http-nio-8080-exec-8] DEBUG - o.k.c.i.c.RemoteObjectInvocationHandler  - Invoking method public abstract void org.kurento.client.MediaElement.connect(org.kurento.client.MediaElement) on object [RemoteObject: type=WebRtcEndpoint remoteRef=0f9aa671-51cd-49fe-961f-d70f86ef09b4_MediaPipeline/502d6885-9dcb-4130-b505-352d2f9d0c80_WebRtcEndpoint

13:51:42.114 [http-nio-8080-exec-8] DEBUG - o.k.j.client.JsonRpcClientWebSocket      - Req-> {"id":4,"method":"invoke","params":{"object":"0f9aa671-51cd-49fe-961f-d70f86ef09b4_MediaPipeline/502d6885-9dcb-4130-b505-352d2f9d0c80_WebRtcEndpoint","operation":"connect","operationParams":{"sink":"0f9aa671-51cd-49fe-961f-d70f86ef09b4_MediaPipeline/644c2019-2996-4d6b-a3e7-098f41f4c6f3_RtpEndpoint"},"sessionId":"e20576ee-d625-4167-8ebe-9ff074d7db7e"},"jsonrpc":"2.0"}

13:51:42.117 [http-nio-8080-exec-8] DEBUG - o.k.j.client.JsonRpcClientWebSocket      - <-Res {"id":4,"result":{"sessionId":"e20576ee-d625-4167-8ebe-9ff074d7db7e"},"jsonrpc":"2.0"}

13:51:42.123 [http-nio-8080-exec-8] DEBUG - o.k.c.i.c.RemoteObjectInvocationHandler  - Invoking method public abstract void org.kurento.client.MediaElement.connect(org.kurento.client.MediaElement) on object [RemoteObject: type=RtpEndpoint remoteRef=0f9aa671-51cd-49fe-961f-d70f86ef09b4_MediaPipeline/644c2019-2996-4d6b-a3e7-098f41f4c6f3_RtpEndpoint

13:51:42.124 [http-nio-8080-exec-8] DEBUG - o.k.j.client.JsonRpcClientWebSocket      - Req-> {"id":5,"method":"invoke","params":{"object":"0f9aa671-51cd-49fe-961f-d70f86ef09b4_MediaPipeline/644c2019-2996-4d6b-a3e7-098f41f4c6f3_RtpEndpoint","operation":"connect","operationParams":{"sink":"0f9aa671-51cd-49fe-961f-d70f86ef09b4_MediaPipeline/502d6885-9dcb-4130-b505-352d2f9d0c80_WebRtcEndpoint"},"sessionId":"e20576ee-d625-4167-8ebe-9ff074d7db7e"},"jsonrpc":"2.0"}

13:51:42.127 [http-nio-8080-exec-8] DEBUG - o.k.j.client.JsonRpcClientWebSocket      - <-Res {"id":5,"result":{"sessionId":"e20576ee-d625-4167-8ebe-9ff074d7db7e"},"jsonrpc":"2.0"}

13:51:42.130 [http-nio-8080-exec-8] DEBUG - o.k.c.i.c.RemoteObjectInvocationHandler  - Invoking method public abstract java.lang.String org.kurento.client.SdpEndpoint.processOffer(java.lang.String) on object [RemoteObject: type=RtpEndpoint remoteRef=0f9aa671-51cd-49fe-961f-d70f86ef09b4_MediaPipeline/644c2019-2996-4d6b-a3e7-098f41f4c6f3_RtpEndpoint

13:51:42.132 [http-nio-8080-exec-8] DEBUG - o.k.j.client.JsonRpcClientWebSocket      - Req-> {"id":6,"method":"invoke","params":{"object":"0f9aa671-51cd-49fe-961f-d70f86ef09b4_MediaPipeline/644c2019-2996-4d6b-a3e7-098f41f4c6f3_RtpEndpoint","operation":"processOffer","operationParams":{"offer":"v=0 o=- 0 0 IN IP4 192.168.1.105 s= c=IN IP4 192.168.1.105 t=0 0 m=video 9011 RTP/AVP 100 a=rtpmap:100 H264/90000 a=recvonly"},"sessionId":"e20576ee-d625-4167-8ebe-9ff074d7db7e"},"jsonrpc":"2.0"}

13:51:42.139 [http-nio-8080-exec-8] DEBUG - o.k.j.client.JsonRpcClientWebSocket      - <-Res {"id":6,"error":{"code":40205,"message":"Error processing offer","data":{"type":"SDP_END_POINT_PROCESS_OFFER_ERROR"}},"jsonrpc":"2.0"}

13:51:42.192 [http-nio-8080-exec-8] ERROR - o.a.c.c.C.[.[.[/].[dispatcherServlet]    - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.kurento.client.internal.server.KurentoServerException: Error processing offer (Code:40205, Type:null)] with root cause

org.kurento.client.internal.server.KurentoServerException: Error processing offer (Code:40205, Type:null)

at org.kurento.client.internal.transport.jsonrpc.RomClientJsonRpcClient.sendRequest(RomClientJsonRpcClient.java:268) ~[kurento-client-5.1.0.jar:5.1.0]

at org.kurento.client.internal.transport.jsonrpc.RomClientJsonRpcClient.invoke(RomClientJsonRpcClient.java:137) ~[kurento-client-5.1.0.jar:5.1.0]

at org.kurento.client.internal.transport.jsonrpc.RomClientJsonRpcClient.invoke(RomClientJsonRpcClient.java:84) ~[kurento-client-5.1.0.jar:5.1.0]

at org.kurento.client.internal.client.RomManager.invoke(RomManager.java:105) ~[kurento-client-5.1.0.jar:5.1.0]

at org.kurento.client.internal.client.RemoteObject.invoke(RemoteObject.java:166) ~[kurento-client-5.1.0.jar:5.1.0]

at org.kurento.client.internal.client.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:156) ~[kurento-client-5.1.0.jar:5.1.0]

at org.kurento.client.internal.client.RemoteObjectInvocationHandler.internalInvoke(RemoteObjectInvocationHandler.java:122) ~[kurento-client-5.1.0.jar:5.1.0]

at org.kurento.client.internal.client.DefaultInvocationHandler.invoke(DefaultInvocationHandler.java:24) ~[kurento-client-5.1.0.jar:5.1.0]

at com.sun.proxy.$Proxy60.processOffer(Unknown Source) ~[na:na]

at org.kurento.tutorial.helloworld.HelloWorldController.processRequest(HelloWorldController.java:53) ~[classes/:na]

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_79]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_79]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_79]

at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_79]

at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221) ~[spring-web-4.1.3.RELEASE.jar:4.1.3.RELEASE]

at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137) ~[spring-web-4.1.3.RELEASE.jar:4.1.3.RELEASE]

at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110) ~[spring-webmvc-4.1.3.RELEASE.jar:4.1.3.RELEASE]

at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:777) ~[spring-webmvc-4.1.3.RELEASE.jar:4.1.3.RELEASE]

at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:706) ~[spring-webmvc-4.1.3.RELEASE.jar:4.1.3.RELEASE]

at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.1.3.RELEASE.jar:4.1.3.RELEASE]

at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:943) ~[spring-webmvc-4.1.3.RELEASE.jar:4.1.3.RELEASE]

at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:877) ~[spring-webmvc-4.1.3.RELEASE.jar:4.1.3.RELEASE]

at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966) ~[spring-webmvc-4.1.3.RELEASE.jar:4.1.3.RELEASE]

at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:868) ~[spring-webmvc-4.1.3.RELEASE.jar:4.1.3.RELEASE]

at javax.servlet.http.HttpServlet.service(HttpServlet.java:644) ~[tomcat-embed-core-8.0.15.jar:8.0.15]

at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842) ~[spring-webmvc-4.1.3.RELEASE.jar:4.1.3.RELEASE]

at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) ~[tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) ~[tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-embed-websocket-8.0.15.jar:8.0.15]

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.15.jar:8.0.15]

at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) ~[spring-web-4.1.3.RELEASE.jar:4.1.3.RELEASE]

at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.1.3.RELEASE.jar:4.1.3.RELEASE]

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.15.jar:8.0.15]

at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77) ~[spring-web-4.1.3.RELEASE.jar:4.1.3.RELEASE]

at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) ~[spring-web-4.1.3.RELEASE.jar:4.1.3.RELEASE]

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) ~[tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) ~[tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) ~[tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) [tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501) [tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142) [tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) [tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) [tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:537) [tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1085) [tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:658) [tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:222) [tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1556) [tomcat-embed-core-8.0.15.jar:8.0.15]

at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1513) [tomcat-embed-core-8.0.15.jar:8.0.15]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79]

at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.0.15.jar:8.0.15]

at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]

Thanks in advance!!

—N

[ignacio...@gmail.com]

Hi Ivan!

The error was produced by having the offer string as one big long line. You need to chop it into pieces with an end of line character between params. Sigh, I was pulling my hair out.

With this string the error is gone:

String offer = "v=0\n" +

                        "o=- 0 0 IN IP4 192.168.1.105\n" +

                        "s=\n" +

                        "c=IN IP4 192.168.1.105\n" +

                        "t=0 0\n" +

                        "m=video 9011 RTP/AVP 100\n" +

                        "a=rtpmap:100 H264/90000\n" + 

                        "a=recvonly";

Now I have a "Failed to set remote answer sdp: Called with SDP without DTLS fingerprint." error ... diving into solving that

—N

...

[Ivan Gracia]

Hey! Thanks for the feedback, and sorry for that ;-) You are getting that error when invoking the processOffer(offer)?

Ivan Gracia

[ignacio...@gmail.com]

I get the error on the console of the webpage of my modified HelloWorld example... so it's a message returned by the server. The console reads:

Loading complete ...

Starting video call ...

Created SDP offer

Local description set

ICE negotiation completed

Invoking SDP offer callback function 192.168.1.106:8080

Received sdpAnswer from server. Processing ...

SDP answer received, setting remote description

Failed to set remote answer sdp: Called with SDP without DTLS fingerprint.

processOffer finishes correctly, and if I print the result string it's:

v=0

o=- 0 3641121252 IN IP4 192.168.1.106

s=Kurento Media Server

c=IN IP4 192.168.1.106

t=0 0

m=video 47788 RTP/AVP 100

a=rtpmap:100 H264/90000

a=sendonly

a=ssrc:2008377309 cname:user1202858140@host-c87339cb

—N

[Ivan Gracia]

Could it be that you are mixing the SDPs? The one you have manually created in your string goes to the RTP endpoint. The webrtc endpoint has to go through the regular SDP dance with your browser's RTCPeerConnection, wrapped in a webrtcPeer object form the kurento-utils library (if you are using it)

Ivan Gracia

[ignacio...@gmail.com]

Hi,

You're right, sorry about that. I completely misunderstood the process.

But now what should I return to the browser? The answer from the RTPEndpoint once it processes the manually created offer?

[ignacio...@gmail.com]

Hmm .. I guess the reply from processing the WebRTC SDP offer with the WebRTCEndpoint

I don't know why it's not working ... I've fixed my setup like they said today in this post

But still no chance. Maybe it's my gstreamer that is not sending the media to the KMS.

The command I'm running is

$ raspivid -n -t 0 -fps 30 -hf -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay ! udpsink host=192.168.1.112 port=9011

Is there a way to look at the output of the console of the KMS, to see if there is a connection incoming that was detected?

Thanks in advance,

—N

...

[Ivan Gracia]

If you are using the dev version of KMS, endpoints have a getMediaState method that indicates if there is media flowing through the endpoint.

Maybe your problem is with the WebRTC endpoint with the client? Try connecting a recorder to the rtp and see if there is media being recorded, so you can focus on one or the other.

--

[ignacio...@gmail.com]

Hum, I'm not using the dev version, I don't have the method

What seems odd to me is that after executing the processOffer method, there is no port open listening on the port I specified

My SDP file specifies port 9011, and still after processing the offer, when I run a netstat, there is no process listening on 9011 for my RTP connection to plug in ...

                String offer = "v=0\n" +

                        "o=- 0 0 IN IP6 0.0.0.0\n" +

                        "s=\n" +

                        "c=IN IP6 0.0.0.0\n" +

                        "t=0 0\n" +

                        "m=video 9011 RTP/AVP 96\n" +

                        "a=rtpmap:96 H264/90000\n" + 

                        "a=recvonly";

[Ivan Gracia]

The offer tells the rtp in KMS where it can find the video. You should be setting an IP in that SDP, more precisely the IP of your R-PI.

Ivan Gracia

[ignacio...@gmail.com]

I'm sorry about the wrong copy paste, I was trying many different things

You're absolutely right. My raspi is on 192.168.1.113, and the offer I'm using is:

String offer = "v=0\n" +

                        "o=- 0 0 IN IP4 192.168.1.113\n" +

                        "s=\n" +

                        "c=IN IP4 192.168.1.113\n" +

                        "t=0 0\n" +

                        "m=video 9011 RTP/AVP 96\n" +

                        "a=rtpmap:96 H264/90000\n" + 

                        "a=recvonly";

The generated answer is:

v=0

o=- 0 3641626268 IN IP4 192.168.1.112

s=Kurento Media Server

c=IN IP4 192.168.1.112

t=0 0

m=video 46882 RTP/AVP 96

a=rtpmap:96 H264/90000

a=sendonly

a=ssrc:395282829 cname:user739796993@host-772d18b5

Which is ok, since the KMS is in 192.168.1.112

And after I execute the processOffer (I see some logs in the console after this line such that I'm sure it's executed), when I check the ports the 9011 is not open

$ sudo netstat -plunt

Active Internet connections (only servers)

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name

tcp        0      0 127.0.1.1:53            0.0.0.0:*               LISTEN      970/dnsmasq     

tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      467/cupsd       

tcp6       0      0 :::8080                 :::*                    LISTEN      4614/java       

tcp6       0      0 :::8888                 :::*                    LISTEN      4586/kurento-media-

tcp6       0      0 ::1:42664               :::*                    LISTEN      2333/java       

udp        0      0 127.0.1.1:53            0.0.0.0:*                           970/dnsmasq     

udp        0      0 0.0.0.0:68              0.0.0.0:*                           4245/dhclient   

udp        0      0 0.0.0.0:57448           0.0.0.0:*                           472/avahi-daemon: r

udp        0      0 0.0.0.0:5353            0.0.0.0:*                           472/avahi-daemon: r

udp        0      0 0.0.0.0:40374           0.0.0.0:*                           4586/kurento-media-

udp        0      0 0.0.0.0:40375           0.0.0.0:*                           4586/kurento-media-

udp        0      0 0.0.0.0:12729           0.0.0.0:*                           4245/dhclient   

udp        0      0 0.0.0.0:631             0.0.0.0:*                           661/cups-browsed

udp        0      0 0.0.0.0:46882           0.0.0.0:*                           4586/kurento-media-

udp        0      0 0.0.0.0:46883           0.0.0.0:*                           4586/kurento-media-

udp6       0      0 :::52245                :::*                                472/avahi-daemon: r

udp6       0      0 :::5353                 :::*                                472/avahi-daemon: r

udp6       0      0 :::43308                :::*                                4245/dhclient   

This makes sense, the KMS will be sending out the RTP broadcast on port 46882 according to the answer, and that port is open ... but why is 9011 not open? KMS seems not to be listening on the port that the raspi is sending the stream

Puzzles me ...

—N

...

[Ivan Gracia]

9011 is the port where the R-PI is serving the video, so KMS will connect to 192.168.1.113:9091. That port does not have to be listening in KMS, but in R-PI.

Ivan Gracia

--

You received this message because you are subscribed to the Google Groups "kurento" group.

[ignacio...@gmail.com]

oh... Ok that changes everything Ivan, many many thanks for the clarification!!!

The thing is that when I run on the raspi:

$ raspivid -n -t 0 -fps 30 -hf -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay ! udpsink host=192.168.1.112 port=9011

, gstreamer will SEND the data to 192.168.1.112:9011 (the KMS server)

I need to then figure out a way such that my raspi is listening in the port 9011 and then reply with the stream once the KMS requests ... if I understood correctly

...

[ignacio...@gmail.com]

Ivan,

IT'S WORKING finally. But, it's a bit hacked

First I process the offer with the RTP Endpoint, and then I look at the response

Offer:

String offer = "v=0\n" +

                        "o=- 0 0 IN IP4 192.168.1.113\n" +

                        "s=\n" +

                        "c=IN IP4 192.168.1.113\n" +

                        "t=0 0\n" +

                        "m=video 9011 RTP/AVP 96\n" +

                        "a=rtpmap:96 H264/90000\n" + 

                        "a=recvonly";

Reply:

v=0

o=- 0 3641640407 IN IP4 192.168.1.112

s=Kurento Media Server

c=IN IP4 192.168.1.112

t=0 0

m=video 35918 RTP/AVP 96

a=rtpmap:96 H264/90000

a=sendonly

a=ssrc:3133676721 cname:user3979159749@host-3f876194

When I look at the ports, Kurento is listening on port 35918, so I plug my gstreamer output there by running on the raspberry pi:

$ raspivid -n -t 0 -fps 30 -hf -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay ! udpsink host=192.18.1.112 port=35918

My question is:

Why is kurento listening on this random port?

Following your last reply: Kurento actually does not ping my raspberry pi on port 9011, if I listen on that port (using $nc -l 9011) nothing comes there, ever ... If I use vlc to serve an rtp stream on 9011, it's never requested by the KMS server

And last thing: it works but EXTREMELY slow. If I open the Kurento logfile, I see:

** (kurento-media-server:6630): CRITICAL **: gst_video_encoder_set_latency: assertion 'GST_CLOCK_TIME_IS_VALID (min_latency)' failed

The gst-launch is sending at clock rate 90000 (I can see it in the terminal when I launch the command) and we are using the same clock rate (it's on the RTP offer)

Someone has already spotted this issue here. I've tried this answer, but it doesn't help.

So to summarize

- It works, but ideally I should solve the port hack, since I can't enter it manually on the raspi every time

- RTPEndpoint opens a random port on KMS, and it does not send a rq to the raspberry pi on 9011

- It's mega slow.. but again, this issue has already been reported in April by someone from the Kurento team

—N

... 

[extr...@gmail.com]

Hello ignacio and Ivan!

Ignacio, your post is very helpful for me, thank you for your posting!

I'm trying same thing according to your post, but unfortunately I cannot watch the video on my web browser.

Could you please tell me what version of your software you are using?

I'm using following software:

Raspberry Pi:

  Linux raspberrypi 3.18.7+ #755 PREEMPT Thu Feb 12 17:14:31 GMT 2015 armv6l GNU/Linux

  raspivid Camera App v1.3.12

  gstreamer-1.0

Kurento Media Server:

$ kurento-media-server -v

Version: 5.1.4~2.g75f094f

Found modules:

        Module: 'core' version '5.1.4'

        Module: 'elements' version '5.1.3'

        Module: 'filters' version '5.1.1'

When I run the following command, the KMS outputs some warnings.

pi@raspberrypi ~ $ raspivid -n -t 0 -fps 30 -hf -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay ! udpsink host=192.168.254.40 port=51898

media-server.log

2015-05-27 12:24:00.936693 7080 [140466487060224] warning kmsutils                  kmsutils.c:314 gap_detection_probe() <kmsagnosticbin2-2:sink> Gap detected

2015-05-27 12:24:00.992386 7080 [140466687301376] warning rtpjitterbuffer           rtpjitterbuffer.c:449 calculate_skew() resync to time 0:01:51.609009347, rtptime 11:54:32.566266666

2015-05-27 12:24:00.992604 7080 [140466487060224] warning kmsutils                  kmsutils.c:314 gap_detection_probe() <kmsagnosticbin2-2:sink> Gap detected

2015-05-27 12:24:01.029325 7080 [140466687301376] warning rtpjitterbuffer           rtpjitterbuffer.c:449 calculate_skew() resync to time 0:01:51.645949577, rtptime 11:54:32.566266666

The following command works good, I can see the SMPTE color bar on my web browser.

gst-launch-1.0 -v \

  videotestsrc pattern=smpte100 \

  ! "video/x-raw,width=(int)640,height=(int)480" \

  ! x264enc \

  ! "video/x-h264,stream-format=(string)avc,profile=(string)baseline" \

  ! rtph264pay \

  ! udpsink host=$1 port=$2

I would really appreciate if you reply to me.

Thanks!

[ignacio...@gmail.com]

Hey there,

Glad someone is also using this setup

I'm using a raspi 1, rev2 (How do you find out the version as you posted it btw?)

$ raspivid -v

v1.3.12

$ gst-launch-1.0 --version

version 1.2.0

$ kurento-media-server -v

5.1.4~1.g595d65f

Regarding the command you use, I use the same one as you saw in previous posts

I've never seen that error, Gap detected. It says something about skew, that's odd. Try specifying in your raspivid command specific sizes and see what happens with different sizes.

Check again the SDP offer you're using. How different is it from the one I posted earlier?

String offer = "v=0\n" +

                        "o=- 0 0 IN IP4 192.168.1.113\n" +

                        "s=\n" +

                        "c=IN IP4 192.168.1.113\n" +

                        "t=0 0\n" +

                        "m=video 9011 RTP/AVP 96\n" +

                        "a=rtpmap:96 H264/90000\n" + 

                        "a=recvonly";

Sorry I can't help you more, I'm only getting started with Kurento ...

—N

[extr...@gmail.com]

Hello Ignacio,

Thank you for your reply!

| I'm using a raspi 1, rev2 (How do you find out the version as you posted it btw?)

"uname -a" shows the version of the running kernel.

pi@raspberrypi ~ $ uname -a

Linux raspberrypi 3.18.7+ #755 PREEMPT Thu Feb 12 17:14:31 GMT 2015 armv6l GNU/Linux

| I've never seen that error, Gap detected. It says something about skew, that's odd. Try specifying in your raspivid command specific sizes and see what happens with different sizes.

|

| Check again the SDP offer you're using. How different is it from the one I posted earlier?

Hmm, My environment is almost the same version as yours.

I'll try them according to your advice.

Here is my sdp offer,  I'll try your sdp offer except the IP addresses.

    offer  = "v=0\r\n";

    offer += "o=- 12345 12345 IN IP4 192.168.1.162\r\n";

    offer += "s=-\r\n";

    offer += "c=IN IP4 192.168.1.162\r\n";

    offer += "t=0 0\r\n";

    offer += "m=video 30010 RTP/AVP 96\r\n";

    offer += "a=rtpmap:96 H264/90000\r\n";

    offer += "a=recvonly\r\n";

| Sorry I can't help you more, I'm only getting started with Kurento ...

Your post is very helpful!

I really really appreciate it!

[Ritesh Patel]

ignacio so finally you solve problem right congo can you share your code so it will be useful for others. i am working on same type of project where raspberry pi send video stream to kms server and do some processing back to client.

[Ignacio Avellino]

Hi Ritesh.

This was back in 2015 .. I am not using Kurento anymore, I'm really sorry. I don't have access to the code, and probably by now it's deprecated ..

Best of luck!

[Israel Robotnick]

Hi Ignacio

May I ask what do u use instead of kurento?
Does it have better latency or operates on better resolutions?

Thanks