rtp reciever java tutorial

[saksh...@gmail.com]

Hello, 

I was trying this example following the steps (apart from using SpringBoot to deploy the webservice, I have bundled a war and deployed it on Tomcat) mentioned on tag 6.7.1 of the code base. When I click on Start I get the following exception - 

Problem 1 -

org.kurento.client.internal.server.KurentoServerException: Event not found (Code:40106, Type:null, Data: {"type":"MEDIA_OBJECT_EVENT_NOT_SUPPORTED"})

at org.kurento.client.internal.transport.jsonrpc.RomClientJsonRpcClient.sendRequest(RomClientJsonRpcClient.java:264) ~[kurento-client-6.7.1.jar:6.7.1]

at org.kurento.client.internal.transport.jsonrpc.RomClientJsonRpcClient.subscribe(RomClientJsonRpcClient.java:130) ~[kurento-client-6.7.1.jar:6.7.1]

at org.kurento.client.internal.transport.jsonrpc.RomClientJsonRpcClient.subscribe(RomClientJsonRpcClient.java:122) ~[kurento-client-6.7.1.jar:6.7.1]

at org.kurento.client.internal.client.RomManager.subscribe(RomManager.java:185) ~[kurento-client-6.7.1.jar:6.7.1]

at org.kurento.client.internal.client.RemoteObject.addEventListener(RemoteObject.java:252) ~[kurento-client-6.7.1.jar:6.7.1]

at org.kurento.client.internal.client.RemoteObjectInvocationHandler.subscribeEventListener(RemoteObjectInvocationHandler.java:197) ~[kurento-client-6.7.1.jar:6.7.1]

at org.kurento.client.internal.client.RemoteObjectInvocationHandler.internalInvoke(RemoteObjectInvocationHandler.java:119) ~[kurento-client-6.7.1.jar:6.7.1]

at org.kurento.client.internal.client.DefaultInvocationHandler.invoke(DefaultInvocationHandler.java:39) ~[kurento-client-6.7.1.jar:6.7.1]

at com.sun.proxy.$Proxy67.addMediaTranscodingStateChangeListener(Unknown Source) ~[na:na]

at org.kurento.tutorial.player.PlayerHandler.addCommonEventListeners(PlayerHandler.java:168) [classes/:na]

at org.kurento.tutorial.player.PlayerHandler.startWebRtcEndpoint(PlayerHandler.java:422) [classes/:na]

at org.kurento.tutorial.player.PlayerHandler.start(PlayerHandler.java:536) [classes/:na]

at org.kurento.tutorial.player.PlayerHandler.handleTextMessage(PlayerHandler.java:90) [classes/:na]

at org.springframework.web.socket.handler.AbstractWebSocketHandler.handleMessage(AbstractWebSocketHandler.java:43) [spring-websocket-4.3.3.RELEASE.jar:4.3.3.RELEASE]

at org.springframework.web.socket.handler.WebSocketHandlerDecorator.handleMessage(WebSocketHandlerDecorator.java:75) [spring-websocket-4.3.3.RELEASE.jar:4.3.3.RELEASE]

at org.springframework.web.socket.handler.LoggingWebSocketHandlerDecorator.handleMessage(LoggingWebSocketHandlerDecorator.java:56) [spring-websocket-4.3.3.RELEASE.jar:4.3.3.RELEASE]

at org.springframework.web.socket.handler.ExceptionWebSocketHandlerDecorator.handleMessage(ExceptionWebSocketHandlerDecorator.java:58) [spring-websocket-4.3.3.RELEASE.jar:4.3.3.RELEASE]

at org.springframework.web.socket.adapter.standard.StandardWebSocketHandlerAdapter.handleTextMessage(StandardWebSocketHandlerAdapter.java:110) [spring-websocket-4.3.3.RELEASE.jar:4.3.3.RELEASE]

at org.springframework.web.socket.adapter.standard.StandardWebSocketHandlerAdapter.access$000(StandardWebSocketHandlerAdapter.java:42) [spring-websocket-4.3.3.RELEASE.jar:4.3.3.RELEASE]

at org.springframework.web.socket.adapter.standard.StandardWebSocketHandlerAdapter$3.onMessage(StandardWebSocketHandlerAdapter.java:81) [spring-websocket-4.3.3.RELEASE.jar:4.3.3.RELEASE]

at org.springframework.web.socket.adapter.standard.StandardWebSocketHandlerAdapter$3.onMessage(StandardWebSocketHandlerAdapter.java:78) [spring-websocket-4.3.3.RELEASE.jar:4.3.3.RELEASE]

at org.apache.tomcat.websocket.WsFrameBase.sendMessageText(WsFrameBase.java:399) [tomcat8-websocket-8.0.32.jar:8.0.32]

at org.apache.tomcat.websocket.WsFrameBase.processDataText(WsFrameBase.java:500) [tomcat8-websocket-8.0.32.jar:8.0.32]

at org.apache.tomcat.websocket.WsFrameBase.processData(WsFrameBase.java:295) [tomcat8-websocket-8.0.32.jar:8.0.32]

at org.apache.tomcat.websocket.WsFrameBase.processInputBuffer(WsFrameBase.java:131) [tomcat8-websocket-8.0.32.jar:8.0.32]

at org.apache.tomcat.websocket.server.WsFrameServer.onDataAvailable(WsFrameServer.java:71) [tomcat8-websocket-8.0.32.jar:8.0.32]

at org.apache.tomcat.websocket.server.WsHttpUpgradeHandler$WsReadListener.onDataAvailable(WsHttpUpgradeHandler.java:186) [tomcat8-websocket-8.0.32.jar:8.0.32]

at org.apache.coyote.http11.upgrade.AbstractServletInputStream.onDataAvailable(AbstractServletInputStream.java:198) [tomcat8-coyote-8.0.32.jar:8.0.32]

at org.apache.coyote.http11.upgrade.AbstractProcessor.upgradeDispatch(AbstractProcessor.java:96) [tomcat8-coyote-8.0.32.jar:8.0.32]

at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:647) [tomcat8-coyote-8.0.32.jar:8.0.32]

at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1520) [tomcat8-coyote-8.0.32.jar:8.0.32]

at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1476) [tomcat8-coyote-8.0.32.jar:8.0.32]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_151]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151]

at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat8-util-8.0.32.jar:8.0.32]

at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151]

The highlighted line corresponds to media transcoding state change event.

Problem 2 (not exactly related to Kurento, but would appreciate if someone can help as I think is related to the first problem) - 

I tried to comment out this transcoding event listener to just see what happens, and as expected this exception was suppressed. Now when I try to use Gstreamer (as mentioned in the documentation) to stream content over RDP to KMS, I get another error saying "erroneous pipeline: no element "rtpbin"". Do we have to enable individual plugin to use with Gstreamer? If so, how? Could problem 1 be related to problem 2?

Problem 3 -

Then I tried using VLC (using a .mp4 file) to stream over RDP. For a certain combination of options (of media files, destination setup, transcoding options) I was able to see that the media state change to FLOWING. Log line - 

2018-04-17 15:41:21.716  INFO 32702 --- [ventExec-e2-t45] o.kurento.tutorial.player.PlayerHandler  : [RtpEndpoint::MediaFlowOutStateChange] source: [RemoteObject: type=RtpEndpoint remoteRef=5231ca48-eb0a-42ad-a50b-3c480655dc28_kurento.MediaPipeline/c1c464ac-ff91-45f7-a65b-0b83417be77c_kurento.RtpEndpoint], timestamp: 1523959881, tags: [], state: FLOWING, padName: default, mediaType: VIDEO

However, the corresponding log on KMS says - 

2018-04-17 15:36:40,805472 5485 [0x00007f4e9d7e2700]   error rtpsynchronizer           kmsrtpsynchronizer.c:369 kms_rtp_synchronizer_process_rtp_buffer_mapped() <KmsRtpSynchronizer@0x7f4f1809c140>  Received an unsorted RTP buffer when expecting sorted (ssrc: 152698957, seq: 2230, ts: 3569738141, ext_ts: 3569738141). Moving to unsorted mode

Please note: streaming by ffmpeg also resulted in the same error. 

I have been stuck here for two days now, it would be great to get some insights.

Thanks,

S

[Sakshi]

An update -

1) Problem 1 and 2 has been resolved (or so it seems!). I used Ubuntu 14 this time and deployed the web application using Spring Boot.

Are there any known issues between Gstreamer and Ubuntu 16? I ask because previously on Ubuntu 16 I could not get Gstreamer to work correctly as mentioned here. A lot of plugins seemed to be missing because of which I kept getting "erroneous pipeline...".

I will update here as I find more explanations on this.

2) Problem 3 still persists. I am trying using ffmpeg: ffmpeg -re -i video_only.mp4 -vcodec h264 -b:v 90k -an -f rtp rtp://a.b.c.d:58040.

3) Note - I am trying this demo on AWS.

Any leads would be appreciated.

Thanks

[Juan Navarro]

Hi,

there are no known problems with running the examples under Ubuntu 16.04, so they should work fine.
However, I have improved small things about the documentation of this particular example, so you may want to follow the latest branch instead of the stable one:
http://doc-kurento.readthedocs.io/en/latest/tutorials/java/tutorial-rtp-receiver.html

there is only one gotcha: in the given instructions, please ignore the "git checkout" command (i.e. don't do "git checkout 6.7.2-dev"), I still have to study the way of conditionally removing that line in the documentation for the nightly builds, but keeping it for the stable releases. For now, just ignore it so you use the master branch of kurento-tutorial-java

[Sakshi]

Hello Juan, thanks a lot for the response. Now we have managed to move ahead from yesterday but have uncovered a few more problems. Sharing the updates here - 

For SRTP

1) On EC2 Ubuntu 14.04 we got the tutorial working. Here are the logs once I played the media using Gstreamer - 

2018-04-19 09:23:13.173  INFO 18767 --- [EventExec-e2-t9] o.kurento.tutorial.player.PlayerHandler  : [RtpEndpoint::MediaFlowOutStateChange] source: [RemoteObject: type=RtpEndpoint remoteRef=c35c0ec3-5c3b-42d4-a1b6-260adaf50454_kurento.MediaPipeline/43e34e68-a1fd-4625-b0fc-1483028ad29d_kurento.RtpEndpoint], timestamp: 1524129793, tags: [], state: FLOWING, padName: default, mediaType: VIDEO

2018-04-19 09:23:13.268  INFO 18767 --- [EventExec-e2-t9] o.kurento.tutorial.player.PlayerHandler  : [WebRtcEndpoint::MediaFlowInStateChange] source: [RemoteObject: type=WebRtcEndpoint remoteRef=c35c0ec3-5c3b-42d4-a1b6-260adaf50454_kurento.MediaPipeline/190760a6-5c36-4d7f-9601-328e9b682eb3_kurento.WebRtcEndpoint], timestamp: 1524129793, tags: [], state: FLOWING, padName: default, mediaType: VIDEO

2018-04-19 09:23:13.841  INFO 18767 --- [EventExec-e2-t9] o.kurento.tutorial.player.PlayerHandler  : [WebRtcEndpoint::MediaStateChanged] source: [RemoteObject: type=WebRtcEndpoint remoteRef=c35c0ec3-5c3b-42d4-a1b6-260adaf50454_kurento.MediaPipeline/190760a6-5c36-4d7f-9601-328e9b682eb3_kurento.WebRtcEndpoint], timestamp: 1524129793, tags: [], oldState: DISCONNECTED, newState: CONNECTED

2018-04-19 09:23:15.657  INFO 18767 --- [EventExec-e2-t9] o.kurento.tutorial.player.PlayerHandler  : [RtpEndpoint::MediaStateChanged] source: [RemoteObject: type=RtpEndpoint remoteRef=c35c0ec3-5c3b-42d4-a1b6-260adaf50454_kurento.MediaPipeline/43e34e68-a1fd-4625-b0fc-1483028ad29d_kurento.RtpEndpoint], timestamp: 1524129795, tags: [], oldState: DISCONNECTED, newState: CONNECTED

However when the stream finishes, these are the logs that I get - 

2018-04-19 09:29:48.126  INFO 18767 --- [ventExec-e2-t10] o.kurento.tutorial.player.PlayerHandler  : [RtpEndpoint::MediaStateChanged] source: [RemoteObject: type=RtpEndpoint remoteRef=c35c0ec3-5c3b-42d4-a1b6-260adaf50454_kurento.MediaPipeline/43e34e68-a1fd-4625-b0fc-1483028ad29d_kurento.RtpEndpoint], timestamp: 1524130188, tags: [], oldState: CONNECTED, newState: DISCONNECTED

2018-04-19 09:29:51.479  INFO 18767 --- [ventExec-e2-t10] o.kurento.tutorial.player.PlayerHandler  : [RtpEndpoint::MediaFlowOutStateChange] source: [RemoteObject: type=RtpEndpoint remoteRef=c35c0ec3-5c3b-42d4-a1b6-260adaf50454_kurento.MediaPipeline/43e34e68-a1fd-4625-b0fc-1483028ad29d_kurento.RtpEndpoint], timestamp: 1524130191, tags: [], state: NOT_FLOWING, padName: default, mediaType: VIDEO

2018-04-19 09:29:51.911  INFO 18767 --- [ventExec-e2-t10] o.kurento.tutorial.player.PlayerHandler  : [WebRtcEndpoint::MediaFlowInStateChange] source: [RemoteObject: type=WebRtcEndpoint remoteRef=c35c0ec3-5c3b-42d4-a1b6-260adaf50454_kurento.MediaPipeline/190760a6-5c36-4d7f-9601-328e9b682eb3_kurento.WebRtcEndpoint], timestamp: 1524130191, tags: [], state: NOT_FLOWING, padName: default, mediaType: VIDEO

Can you please help me understand the state of media flow through the media pipeline on Kurento? When the media flow starts through GStreamer, for the RTPEndpoint, why does the MediaFlowOutStateChange happens first (to Flowing), and later the MediaStateChanged changes to Connected? Is it just that the logs are getting printed out of order or is it how the media flows through the pipeline?

On the other hand, the logs when the media flow stops through Gstreamer make sense in the way that MediaStateChange is logged first (hence DISCONNECTED) and then the MediaFlowOutStateChange goes to NOT_FLOWING.

2) On EC2 Ubuntu 16.04 we coudn't  get the tutorial working. Here are the logs once I played the media using Gstreamer - 

2018-04-19 09:30:13.156  INFO 21454 --- [ventExec-e2-t25] o.kurento.tutorial.player.PlayerHandler  : [RtpEndpoint::MediaFlowOutStateChange] source: [RemoteObject: type=RtpEndpoint remoteRef=b6b5b63c-2ab4-459b-98b2-bef03ddf1465_kurento.MediaPipeline/ddd39736-067d-4119-ad35-b6027907ba2a_kurento.RtpEndpoint], timestamp: 1524130213, tags: [], state: FLOWING, padName: default, mediaType: VIDEO

2018-04-19 09:30:14.655  INFO 21454 --- [ventExec-e2-t25] o.kurento.tutorial.player.PlayerHandler  : [RtpEndpoint::MediaStateChanged] source: [RemoteObject: type=RtpEndpoint remoteRef=b6b5b63c-2ab4-459b-98b2-bef03ddf1465_kurento.MediaPipeline/ddd39736-067d-4119-ad35-b6027907ba2a_kurento.RtpEndpoint], timestamp: 1524130214, tags: [], oldState: DISCONNECTED, newState: CONNECTED

2018-04-19 09:31:09.450  INFO 21454 --- [ventExec-e2-t25] o.kurento.tutorial.player.PlayerHandler  : [RtpEndpoint::MediaStateChanged] source: [RemoteObject: type=RtpEndpoint remoteRef=b6b5b63c-2ab4-459b-98b2-bef03ddf1465_kurento.MediaPipeline/ddd39736-067d-4119-ad35-b6027907ba2a_kurento.RtpEndpoint], timestamp: 1524130269, tags: [], oldState: CONNECTED, newState: DISCONNECTED

2018-04-19 09:31:13.186  INFO 21454 --- [ventExec-e2-t25] o.kurento.tutorial.player.PlayerHandler  : [RtpEndpoint::MediaFlowOutStateChange] source: [RemoteObject: type=RtpEndpoint remoteRef=b6b5b63c-2ab4-459b-98b2-bef03ddf1465_kurento.MediaPipeline/ddd39736-067d-4119-ad35-b6027907ba2a_kurento.RtpEndpoint], timestamp: 1524130273, tags: [], state: NOT_FLOWING, padName: default, mediaType: VIDEO

I followed the tutorial like for both the OSes. The only difference in v16 and v14 that I encountered while setting up was v16 is on Java 8 while v14 is on Java 7. Could this be a problem?

For RTP (Comedia)

3) On both 14 and 16, I could not get the tutorials to work.  Sample of logs from Ubuntu 14 - 

Signalling Server logs - 

2018-04-19 09:53:22.301  INFO 18767 --- [ventExec-e2-t44] o.kurento.tutorial.player.PlayerHandler  : [RtpEndpoint::MediaStateChanged] source: [RemoteObject: type=RtpEndpoint remoteRef=2cbbd2f9-022a-4550-8d83-66ef548d3dfb_kurento.MediaPipeline/9bb89c28-91cd-4e60-836c-6dd1be80dd75_kurento.RtpEndpoint], timestamp: 1524131602, tags: [], oldState: DISCONNECTED, newState: CONNECTED

KMS logs - 

2018-04-19 09:52:35,907358 18752 [0x00007f31ea015700]    info KurentoWebSocketTransport WebSocketTransport.cpp:258 keepAliveSessions()  Keep alive 316f7871-2fce-4cae-ac3f-09cc581ab063

2018-04-19 09:53:24,927245 18752 [0x00007f31b6840700] warning rtpendpoint               kmsrtpendpoint.c:813 kms_rtp_endpoint_comedia_on_ssrc_active() <RTPSession@0x7f31cc1593c0>  COMEDIA: 'rtp-from' not available yet

2018-04-19 09:53:28,868226 18752 [0x00007f31b6840700] warning rtpendpoint               kmsrtpendpoint.c:813 kms_rtp_endpoint_comedia_on_ssrc_active() <RTPSession@0x7f31cc1593c0>  COMEDIA: 'rtp-from' not available yet

2018-04-19 09:53:35,907606 18752 [0x00007f31ea015700]    info KurentoWebSocketTransport WebSocketTransport.cpp:258 keepAliveSessions()  Keep alive 316f7871-2fce-4cae-ac3f-09cc581ab063

2018-04-19 09:54:01,315194 18752 [0x00007f31ec81a700]    info KurentoServerMethods      ServerMethods.cpp:795 ping()  WebSocket Ping/Pong with sessionId 316f7871-2fce-4cae-ac3f-09cc581ab063

One noticable difference between the SRTP and RTP way is that SRTP uses gst-launch-1.5 while RTP uses gst-launch-1.0, but I doubt if this is causing any problems since in both the versions of gst-launch I could see the details of the stream that were streamed on the RTP port.

Also is there a tutorial for doing this with audio over SRTP? Our use case is only audio, but the video tutorial is a great place to start.

Thanks, 

Sakshi

[Juan Navarro]

Hi,

The events are explained here: http://doc-kurento.readthedocs.io/en/latest/features/events.html
And also there is a bit more of documentation in the corresponding Client API JavaDocs.

Basically, FLOWING events describe actual flow of packets through the Endpoint in/out pads; they are a reliable way to know if there is actual data travelling through the Media Pipeline. Meanwhile, CONNECTED events are related to RTCP packets; they indicate whether the RTP tracks (audio / video) are still active or not at the transport level (i.e. if the corresponding RTCP packets are still being sent). RTCP packets are sent in specific intervals (e.g. once each 5 seconds), so CONNECTED events usually take some time to appear.

There is no reason that the exaple shouldn't work in Ubuntu 16; I myself do tests in a clean Ubuntu 16.04 virtualbox machine. At the minimum, the example should work locally in your development machine, and upgrading it from there to an AWS machine should be a problem of sysadmin, not application code.

The logs you show:

COMEDIA: 'rtp-from' not available yet

indicate that KMS hasn't yet received RTCP Sender Reports packets from the RTP sender (the gst-launch application). This means that the gst-launch application is either not sending RTCP packets, or that they are not arriving properly to KMS. In any case, the path from KMS to the sender is only used in this tutorial to send RTCP Receiver Reports from KMS back to the sender, which just prints them in console, so this is not actually crucial for the video stream to work. The rest of the logs seem fine.

Aren't there any other WARNING or ERROR messages in the KMS logs?

--
You received this message because you are subscribed to a topic in the Google Groups "kurento" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kurento/onDiYOhyk20/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kurento+u...@googlegroups.com.
To post to this group, send email to kur...@googlegroups.com.
Visit this group at https://groups.google.com/group/kurento.
To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/109af651-9ac4-4733-a258-f084c515b390%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sakshi]

Thanks Juan! This helps a lot.

There were no other WARNING or ERROR logs in KMS for that instance.

I am trying to get only audio working for SRTP in that example. I have modified the Java code to respond back with audio channel details for SRTP but am kind of stuck with sending audio stream using Gstreamer. Could you please take a look at this gst launch command and let me know if this should be able to stream an audio stream on an RTP port - 

gst-launch-1.5 -e rtpsession name=r sdes="application/x-rtp-source-sdes,cname=(string)\"user\@example.com\"" srtpenc name=e key="$SELF_KEY" rtp-cipher="aes-128-icm" rtp-auth="hmac-sha1-80" rtcp-cipher="aes-128-icm" rtcp-auth="hmac-sha1-80" srtpdec name=d filesrc location="$SELF_PATH" ! decodebin name=decode decode. ! queue ! audioconvert ! opusenc ! rtpopuspay ! "application/x-rtp,payload=(int)96,clock-rate=(int)48000,ssrc=(uint)$SELF_ASSRC" ! r.send_rtp_sink r.send_rtp_src ! e.rtp_sink_0 e.rtp_src_0 ! udpsink host=$PEER_IP port=$PEER_A r.send_rtcp_src ! e.rtcp_sink_0 e.rtcp_src_0 ! udpsink host=$PEER_IP port=$((PEER_A+1)) sync=false async=false udpsrc port=$((SELF_A+1)) ! "application/x-srtcp,$SRTP_CAPS" ! d.rtcp_sink d.rtcp_src ! tee name=t t. ! queue ! r.recv_rtcp_sink t. ! queue ! fakesink dump=true async=false

Thanks a lot for your help.

Thanks, Sakshi