To Developers: Connecting Kurento to an external Python program based on opencv

[KittyCatty]

Hello everyone,

We have developed a software in Python based on OpenCV. Basically, we use the OpenCV's VideoCapture function.

We would like to connect this program (vi a stream) to KMS. That is, we would like it to receive the WebRTC sessions between clients and do some analysis on the streams.

What is the best way to do this? I have tried to use RTP streaming, but no luck!

KMS developers? Any ideas?

[Dobes Vandermeer]

I guess it depends how much latency you can tolerate.

If you don't care about latency at all, you can use the recorder endpoint to record to a file, and that file is easy to read once the recording is done.

If you do want it to be somewhat real-time, but a few seconds delay isn't an issue, you stream from a recording file while the recording is ongoing.

The recorder endpoint will upload a webm file via HTTP if your OpenCV application is accessible to your KMS via HTTP, I'm not sure how the latency of this compares to writing a file and reading the file.  It seems like it should be slightly better.

If you need it as "real time" as possible, you probably need to stick with WebRTC somehow, or figure out how to make the RTSP approach work.

Another route would be to customize the KMS itself - add a plugin to do the computation you are trying to do.  Definitely a lot more complex if you don't have any good C++ programmers.  This would probably have the lowest latency and best performance, but has the highest development cost.

[KittyCatty]

Thanks, I got the RTP solution to work in VLC. I am now able to play the RTP stream in VLC. I basically just connected the WebRTC endpoint to the RTP endpoint.

I then gave kurento the SDP file below and did rtpEndpoint.processOffer("<my SDP file>"):

v=0

o=- 0 0 IN IP4 192.168.1.102

s=

c=IN IP4 192.168.1.102

t=0 0

m=video 8081 RTP/AVP 100

a=rtpmap:100 H264/90000

a=recvonly

Finally, used VLC to open the SDP file on the client side.  The video is showing.

However, in opencv when I do:

cap = cv2.VideoCapture("rtp://play.sdp")

   while(True):

  7     # Capture frame-by-frame

  8     ret, frame = cap.read()

It simply hangs on the VideoCapture() function. I tried removing rtp, but to no avail...any ideas?

Thanks!