extracting RTP/RTCP timestamp

[laszl...@gmail.com]

Hi Kurentoers

I'm working on a project that aims to analyse user behaviour via webcam video recording. We're currently evaluating Kurento as media server for recording webcam media stream.

Our main concern is that we must find a way to map individual frames in the recorded video to user actions. We record these actions on the client side with Javascript based on local time. We'd like to use timestamps in the RTP/RTCP packets to timestamp each frame in the recorded video so that we can match video frames to user actions as precisely as possible.

As a first step could somebody advice me how could we extract these timestamps in Kurento? Is there a way to achieve this without modifying Kurento's source code?

Thanks in advance,

Laszlo

[Ivan Gracia]

Hi Laszlo,

I'm afraid there is no out-of-the-box way to get that timestamp. You will need to modify the desired endpoint to add a getCurrentFrameTimestamp method, for instance, and get the timestamp of that frame. However, bear in mind that timestamps are in milliseconds, so by the time the request reaches to the server, gets processed and is sent back, you can have values that are no longer valid for detected gestures.

Just out of curiosity, what type of actions do you want to record?

Ivan Gracia

--
You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[laszl...@gmail.com]

Hi Ivan,


Thanks for you answer.

In fact I'd need the timestamp to save it along with the recorded video and later match frames in recorded video to actions registered on the client side. So processing delay is not a concern for us, we won't send these timestamp back.

We record user actions on the client side with client time based timestamp and save them to database. Then we'd like to extract RTP/RTCP timestamps from the stream, save frames' timestamps also into the database then do matching of recorded video frames and recorded user actions.

For you curiosity, user action can be anything like clicking on a button, scrolling down the page, etc.

It would be a big help if you could point out the module/file I'd need to modify in order to be able to extract these timestamps  from the Webrtc stream.


Thanks in advance,


Laszlo

[Ivan Gracia]

Thanks for the info ;-) Looks interesting!

Here is the code of the base RTP endpoint, where you can get the timestamps in the RTP header.

Cheers,

Ivan Gracia

[Laszlo Vass]

Hi Ivan,

Thanks a lot for your help, I'll have a look at it!
Will I also find there RTCP header timestamps? I guess I'd also need ntp timestamps from the RTCP Sender report to have a correct match of RTP timestamps to client side sampling time...

Thanks,

Laszlo

You received this message because you are subscribed to a topic in the Google Groups "kurento" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kurento/GBPTgKC5xZU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kurento+u...@googlegroups.com.

[Ivan Gracia]

Yes, it should be there.

Cheers,

Ivan Gracia

[Laszlo Vass]

Fine, thanks a lot!

Laszlo