How can I set or read the RTP timestamp in the iOS objc api?

[Christoph Müller]

Hello,

I am working on an iOS video telephony app that sends its own custom video source via WebRTC.
This video source is implemented as RTCVideoCapturer, which passes each frame via delegate.capturer(self, didCapture: frame).
For. every frame I generate and set the timeStampNs.
The video stream is then sent to the server and from there to all participants in a session.

Each participant can interact with the video and click with the mouse at a point in the video image. The coordinates are then sent back to the iOS device together with the RTP timestamp.
The iOS device must now process the interaction.
Due to transmission delays and movements of the device, the click coordinate never matches the actual position where the other participant clicked in the video image.
For this reason, I need the timeStampNs I originally generated per frame so I know which video frame the user saw when they clicked to compensate the delays.

However, the RTP timestamp sent by WebRTC, which arrives at the server and the participants, no longer correspond to my timeStampNs of the frame that I generated.

The RTP timestamp that arrives at the server counts up, but starts at a random starting point for each new session, that cannot be read anywhere.

Can I set or read the RTP timestamp that actually goes to the server?

I have already tried to derive from RTCDefaultVideoEncoderFactory to set a callback for the generated encoder to get the RTCEncodedImage, but the callback is never called.

Are there any possibilities or suggestions?

Thank you very much,
Christoph

[Philipp Hancke]

The random start of the RTP timestamp is intentional. If you want the clients local clock (which may not be synchronized among the clients) you will need to resolve it to the absolute timestamp using the RTCP SR.

https://bloggeek.me/lip-synchronization-webrtc/ has a pretty good explanation.

--
This list falls under the WebRTC Code of Conduct - https://webrtc.org/support/code-of-conduct.
---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/12cb6be3-275c-4ad6-8df5-058a073ba64fn%40googlegroups.com.