NTP timestamp in RTCP SR from Ant media server

[Julie Huang]

Hi, 

I are trying to do lip synchronization for video and audio. I read some doc and the solution is to calculate the absolute time with NTP and RTP timestamp in RTCP SR and timestamp in each RTP package of video and audio. So I captured network packages via wireshark after Ant media server publish a stream. However, the NTP timestamp looks messed up with random time, like the year is 2045 at a time and next SR could be year 1971, month, day and minute are all different in SRs. 

Does anyone know what is the reason for this? Any setting is missing for Ant media server? Is there any other way to do synchronization without RTCP? Thank you very much!

[Tahir Gölge]

Hello,

On the audio and video there was a bug in lip sync that we resolved in 2.3.0, we don't expect it to happen. I asked about this and researched it, senior developers told me that they observed those timestamps are not meaningful every time, they are sometimes used for reference. The lip sync is a problem that it should be resolved on the upper layers of application.

Thanks

[Julie Huang]

Thank you very much for your reply.

Sorry, I think I made you confused. I am talking about here is to sync video and audio which are from different RTP sessions, not to display. And as I know, the regular way for multiple RTP sessions synchronization is to use the timestamps in RTCP SR packets. But so far, like you said, not meaningful every time. 

Do you mean it has been solved and timestamps are all good in v2.3.0? I'll try this version then. Thanks!

[Tahir Gölge]

Hi,

Sorry for the confusion, I didn't meant that the timestamps are all good in 2.3.0 they are the same it will not be possible to use them in the case. What task you are trying to do exactly maybe we can help if you give us some details, maybe there is another way or something.

Cheers

[Julie Huang]

Sorry for the late response. Thank you for your clarification!

In our project, we built a webRTC client with webRTC native sdk, which receives video and audio from a SDI card(1080p60) and send out to an Ant media server. The webRTC client encodes the coming video using our own hardware encoder and handles audio by using the built-in encoder, then send out to an Ant media server and display on Chrome browser. I checked the timestamp for both video and audio and they all come at the same time from SDI. The hardware encoding time is around 10ms per frame, which means each frame should go smoothly without delay. 

We setup the ntp_time_ms and capture_time_ms of webrtc::VideoFrame as the ntp time from SDI and timestamp_rtp is (int32_t)ntp_time*90 in our webRTC client. But it doesn't seem helpful. The video and audio are synchronized at the first several minutes. We can see the lip sync is working at the beginning since sometimes the video run faster to catch up with the audio. However, after a while, they are out of sync. Video is always slower than audio. If timestamp doesn't work, I don't know which way we can do.

We are using Ant media server Enterprise Edition 2.1.0 20200720_1328, waiting for another team to update it to 2.3.0. Do you think 2.3.0 could solve the issue? Any other ideas or suggestions?

Thanks a lot!

Julie