SDK or Tool for testing WebRTC link from a webapp to the antmedia Server?

[Robert Berger]

Our customers are seeing different performance (primarily frame drops it seems) even with the same setup but at different times.

Wondering what tooling is available to help us detect and debug these issues both as a test before an event and to diagnose situations during an event.

Hard to figure out right now if its their LAN/WiFi, their Internet connection, the TUN connection, their device, the server or something else.

Pretty sure its not the server. The server is very underutilized right now running under 10% CPU 99% of the time. 

Todays' problem was the video was very jerky though the audio was perfect.  Though when we had run test streams earlier we did not have this problem.

Looking at the Antmedia server logs during the actual event, I see that what I believe was significant amount of drop frames by the WebRTC encoder. I put a snippet of the WebRTCEncoder log lines below. Other times and places we get fine video quality....

Are there any tools we could use to help with figuring out where the bottleneck is? Any other suggestions? 

Are there other ways to have the dashboard show these stats in a way that is more clear? 

Here's a sample of some of the log during that time

2021-02-23 00:14:00,164 [network_thread - 12902] INFO  i.a.e.adaptive.WebRTCEncoderAdaptor - Publish Stats StreamId: 0995eb7f-47ad-4948-8913-b6e7d037803d TransferedByte: 1489058 Stream Duration: 39513 Bitrate: 37 Source IP:  User-Agent: N/A

2021-02-23 00:14:05,307 [IncomingVideoSt - 12911] INFO  i.a.e.adaptive.WebRTCEncoderAdaptor - Number of video frames pending in the queue: 1, received total video frames: 300  received fps: 6 for stream 0995eb7f-47ad-4948-8913-b6e7d037803d

2021-02-23 00:14:10,167 [network_thread - 12902] INFO  i.a.e.adaptive.WebRTCEncoderAdaptor - audio stats: { timestampUs: 1614039250166406, type: inbound-rtp, id: RTCInboundRTPAudioStream_4252049050, ssrc: 4252049050, isRemote: false, mediaType: "audio

", kind: "audio", trackId: "RTCMediaStreamTrack_receiver_139", transportId: "RTCTransport_0_1", codecId: "RTCCodec_0_Inbound_111", packetsReceived: 2475, bytesReceived: 197147, headerBytesReceived: 59400, packetsLost: 14, lastPacketReceivedTimestamp:

1.9022110955E7, jitter: 0.027 }

2021-02-23 00:14:10,167 [network_thread - 12902] INFO  i.a.e.adaptive.WebRTCEncoderAdaptor - Publish Stats StreamId: 0995eb7f-47ad-4948-8913-b6e7d037803d TransferedByte: 1921050 Stream Duration: 49516 Bitrate: 38 Source IP:  User-Agent: N/A

2021-02-23 00:14:16,657 [IncomingVideoSt - 12911] INFO  i.a.e.adaptive.WebRTCEncoderAdaptor - Number of video frames pending in the queue: 1, received total video frames: 400  received fps: 7 for stream 0995eb7f-47ad-4948-8913-b6e7d037803d

2021-02-23 00:14:20,162 [network_thread - 12902] INFO  i.a.e.adaptive.WebRTCEncoderAdaptor - audio stats: { timestampUs: 1614039260161466, type: inbound-rtp, id: RTCInboundRTPAudioStream_4252049050, ssrc: 4252049050, isRemote: false, mediaType: "audio

", kind: "audio", trackId: "RTCMediaStreamTrack_receiver_139", transportId: "RTCTransport_0_1", codecId: "RTCCodec_0_Inbound_111", packetsReceived: 2974, bytesReceived: 237682, headerBytesReceived: 71376, packetsLost: 18, lastPacketReceivedTimestamp:

1.9022120917E7, jitter: 0.021 }

2021-02-23 00:14:20,162 [network_thread - 12902] INFO  i.a.e.adaptive.WebRTCEncoderAdaptor - Publish Stats StreamId: 0995eb7f-47ad-4948-8913-b6e7d037803d TransferedByte: 2221367 Stream Duration: 59511 Bitrate: 37 Source IP:  User-Agent: N/A

2021-02-23 00:14:30,163 [network_thread - 12902] INFO  i.a.e.adaptive.WebRTCEncoderAdaptor - audio stats: { timestampUs: 1614039270162768, type: inbound-rtp, id: RTCInboundRTPAudioStream_4252049050, ssrc: 4252049050, isRemote: false, mediaType: "audio

", kind: "audio", trackId: "RTCMediaStreamTrack_receiver_139", transportId: "RTCTransport_0_1", codecId: "RTCCodec_0_Inbound_111", packetsReceived: 3470, bytesReceived: 277593, headerBytesReceived: 83280, packetsLost: 22, lastPacketReceivedTimestamp:

1.9022130962E7, jitter: 0.027 }

2021-02-23 00:14:30,163 [network_thread - 12902] INFO  i.a.e.adaptive.WebRTCEncoderAdaptor - Publish Stats StreamId: 0995eb7f-47ad-4948-8913-b6e7d037803d TransferedByte: 2570669 Stream Duration: 69512 Bitrate: 36 Source IP:  User-Agent: N/A

2021-02-23 00:14:40,165 [network_thread - 12902] INFO  i.a.e.adaptive.WebRTCEncoderAdaptor - audio stats: { timestampUs: 1614039280164747, type: inbound-rtp, id: RTCInboundRTPAudioStream_4252049050, ssrc: 4252049050, isRemote: false, mediaType: "audio

", kind: "audio", trackId: "RTCMediaStreamTrack_receiver_139", transportId: "RTCTransport_0_1", codecId: "RTCCodec_0_Inbound_111", packetsReceived: 3970, bytesReceived: 317387, headerBytesReceived: 95280, packetsLost: 23, lastPacketReceivedTimestamp:

1.9022140944E7, jitter: 0.024 }

[ahmet oguz mermerkaya]

Hi Rob,

As far as I see received fps is really low. It's says that  "received fps: 7"

I think the client cannot send the video with high quality because of its internet connection. 

We provide statistics on the JS side to see the frame rate, packet lost, bitrate etc. It's available in the WebRTC Publish page(WebRTCAppEE/index.html) 

I mean it starts to appear after publishing starts. Take a look at the screenshot

Regards,

A.  Oguz

[Robert Berger]

I agree its probably that its their Internet connection. Would the client stats be able to indicate the link quality or just what it ends up using?

Is there a way to get the same stats on the server side? Or any existing examples of getting the client stats to the backend?

Any other nice tools that could be used to test the capacity / reliability of the end to end link from a client to the Antmedia server that could be run to determine the network quality / issues when setting up a venue or an event?

Thanks

[Selim Emre]

Hi Rob,

Have a good day. 

We are providing also WebRTC Client statistics which are bitrate, packet loss, jitter etc. You just need to click option button on WebRTC Player page(WebRTCAppEE/player.html).

You can able to access Client WebRTC stats in this REST API -> https://antmedia.io/rest/#/BroadcastRestService/getWebRTCClientStatsListV2

We have WebRTC Measurement Test Tool in v2.3.0. You can access webrtc-test-tool.html page under the application like WebRTC Publish/Play pages.

Bonus:

You can also check other WebRTC Measurement Test tool -> https://test.webrtc.org/

https://antmedia.io/webrtc-network-measurement-test-tool-introduction/

I hope, I could help you.

Best Regards,

Selim

[Robert Berger]

Those are very helpful. Thanks 

Robert J Berger +1 408-838-8896

Omnyway CTO/SVP Engineering

http://www.linkedin.com/in/rberger

http://www.omnyway.com

--
You received this message because you are subscribed to a topic in the Google Groups "Ant Media Server" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ant-media-server/Y_WBBlOuzzc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ant-media-serv...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ant-media-server/f9786409-6ea3-4a8d-90fb-9214c20a2746n%40googlegroups.com.

[Robert Berger]

Would the webRTC Test Tool tell us if there is an issue with the TURN server setup? If not is there some what to determine that?

Robert J Berger +1 408-838-8896

Omnyway CTO/SVP Engineering

http://www.linkedin.com/in/rberger

http://www.omnyway.com

On Feb 23, 2021, 11:41 AM -0800, Selim Emre <selime...@gmail.com>, wrote:

--

[Selim Emre]

Hi Rob,

Have a good day. 

As I understood, you are referring Ant Media Server WebRTC Test Tool. Am I right? You need to change pc_config parameters in webrtc-test-tool.html page codebase. Here is a guide for add TURN server to webrtc-test-tool.html page. -> https://stackoverflow.com/questions/62673036/how-to-use-turn-server-in-ant-media-server

Btw, thank you for your feedback. I think we may add TURN server details in GUI settings in webrtc-test-tool.html. 

Best Regards,

Selim