video freezes with multiple streams on single peer connection

[Bryan Donnovan]

I have a web conference app where each client has a single peerConnection to a central server.  The peer connection SDP has SSRCs for multiple streams corresponding to the current set of people speaking. 

The Session Description for the peer connection is regularly updated, as users start and stop speaking.  The onstreamadded and onstreamremoved callbacks fire as expected.  Care is taken to make sure that every receiver's session description is properly updated before a new sender begins transmitting on a new SSRC so that there is no worry of missing the keyframe when a new stream begins.

With one active stream on the peer connection it works pretty reliably, at least for the first several add/remove cycles.  Different users can take turns speaking, and things are pretty good.  However, with more than one active stream, or after some number of add/removes I start to see freezes of video (audio works fine) on a subset of the clients.  

At a guess, I would say that the problems happen most often when stream A is being received successfully, then the session description is updated to add stream b, so that 2 streams are being processed concurrently on the same peer connection.

The problems may be completely in my own application, and that would be my best case scenario, so I could find and fix them.  The purpose of this post is to see if there are any known issues related to what I am describing, or test cases involving multiple streams on a single peer connection, with many streams added/removed over time?

I could try to build repro case apart from my application, but I have a feeling that would be somewhat complex and time consuming, so I hesitate to go there unless it is needed.  The behavior I'm seeing happens on Canary 22.0.1206.0 (the latest as of this post) and also on the 21.x beta channel.

Thanks for any insights / advice / help

[Justin Uberti]

We're not aware of any problems like what you describe. I suggest looking at the logs from Chrome to see if there are decoder problems or other things that would cause freezes.

If you are able to make a standalone sample, that would probably be very useful for debugging, now and in the future.

[Bryan Donnovan]

I probably should have updated sooner -- I did write a test page app to torture the peer connection and I could not reproduce with host-only networking.

for development, I run a server inside a vm on a macbook connected to the lan via wifi.  I believe that much of the problem was packet loss due to vm issues (parallels) and wireless router issues.  power cycling the wifi router fixed some of it, and running a server on the internet makes the problem disappear almost entirely.

I still do see occasional issues, and my conclusion is that problems are due to packet loss.  

I have yet to turn on chrome logging so I don't know what it would show me -- I remember a post about how to do it a while back.

[Justin Uberti]

OK, thanks for explaining. 

FWIW, minor packet loss (< 5%) shouldn't lead to major freezes, especially with low RTT - any missing packets should be NACKed and retransmitted quickly.

--
 
 
 

[Bryan Donnovan]

The only packets I see at the server are RTP audio/video, and RTCP SR/RR.  

I expected to see RTCP Transport Feedback (Nack) and Payload Specific Feedback (FIR, PLI) but none of those packets ever came through.  It has been 2 weeks since I looked at that problem, however, so maybe canary sends them now.  My understanding from a previous post is that I do not have to explicitly signal RTCP-FB since the SDP parser does not parse those lines.

The server forwards the RTP and SR packets but not the RRs -- still trying to figure out exactly what should happen with those since there are many receivers on the other side of the MCU.  

The lack of RRs reaching the sender does not seem to have any adverse effect.  Just the opposite, really.  When I do forward the RRs from just one receiver, then the sender continually ramps up bandwidth, which is not what I want at all, so the lack of RRs makes the system more stable and scalable.

If the server was receiving Nacks as I expect, and I forwarded those to the sender, then that really should clear up the problem as you say.

[Justin Uberti]

NACK is sent as compound RR, did you look for that?

--
 
 
 

[Bryan Donnovan]

No.  but I will look now!

[Bryan Donnovan]

I'm using Netty , and I've built my own STUN / RTP handlers on top of that.  I expect that Node would be fine also.

On Thursday, July 26, 2012 2:04:45 PM UTC-7, sam artioli wrote:

Hi Bryan,

What kind of platform are you using for your server... Me and some friends are trying to do the same thing with node.js 

 

[Francois Temasys]

Hi all,

I'm currently facing almost the same problem instead that I'm using a native MCU.

Here is my testing process:

I launch my MCU, then 1 web client and finally 5 "robots" (they are sending a stream with only a video looping). The webclient and the 5 robots connect to the MCU which rebroadcasts everything though ONE peer connection per client.

Usually when I launch 5 robots or more, some videos on the webclient freeze.
I just recorded the log of the MCU during a session that froze (desktopPeer5Rfine.log) and one during a session that didn't freeze (desktopPeer5Rfreeze.log).

Few words about the logs:

-the level of information is LS_INFO

-there is not only webrtc log, I add my customs logs also

-you can recognize the robots with the name of the label (containing ZERG)

-when the web client connects, I always have this error:

Error(webrtcvideoengine.cc:1416): webrtc: NetEq-0 Error in function RecOutMasterSlave, error-code: 2003, error-string:  RECOUT_ERROR_SAMPLEUNDERRUN
Error(webrtcvideoengine.cc:1416): webrtc: RecOut: NetEq, error in pulling out for master
Error(webrtcvideoengine.cc:1416): webrtc: NetEq-1 Error in function RecOutMasterSlave, error-code: 2003, error-string:  RECOUT_ERROR_SAMPLEUNDERRUN
Error(webrtcvideoengine.cc:1416): webrtc: RecOut: NetEq, error in pulling out for slave

Errors that I don't have with the robots

- Many times I have this error:

Error(webrtcvideoengine.cc:1737): No stream added
Error(channel.cc:2073): Failed to SetSend on video channel

- When all the robots disconnect, I receive this but not for all the disconnections:

Error(webrtcvideoengine.cc:1416): webrtc: GetDelayEstimate() failed to locate channel (error=8002)

I hope it can help to investigate.

Cheers,

Francois