basic simulcast issue
LOG
Hi all, I have an issue specifically with simulcast receiving in our libwebrtc c++ server. I've stripped the codecs down to the bare minimum to help isolate, so only vp8, yet even after that last brainwave our server is still re-creating default stream for every new ssrc in simulcast. As you can imagine it is doing this very quickly as each of the 3 different packets come in one after the other. My browser (Chrome: 140.0.7339.185) is sending simulcast correctly in so far as I can tell, confirmed with webrtc-internals, and again confirmed on the wire where I have used Wireshark to observe the packets, especially those at the start of the stream which has all the header extensions, and I can identify rid as l, m or h in the extension and see the different SSRC. Later ones do not carry all the header extensions but I suppose by then it should have been able to match the msid, ssrc and rid and not require it for every packet. I feel I have probably not set something up correctly on the server even though from what I can gather absolutely EVERYTHING is correctly done, but i must be missing something???? I am out of ideas as to what I could even try next to try to fix this issue. It is kUnifiedPlan btw. I have enclosed my offer/answer (offerer is Chrome btw) where to me they appear to be correct but I cannot stop my FrameTransformer from being the victim of a constant register/unregister hell when the incoming ssrc is different from the previously processed ssrc. One thing to note is that the answer is munged in that the 4 lines below are added in our code as they are not auto generated when creating the answer from the offer. a=rid:l recv a=rid:m recv a=rid:h recv a=simulcast:recv l;m;h The stacktrace points the finger at MaybeCreateDefaultReceiveStream which always calls ReCreateDefaultReceiveStream in webrtc_video_engine.cc when the ssrc switches. I imagined it was meant to be building up a map of mid to rid to ssrc but I guess it does not want to do that. The register/unregister on my FrameTransformer is a symptom of something under the hood that does not appear to be working according to how I envisaged it would from what I've been able to pull together online. Offer/Answer below. Any help much appreciated. Best Lynn Also note I have a verbose log on the server if there is any further information needed that might help track it down. offer=v=0 o=- 4587056080887895682 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE 0 1 a=extmap-allow-mixed a=msid-semantic: WMS f9d39db3-a921-43e8-9961-6e179b015947 m=audio 9 UDP/TLS/RTP/SAVPF 111 c=IN IP4 0.0.0.0 a=rtcp:9 IN IP4 0.0.0.0 a=ice-ufrag:R7JN a=ice-pwd:T/q1aigaMgxQG7hhMN5+mF80 a=ice-options:trickle a=fingerprint:sha-256 25:A0:8A:C1:B3:55:19:B8:24:5B:AC:46:CA:ED:E1:78:A6:F2:13:02:44:5A:2E:C0:2C:E1:48:D8:C8:09:11:3E a=setup:actpass a=mid:0 a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01 a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid a=sendonly a=msid:f9d39db3-a921-43e8-9961-6e179b015947 1da2d198-0e49-4dff-8859-e4db01385cfc a=rtcp-mux a=rtcp-rsize a=rtpmap:111 opus/48000/2 a=rtcp-fb:111 transport-cc a=fmtp:111 minptime=10;useinbandfec=1 a=ssrc:1813919620 cname:REV3LzpS5DpHofYu a=ssrc:1813919620 msid:f9d39db3-a921-43e8-9961-6e179b015947 1da2d198-0e49-4dff-8859-e4db01385cfc m=video 9 UDP/TLS/RTP/SAVPF 96 c=IN IP4 0.0.0.0 a=rtcp:9 IN IP4 0.0.0.0 a=ice-ufrag:R7JN a=ice-pwd:T/q1aigaMgxQG7hhMN5+mF80 a=ice-options:trickle a=fingerprint:sha-256 25:A0:8A:C1:B3:55:19:B8:24:5B:AC:46:CA:ED:E1:78:A6:F2:13:02:44:5A:2E:C0:2C:E1:48:D8:C8:09:11:3E a=setup:actpass a=mid:1 a=extmap:14 urn:ietf:params:rtp-hdrext:toffset a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time a=extmap:13 urn:3gpp:video-orientation a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01 a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id a=extmap:12 https://aomediacodec.github.io/av1-rtp-spec/#dependency-descriptor-rtp-header-extension a=extmap:9 http://www.webrtc.org/experiments/rtp-hdrext/video-layers-allocation00 a=sendonly a=msid:f9d39db3-a921-43e8-9961-6e179b015947 fae16540-f001-401e-934d-9b9e207a87f8 a=rtcp-mux a=rtcp-rsize a=rtpmap:96 VP8/90000 a=rtcp-fb:96 goog-remb a=rtcp-fb:96 transport-cc a=rtcp-fb:96 ccm fir a=rtcp-fb:96 nack a=rtcp-fb:96 nack pli a=rid:l send a=rid:m send a=rid:h send a=simulcast:send l;m;h answer=v=0 o=- 3692800017751382310 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE 0 1 a=extmap-allow-mixed a=msid-semantic: WMS m=audio 9 UDP/TLS/RTP/SAVPF 111 c=IN IP4 0.0.0.0 a=rtcp:9 IN IP4 0.0.0.0 a=ice-ufrag:hAGP a=ice-pwd:HZoZwpFNMLIRjviCqYHJKqsU a=ice-options:trickle a=fingerprint:sha-256 0D:34:D0:9A:CA:BD:A4:3A:6E:0A:A2:AE:1E:28:E3:EA:E8:0B:3E:4A:2E:CE:95:B1:11:6E:9B:E6:56:8F:1F:8E a=setup:active a=mid:0 a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01 a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid a=recvonly a=rtcp-mux a=rtcp-rsize a=rtpmap:111 opus/48000/2 a=rtcp-fb:111 transport-cc a=fmtp:111 minptime=10;useinbandfec=1 m=video 9 UDP/TLS/RTP/SAVPF 96 c=IN IP4 0.0.0.0 a=rtcp:9 IN IP4 0.0.0.0 a=ice-ufrag:hAGP a=ice-pwd:HZoZwpFNMLIRjviCqYHJKqsU a=ice-options:trickle a=fingerprint:sha-256 0D:34:D0:9A:CA:BD:A4:3A:6E:0A:A2:AE:1E:28:E3:EA:E8:0B:3E:4A:2E:CE:95:B1:11:6E:9B:E6:56:8F:1F:8E a=setup:active a=mid:1 a=extmap:14 urn:ietf:params:rtp-hdrext:toffset a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time a=extmap:13 urn:3gpp:video-orientation a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01 a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id a=extmap:12 https://aomediacodec.github.io/av1-rtp-spec/#dependency-descriptor-rtp-header-extension a=recvonly a=rtcp-mux a=rtcp-rsize a=rtpmap:96 VP8/90000 a=rtcp-fb:96 goog-remb a=rtcp-fb:96 transport-cc a=rtcp-fb:96 ccm fir a=rtcp-fb:96 nack a=rtcp-fb:96 nack pli a=rid:l recv a=rid:m recv a=rid:h recv a=simulcast:recv l;m;h