Mixing multiparty audio from multiple peer connections on iOS/Android

[Alin Radut]

Hello!

I'm trying to wrap my head around writing an iOS/Android WebRTC based MCU. Having experienced the full mesh topography, obviously I reached the conclusion that it's not good enough to ensure good quality for all participants as it requires that all of them are on a good connection and on good hardware. I don't want to involve dedicated MCU server as that would incur non-negligible monetary and operational overhead.

What I have in mind is that if we have 3 participants, A, B an C, A being the MCU, A would directly communicate with B and C, while B would communicate to C and C to B only through A.

I've been investigating for the past few days any way of doing this and I found a couple of older threads (https://groups.google.com/d/topic/discuss-webrtc/EEm3JY9Y7pk/discussion, https://groups.google.com/d/topic/discuss-webrtc/GlSVfKdAmG4/discussion) that touched it, but it's not clear what the correct approach would be. With the web API this appears to be a breeze: https://webrtchacks.com/web-audio-conference/

What I've tried so far is: when setting up a new peer connection between A and C I check whether B is in the room. If B is present, I call pcAC.addStream(pcAB.remoteStream). This leads to some garbled audio being passed between C and B, but no the other way around. If I mute A's speaker, C can still hear the garbled, almost-intelligible audio from B. The codecs are the same in both peer connections and the sample rates as well. the CPU usage doesn't get past 70% on device A.

Can anyone give me an idea, pointers or reference how to get audio flowing between B and C and to tackle the garbled audio issue?

Thank you!

[Alin Radut]

Some more progress on this front: now I have duplex audio flowing between B and C, after calling pcAB.addStream(pcAC.remoteStream). The curveball was that this triggers a "renegotiation needed" so the call initiator should generate a new SDP offer when that happens.

But the audio is still garbled between B and C. Audio straight from A's microphone is clear on both B and C's speakers.

I am using the same sample rate on all devices and using OPUS.

Any ideas?

[Harita Bhatt]

I have the same problem. Did you find any solution to this?

[Alin Radut]

Hello!

No, I ended up using a full-mesh for conferences with a small number of participants and Jitsi for a larger number of participants.

Alin

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/790ba3c3-9ec6-4522-9e1f-10f0562de324n%40googlegroups.com.

[Harita Bhatt]

Thank you for your reply.

Ok, Can you share some examples or demo link for the same?

[Alin Radut]

Unfortunately I don't have examples to share for full-mesh as the code is too complex for that, but the idea behind it is:

- you have person A who initiates the call

- A sends their invite to B and C for the calls with the IDs CALL_AB and CALL_AC

- A also tells C: "you need to set up a call with B, you will receive the invite shortly for call CALL_AB_C"

- A also tells B: "you need to set up a call with C, you will be the initiator, for call CALL_AB_C"

If A wants to add D to the conference, they would send the invite CALL_AD and once it's established, it would tell D "expect and invite for calls CALL_AB_D and CALL_AB_C" and tell B and C: "D has been added to the conference, send him an invite"

D will receive invites for calls CALL_AB_D and CALL_AC_D and establish connectivity with them.

This quickly gets out of hand as you add more and more participants because you need to have N-1 peer connections where you are sending the same data, which is not ideal. We never do full-mesh video because the load would be too great on the devices.

If you can afford a server infrastructure then JitsiMeet is the better choice when it comes to performance because you will have only one peer connection, to the Jitsi Videobridge which is responsible for mixing audio and video from all parties.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/34ed2380-0013-4246-a114-dfd296c63222n%40googlegroups.com.

[Harita Bhatt]

Thank you. This is really very helpful information to kick start. I will try to implement this. 

You received this message because you are subscribed to a topic in the Google Groups "discuss-webrtc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/discuss-webrtc/ruhDuK8N8KA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CAKCB6Gzxi6sf0MSJrW9jfWZ1bhczhzHYU%2BzMtA3KHQ__vvw5rA%40mail.gmail.com.