Audio and video conference in Webrtc

[silviu.cpp]

Hello Guys,

I have started to implement audio and video conference using libjingle and webrtc.

The first approach was to create a "full-mesh" implementation (there is a link between each participant) . This is working very nice for small conference size (around 6 participants) where participants have good internet connections.

I'm thinking to improve this by mixing the audio signal on one client and sending the mixed signal over the other channels. Have somebody an idea how to mix the signal on the transmission side on WebRTC? Basically having 4 channels :
A B C and D - > Channel A will receive B+C+D channel B -> A+C+D and so one.

Any help is appreciated,
Silviu

[Albert Abello]

Hi,

If i understand correctly you are willing to do a MCU style setup where one client is the actual MCU that mixes the channels right? Guess that browser is gonna suffer a lot on the performance side, if you are thinking about centralized MCU there are few approaches on the discussion group.

If you want to setup one client/browser as MCU + client you need to think about the performance issues and how many media streams will you be carrying into the stream. That guy will setup 6 peer connections meanwhile the other will just have one? Is this your approach?

Cheers

[Caragea Silviu]

Hello,

Basically I want follow the Skype architecture. I don't want to
involve any other costs with server side mixing . On skype for voice
conference which is free one of the participants is mixing all the
audio and send to the others.
The mixer is picked based on the best hardware configuration and
larger bandwidth available.

The initial setup that I have done using full mesh implementation is
working great with 7 desktop clients but on mobile (ios 4) is not
working up to 5 participants (the CPU is going to 100% because it has
to decode 4 streams).

Silviu

> --
>
>
>

[Albert Abello]

Well with webrtc you don't have access to that low level architecture without building your own plugin, but I don't even know how could you gather which is the client with the best hardware... maybe once the stats are fully implemented you could catch up with the RTT to see who has the best BW but now i think the best approach would be MCU even the anoying part of the server processing.

Cheers

[Caragea Silviu]

I'm using Jingle for signaling through a Jabber server. I already have
there integration with PSTN and more other stuff. making the signaling
work (including detection for the best PC config and highest available
bandwith is my last problem). What I'm trying to get if there is any
simple way to mix channels without changing the transmission mixer.

> --
>
>
>

[Caragea Silviu]

Ok . I fixed it. I have changed the transmission mixer to mix the
frames in the way I want . seems to works very nice .

[Luis Toro]

Hi, i'm very interested in your approach, i'm developing a webRTC client with Jingle and a jabber server too, and i want to know if you are switching the video streams from the same user that do the mixing or the video is still using the full mesh implementation, and how many users are you able to connect with your approach.

Regards,

LT

[Caragea Silviu]

Hello I did only audio conference in this way. If the mixer have a decent internet connection works fine with around 15- 17 users.

For video I'm implementing mixing on server.

Silviu

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Luis Toro]

Ok!!Thank you very much for your quick response.

[Gary YU]

Hi Silviu,

Don't you want share us how to implement this audio conference mixing?

Best Regards

Gary.

[itay bia]

Hi Silviu,

a few questions if you don't mind:

1. do you mind sharing what you did? what changes were necessary?
   
2. did you do this on a mobile device?
   
3. how do you take care of the delay from one side to the other? meaning, if A, B and C are in a conference with C being the MCU, how did you get the packets of A mixed with C and sent to B in a way that A isn't delayed compared to C?
   

Thanks

[hu]

silviu.

how many users are support in mobile device for video conference by your mesh solution? 

thx