Implementing "who is talking" in conference with webrtc.

[Igor Karymov]

Hi! I want to implement real time "visual indication" of active speaker in my

conference application based on webrtc and freeswitch.

Freeswitch can detect such activity and send events to api interfaces.

So you can capture events and send them to clients with some kind of transport,

for example with the same connection that being used for signaling protocol.

This approach is simple but not a work in real life because:

 * you will experience delays in delivering events (your signaling transport usually

is not suitable for realtime data, tcp based etc)

 * you will experience asynchronization between what you hear and what you

see. (again because different channels with pretty different features)

From my own experience common working solution for this task it's send

"who is talking" data together or nearby with media data. For example we

can extend rtcp a bit or use webrtc "data channels" as a transport.

Unfortunately after quick look to this approach I faced next issues:

 * there is no easy way to work with rtcp on client side

 * there is no implementation of data channels in any popular conference server

solutions (freeswitch, asterisk etc...). And i belive that it will be pretty complicated

task to implement "data channels" support.

Could somebody share some experience or ideas about implementation of similar features?

Thanks.

[Ben Schwartz]

You should be able to use the audioLevel stat without changing the network configuration at all: http://w3c.github.io/webrtc-stats/#widl-RTCMediaStreamTrackStats-audioLevel

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/555b59a9-2c99-4cb2-bc48-cf17b6fd450d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Philipp Hancke]

or use the recently added support for remote webaudio stuff in Chrome (supported in Firefox for... a long time) with something like https://github.com/otalk/hark

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CAHbrMsDid5M4D88OZ3Xv0nZPJ3Yn2EJM5heNE3g_wVLsMHeDOQ%40mail.gmail.com.

[Igor Karymov]

Thanks for reply. But looks like with proposed solution I will be able detect only when I speaking myself. But I want detect, which particular user talking  in conference now. For example:

I have a 3 clients with 1,2 and 3 ids, all of them connected to conference server (which mixing all audio streams together and generate one final stream to "broadcast" to all clients).

When user with id 3 start talking, i have to notify user 1 and 2 about this somehow. This information also should include id of the user (3),  so all another clients will be able to handle such events (for example draw a waves around user 3 avatar).

[Christoffer Jansson]

Hi,

I think the best option is to use the audioInput level (available via getStats) on each client and then signal this to the conference server as Ben Schwartz suggested.

/Chris

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/8cb3919d-07d7-4bf4-9aa0-306694addb73%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

/Chris

[Philipp Hancke]

For mixed streams you would need support for RFC 6465. Star https://bugs.chromium.org/p/webrtc/issues/detail?id=3333 for updates.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

[Ben Schwartz]

audioLevel should also be available on the receive streams, so receivers can do this without involving the conference server.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CAAdOTUOpq9Q%3DgVzb0yaf8rW-JR5BkLU%2BiLmtVgL2xf-ixWwV%2BQ%40mail.gmail.com.

[Andrey Grusha]

Looks like this issue is abandoned for almost half a year... All I've found in source code were some TODOs. Is there a chance that it will be completed in any reasonable period? 

понедельник, 7 марта 2016 г., 20:37:39 UTC+3 пользователь Philipp Hancke написал:

[Bharat Gupta]

Hi, I'm also looking for the same thing. So have you found the solution?
Thank you 

Confidentiality Notice and Disclaimer: This email (including any attachments) contains information that may be confidential, privileged and/or copyrighted. If you are not the intended recipient, please notify the sender immediately and destroy this email. Any unauthorized use of the contents of this email in any manner whatsoever, is strictly prohibited. If improper activity is suspected, all available information may be used by the sender for possible disciplinary action, prosecution, civil claim or any remedy or lawful purpose. Email transmission cannot be guaranteed to be secure or error-free, as information could be intercepted, lost, arrive late, or contain viruses. The sender is not liable whatsoever for damage resulting from the opening of this message and/or the use of the information contained in this message and/or attachments. Expressions in this email cannot be treated as opined by the sender company management – they are solely expressed by the sender unless authorized.

[Phương Nguyễn]

How about emitting a request contain your audioLevel via socket and broadcast to everyone in the room. So everyone know who message is that and their volume as well