Accessing raw audio data

[Benjamin Blanchard]

Hi,

I've installed bigbluebutton using the VM and set up the development environment. Thanks to the clean design and clear instructions it has so far been a smooth process. 

I'm interested in developing a module which will present a visual representation of the speech signals of class participants. I have a few points of confusion regarding the voice conferencing architecture:

1) On the "Architecture Overview" diagram on this page: https://code.google.com/p/bigbluebutton/wiki/ArchitectureOverview

Item #9 is described as "A voice application the connects to Asterisk" Does bigbluebutton use both Asterisk and Freeswitch at this point?

2) Is it case that participating in a bigbluebutton session with synced audio and video is only possible using the "embedded voip phone in the client"?

And finally, perhaps a less quibbly and more theoretical question...

3) Would it be practical to produce an analysis of the speech (e.g. using the java fft library) and send the data to be displayed visually in a module on the client in real time (maybe by temporarily replacing the video feed)?

Thanks,

-Ben Blanchard

[Fred Dixon]

Hi Ben,

> "A voice application the connects to Asterisk" Does bigbluebutton use both Asterisk and Freeswitch at this point?

That's a bug in the docs -- we haven't used Asterisk in about two years.  We'll fix that as part of the upcoming 0.81-beta release.

> 2) Is it case that participating in a bigbluebutton session with synced audio and video is only possible using the "embedded voip phone in the client"?

On approach to getting better sync in the audio and video would be to transmit them together.  Currently, they are transmitted through separate paths, through separate subsystems, and are received by viewers on separate streams.

If we transmitted both audio and video together to red5, for example, we would need to use red5 to mix the audio channels.  This is possible, but it's a design decision we're making to use FreeSWITCH to mix the audio.  It's more efficient at mixing than Java, it provides full conferencing capabilities, and it enables access via the phone system to the conference bridge.

> 3) Would it be practical to produce an analysis of the speech (e.g. using the java fft library) and send the data to be displayed visually in a module on the client in real time (maybe by temporarily replacing the video feed)?

This sounds entirely possible.  If you can publish a video stream via RTMP, you can publish it via red5 as a video stream to all users in a session.  Obviously, there are more details than such a high-level statement, but it definitely possible.

Regards,... Fred

-- 
BigBlueButton Developer

http://bigbluebutton.org/
http://code.google.com/p/bigbluebutton

BigBlueButton on twitter: @bigbluebutton

--
You received this message because you are subscribed to the Google Groups "BigBlueButton-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bigbluebutton-...@googlegroups.com.
To post to this group, send email to bigblueb...@googlegroups.com.
Visit this group at http://groups.google.com/group/bigbluebutton-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.