Scaling out to HLS / RTSP(RTMP) From HTML5

[abrend...@gmail.com]

Hello BBB Developers,

I'm trying to get ramped up to speed on BBB and had an idea of an implementation in mind, but had some Q's.

Our implementation will be 1-on-1 with educators, but also could be webcasts that educators give to their entire incoming classes of students(1-to-10,000). 

Obviously WebRTC isn't a great technology for this, but WebRTC gets me excited as a presenter, especially if we have 2 or 3 folks presenting to the larger group(low latency and great html5 interface). BBB brings a very nice presenter interface and is what got us hooked.

So, in the HTML5 client/server, we are looking to build or augment the server, with the ability to broadcast the stream(current talking presenter) via HLS so we can scale out. For some events, there is interest in broadcasting also to Facebook Live, YouTube and Periscope. I think we are looking at creating an api, that would "Join" special users that would actually be processes in ffmpeg or somewhere. And that you would see in BBB's userlist, "You" "Other Presenter" "Facebook Live" "Audience"(HLS) "Periscope" etc. and our custom implementation would just be "viewing" in listen only mode.

So my questions:

 - Is there any existing way to do this or planned work in this area. Totally don't mind contributing some of this back if it can help others!

 - It appears red5 is only being used for the Flash client, is the plan to sunset support for red5? If so, I assume my hooks would be either directly in BBB or Kurento.

 - We would love, for the HLS stream, to only replace the WebRTC portion of the stream, and keep the slides separate, for higher quality and lower bandwidth. I'm assuming that would be a custom player that could handle the delay to implement that.

 - Any easy way to load balance between 2 BBB servers. We are currently exploring just standing up a microservice that would call getMeetings() on each host and then give you back the host with the least # of participants, or lowest CPU or something.

Thank you for any direction, and I look forward to the call about RC5 later this week! BBB seems to be making leaps and bounds of progress all the time, I look forward to joining this train!

[Chad Pilkey]

To scale to something like 1k viewers (or even 500 viewers) you really need a clustered approach so that you aren't overloading one server that has to handle the messaging for every user. Unfortunately with BBB we have yet to go down the road of trying to change to that type of architecture. If you wanted to only change the media you'd still run into scaling issues with the other components. You'd see breakdowns in meteor/node for sure and potentially bbb-web.

I think the quickest way forward for the bigger meetings would be something like OBS or ffmpeg streaming the full browser to a video delivery service and directing the listen-only users there. Scalable live video delivery is basically a project in itself that is difficult and expensive to solve. The frequency of the bursts of concurrent users would probably drive what solution would be chosen because if the need is infrequent enough then a fully automated solution is maybe not needed.

Red5 has served us very well over the years, but we only use it for RTMP connections so once those are no longer needed I can foresee us removing it down the road once the Flash client is no longer needed.

The basic load balancer would need to monitor N servers and check # of users and resource usage (maybe app health checks too) and pick an under-utilized server on meeting creation. After the meeting is created every user that wants to join the same meeting needs to be sent to the same BBB server. There's a limited number of API calls that a load balancer needs to implement (create, join, recordings) depending on what capabilities you want to give your front-end.

We always recommend 50 or less users in each session as it's much easier to load balance those scenarios and they shouldn't cause the server to struggle at all. With good servers you can get 100+, but beyond that I would test the capabilities in a live scenario to see if bottlenecks develop.

[abrend...@gmail.com]

Thanks Chad -

Thats why I'm hoping HLS/RTMP could take off that load of BBB or its UIs supporting hundreds of users. And offload that to Facebook(RTMP)/CDNs(HLS). We typically only have at max 10 webcasts happening at a time, with an average of 1.5 presenters. so splitting that load over 2 dedicated servers will probably be ok. (15 presenters, and 30 listen only ffmpeg users in my example above, say 3 per webcast).

I think your example of OBS/ffmpeg is where I am looking to go, where it would be automated and run on a server somewhere. Our API would just say, "join a user to xyz meeting, and stream the output to HLS/Facebook" and it would join in and combine the WebRTC streams(still working on this one), and send them off.

[Paulo R. Lanzarin]

If you wanna bundle multiple WebRTC streams of a webconference session in a single outbound stream via RTMP/RTSP, capturing the webconference view is definitely the easy way to go.
You could implement it with a headless browser that joins the conference with some special metaparameters (html5hidepresentation, html5autoswaplayout are interesting ones,
you would have to implement a few more to e.g. disable the chat view), and then fire an ffmpeg that grabs the screen and audio from the headless environment and stream
it out to your desired URI. It'll solve your problem in a relatively easy way without having to meddle with too much code. How you autoomed the firing of that headless environment
is up to you though, there's a myriad of ways to do it (e.g. containerized environments and a small app that you can send commands to orchestrate your streaming environments).
Be aware that this approach might be a bit CPU intensive, so you'd better run it in an isolated host.

s,

Paulo.

--
You received this message because you are subscribed to the Google Groups "BigBlueButton-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bigbluebutton-...@googlegroups.com.
To post to this group, send email to bigblueb...@googlegroups.com.
Visit this group at https://groups.google.com/group/bigbluebutton-dev.
For more options, visit https://groups.google.com/d/optout.

[abrend...@gmail.com]

I was able to pretty easily get the recordings working, but was curious if there were any tips around creating a highly custom UI, but re-using some BBB components from the HTML client.

My end goal, would be to write a flexible js api, where you pull in a snippet of JS(probably the BBB html5 client, somehow), and it would let you design your own layout dynamically. By dynamically I mean, in my page that has the injected js snippet, I could say <div id="presenter"> and that would be where the webcams go, and <div id="slides"> and thats where the slides/deskshare would go, and I could set fixed width/height of those, and brand around it. This proposed interface would of course be listen-only for the ffmpeg stream to pick up.

The main things I would love to re-use, is the websocket signaling api / whiteboard / polls(this would not be reused by ffmpeg, but instead by the HLS end user stream on a delay). I could re-write the whiteboard feature I guess, but as it evolves and changes, figured, why reinvent the wheel! I am unfortunately a big unfamiliar with React however, and not sure if anything is that portable.

I wasn't sure if a mode like this, where the layout is manually defined, was anything BBB was interested in? Definitely willing to work together and get something solid out there, but being able to essentially have the HTML5 client be extremely customizable in layout could be pretty cool.

This may seem like an odd request, but we really love BBB for presenters, and for small groups and plan to use the client as is for that part. Its more for this webcast scale out mode where we need the more customized interface.

If this is something that we would need to fork the client for, and just maintain our own "views" essentially, that is ok too(get to QA our players more easily ;) ), was curious if anyone had a nudge for a place to look within the app for things of that nature. I'm assuming I would need to add in a new route / main component at the least. And if we go this route, we may end up creating some barebones views, that then can be later composed via iFrame depending on which components are needed. Not sure yet. Starting exploring now =)

Thanks for any tips or nudges in a better direction!

On Wednesday, August 22, 2018 at 10:33:25 AM UTC-4, abrend...@gmail.com wrote:

[Fred Dixon]

Hi,

> but was curious if there were any tips around creating a highly custom UI, but re-using some BBB components from the HTML client.

We're not there yet.  We've seen other companies put out individual components out that you can assemble, but our target market is online learning and, like the BigBlueButton Flash client, we want to deliver a working product that offers an immediate solution for the instructor.  It's much easier to build a community around a working product than individual components that require assembly.

We do want to offer a platform that others can build upon.  With the Flash client, we added ways for developers to configure/extend it through the API, dynamic config.xml, branding, localization, layout manager, and Javascript API.  These gave a third-party developers many options for configuring, customization, and embedding the Flash client.

We are planning on going down the same path with the HTML5 client.  In other words, we're thinking in terms of configuration (i.e. easily set the default presentation, the default local, enable/disable features, etc.), extensible (providing a simple plug-in architecture to extend the '+' with new commands), making it easy to embed in an existing HTML5 application through a Javascript API (i.e. start/stop recording through your app).

In other words, we're not planning to break up the HTML5 client into individual components, but rather make the HTML5 client itself highly configurable.  We want to provide developers, for example, to be ability to easily implement 

 

  (a) a simple click-to-call application that provides two-way audio and users list

  (b) a simple video chat application that provides two way audio and video (no user's list or chat)

  (c) a simple chat application that provides a users list and (you guessed it) chat

We're not there yet (still focusing on building BigBlueButton out as a single application).  The idea from the above list is that you could implement it with the same client, but through a configuration file/API calls you could disable/enable/hide specific features and the client would still work.  For example for (b) the have the chat, presentation, desktop sharing disabled and the user's list is hidden.  

>  I could re-write the whiteboard feature I guess, but as it evolves and changes, figured, why reinvent the wheel!

We hope that instead of recreating such applications from scratch, others could build upon BigBlueButton.  On our end, we tend to focus our development efforts on making sure the core product is stable, easy to use, and implements the features of our target market (online learning).

Some of the use cases we design around are (in no particular order)     

      - tutoring and virtual office hours

      - student collaboration

      - flipped classrooms

      - online classes

Implicit in the above is we're not designing BigBlueButton to scale to 500 users (at least not initially anyway).  In analyzing large samples of meetings, very few online classes have over 100 students.  If we do very good job at implementing the above use cases for 100 students or less, we're covering off a large segment of the market.

For now, the idea running a BigBlueButton client in a virtual frame buffer (xvfb) and capturing the audio/video with ffmpeg and then streaming it to a CDN will get you to having a large audience.  There are use cases for that, but it's not the focus for the core product.

Regards,... Fred

--

You received this message because you are subscribed to the Google Groups "BigBlueButton-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bigbluebutton-...@googlegroups.com.
To post to this group, send email to bigblueb...@googlegroups.com.
Visit this group at https://groups.google.com/group/bigbluebutton-dev.
For more options, visit https://groups.google.com/d/optout.

--

BigBlueButton Developer

http://bigbluebutton.org/

@bigbluebutton

[abrend...@gmail.com]

Fred,

Thank you for the quick reply!

I think I had quite a stream of consciousness going last night, my apologies. I totally see the vision of this client now, and that part of what we are looking for fits in it, the rest will need to be a separate custom integration which is totally fine. I would rather not distract the core of the product, for our edge case.

Majority of our users will be small groups, or one on one with educators, so I think the client as is we will end up using quite a bit! And we are very excited to how it is coming along! If we see any customizations to the client that would make sense in a classroom scenario, I will float them by you guys and send in a PR, but for the most part it sounds like our customizations will be a bit off the rails.

Thank you for all the hard work on this! Very excited to start checking this out more

On Wednesday, August 22, 2018 at 10:33:25 AM UTC-4, abrend...@gmail.com wrote: