Scaling Big Blue Button with Scalelite design Considerations

[Sajith Thrimavithana]

Hi All, 

We have a BBB server (2.3.4) running integrated to Moodle for a School, and it's running great. However, we need to scale this up to support multiple schools (200+) which obviously require at least 4 -10 servers. 

After a searching in this forum we understand that we need;

1. Scalelite Server (with postgres db, Redis)

2. NFS server for recording storage (Centralized one mapped to all bbb servers) 

3. Greenlight server for front end. Meeting scheduling will done from moodle plugin. 

Now we assumed we can use one Public IP (for scalelite load balancer) and use private IPs for bbb server, and traffic will come through scalelite. We like to get this done very much since we are in the process of getting our Public IP whitelisted from ISPs in our country so children are not charged for their meetings. 

Is this possible ? From initial search, it seems we need public IPs for all bbb servers. 

And next issue is, when Scalelite is scheduling a meeting, how does it choose the server with least load ? Is Scalelite simply send any moodle meeting schedule request to any available server or does it consider actual meeting start / end time when checking server load ? For ex what is the free bbb server for a 4pm - 7pm meeting ? We are not clear how the load balancing part works.

Appreciate any support for this. We would like to know the optimal design for this please. If any of you have experience in deploying scalelite please share it too. 

Thank you, 

[basisbit]

> Now we assumed we can use one Public IP (for scalelite load balancer) and use private IPs for bbb server, and traffic will come through scalelite. We like to get this done very much since we are in the process of getting our Public IP whitelisted from ISPs in our country so children are not charged for their meetings. 

This is not how Scalelite works. It is not a proxy, but instead only forwards requests for creating and joining BBB rooms to the different BBB servers. You'll need a public IPv4 for each BBB server. If Scalelite would act as web proxy server instead, it would severely limit scaling BBB installations as eventually you would hit the network bandwidth limit of the Scalelite server.

With the architecture of Scalelite, you can operate hundreds or thousands of BBB servers while your moodle plugin or Greenlight only sees that one Scalelite server to talk to, and thus BBB installations can scale well for hundredthousands of concurrent users, if you can get the network and hardware resources for it. Personally, I'd suggest renting the resources at a big data center in your region or choosing infrastructure providers like Digital Ocean or Hetzner or Amazon Web Services and on-demand renting as much infrastructure as you need at the moment.

Scalelite of course does not know when your meetings start or end, thus it can not consider this data for scheduling where to put the meeting. Instead, it monitors how many meetings of what size and with how many enabled webcams are on the servers, and then tries to choose the server with the lowest calculated "load", whenever you send a room-creation-request.

[mailp...@gmail.com]

Instead of getting one Public IP, get a dedicated IP CIDR block which you can still whitelist as a single instruction and assign all BBB servers to the IP's within the CIDR block.

[Militades sunfire]

* if you want to you can put greenlight on your scalelite server and also have the box provide the NFS for the recordings.

* every bbb server needs it's own public IP. scalelite manamges where to open meetings, but does not route the traffic there. that would be a bad idea since the scalelite servers connection would likely be a bottleneck then ...

* scalelite got some simple load balancing as in it opens a new meeting on the server with the fewest meetings. that usually works fine as long as you don't have huge meetings (60+ ppl)

--
You received this message because you are subscribed to the Google Groups "BigBlueButton-Setup" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bigbluebutton-s...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bigbluebutton-setup/56029b57-5b51-44b0-ac59-015e721bdf6en%40googlegroups.com.

[Sajith Thrimavithana]

Thank you very much for your suggestions. Then the only way to proceed with is having public IP per server, correct ? 

We'll proceed that manner. 

[Michael Plotsker]

Where does a turn/co-turn/stun server fit into this diagram/setup, is it necessary, preferable or not needed? Also, can the mp4 single file recording project be integrated into this setup? BBB monitoring in this too?

what about ansible to reduce costs of having idle servers running?

Has anyone successfully set something like this up? It would be great to have a howto to do all or most of this together with server sizing info and offloading recordings to s3 etc... if anyone has the knowledge. 

I have been trying to put together a project plan but have not been able to get server sizing info for some of these projects.

Just a new years wish list.

Thanks for listening

Michael Plotsker