Sharding in high-traffic scenarios

[David Ing]

As traffic increases, we cross a point where a single server is over-saturated with traffic.

Are there any easy solutions to encompass:

• Gracefully scaling a cluster up and down
• Load balancing

[Warren McDonald]

Round robin DNS is fine, if you just want to share load across available servers.  Since all of the allocations are done using IP address, it works fine if you use a single DNS name with multiple servers in the A record.  Adding a DNS health check is a good idea to make sure that if a server is down, it won't be included in the DNS answers. AWS Route53 can do this

This generally works best if you have multiple servers available at the start of your peak usage hours. You can get away with adding more as load increases, if you have relatively short TURN sessions. Not so well if you have lots of 60 min sessions, as you will still be sending requests to the already busy servers.

Scaling in is a bit harder, but not impossible. You can remove specific server entries from the DNS list, which will mean they won't get any new sessions. Knowing when a server is no longer relaying any sessions requires you to connect to one of the monitoring points like the telnet interface, web console or the configured stats DB for the turnserver process. Whilst this could be automated, it is not a trivial exercise for a dynamic server pool.  Again, if you have many short sessions, you could just add a wait period after removing the server from the DNS list, say average session time x 2, before shutting down a de-listed server.   

Having said all of that, it is worth reassessing you current server config to ensure you are balancing available network capacity, sockets and CPU. If you are nearing the limits of the NIC and still have available sockets and CPU you could look at adding NICs and reconfiguring the turnserver.conf to use them. If you are in cloud env, then look to higher network throughput instance type.

Cheers,

Warren

[David Ing]

Thanks, this comment is helpful.