How to load balance a phoenix application?

[Railsmechanic]

Hi!

As a start and to get familiar with phoenix we're (at our company) rewriting an existing RESTful API with phoenix.

So far, everything works as expected, but as the deployment comes closer we need some information on how to do a load balanced setup with phoenix.

For other (non Elixir) setups we would do the load balancing the "old fashioned way":

 1. Put a load balancer in front

 2. Start the web application on a bunch of machines 

 3. Let the load balancer distribute the load among this machines

But after reading the impressive "The Road to 2 Million Websocket Connections in Phoenix", where the phoenix application is setup as a cluster on multiple nodes to handle this amount of connections, I don't know whether a clustered setup is only working with channels. Is this technique also suitable to setup a cluster of nodes to automatically balance the (http) load of our API project, or is this PubSub-only thing?

So, does phoenix offer an "internal" option to distribute load through elixir nodes or is the "old fashioned setup" the way to go?

Many thanks.

[Michał Muskała]

AFAIK in the "The Road to 2 Million Websocket Connections in Phoenix"
they are describing a single phoenix application working on a single
node. This one node was receiving all the connections.

Michal.

> --
> You received this message because you are subscribed to the Google Groups
> "phoenix-talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to phoenix-talk...@googlegroups.com.
> To post to this group, send email to phoeni...@googlegroups.com.
> Visit this group at https://groups.google.com/group/phoenix-talk.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/phoenix-talk/aebc75f8-1e89-4a9b-a0bb-b0bf217ae1b2%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Steve Domin]

Michal is correct on the blog post setup so, to answer your question, the old fashion way works really well with Phoenix, even if you are using channels.

To view this discussion on the web visit https://groups.google.com/d/msgid/phoenix-talk/CAGAFNp%3DpP6yKo%3D5M08YfiERkcOdwEsKXz0BQwOZJry98tdqmAA%40mail.gmail.com.

[Cristian Garcia]

I think (correct me if I am wrong) that the whole purpose of load balancing within the same machine (does not apply load balancing multiple nodes) is that the language + framework you are using runs on a single system thread and/or doesn't support asynchronous calls. I guess Rails, Django, .NET < 5, and many Java frameworks follow this scheme.

In Erlang/Elixir "Load Balancing" between all your cores is done by the VM (BEAM) for you. Phoenix creates a new process per request/connection and the VM decides how it distributes work as evenly as possible. Some times this is just a little too beautiful.

To view this discussion on the web visit https://groups.google.com/d/msgid/phoenix-talk/CAB-Ba_ZxrFmQDvedtxvzJCZKVH_LJHiM8YS_ZhfGEDyugjmNDA%40mail.gmail.com.

[Michael Schaefermeyer]

To echo what has been said before:

• For ye good old REST any load balancer (HA Proxy, AWS ELB, ...) works perfectly. No extra set up needed. You can treat Phoenix like any rails app you have experience deploying!
• For web sockets (not REST!) we were worried about how multiple users in the same channel would "hear" one another when on different servers. Phoenix comes equipped for this: There is a PubSub layer that allows you to connect multiple isolated instances via a multitude of PubSub services (Postgres, RabbitMQ, Redis, ...). Again, this is only needed if you use web sockets. 

So to answer your question: Load balancing (if talking about the one across different nodes) works very well in both scenarios. Feel free to ask further questions.

Cheers

[mads.ha...@gmail.com]

Hi

I have an application that exposes both a REST api, while also having a websocket layer. What would be your recommended approach for this? Can I just pop an AWS Elastic loadbalancer infront, or does BEAM handle this for me?

[Sonny Scroggin]

Have a look at this guide: http://www.phoenixframework.org/docs/serving-your-application-behind-a-proxy

To view this discussion on the web visit https://groups.google.com/d/msgid/phoenix-talk/fde7dff8-543a-4d97-a016-44dc8fc8361e%40googlegroups.com.

[Peter Hamilton]

You'll want to use Amazon's new Application Load Balancer for websockets, as ELB's only support TCP load balancing of websockets.

To view this discussion on the web visit https://groups.google.com/d/msgid/phoenix-talk/CAP2ZXUUYD4vLJnrwAMeTnxU2H%3DE%3DOkv3zawkb%3DGFVXRS2nK6Fw%40mail.gmail.com.

[Jordan Day]

I'd love to read more about the second scenario -- multiple backend instances serving clients communicating over channels (websockets transport). This is something I'm just starting to dig into so I'm not even sure what questions to ask, but basically, if I have 3 backend nodes balancing traffic between a few thousand clients, what do I need to do to keep the all the clients online if one of the backend nodes goes down?

[Peter Hamilton]

The Phoenix client has reconnect logic. So if a connection dies it will reconnect. If all connections go through the load balancer, then on reconnect it will find a healthy node.

The one caveat here is a partial split brain scenario. If backends A & B can't communicate with backend C, but all three are connected to the load balancer, then your clients will not all be able to communicate with each other. I don't believe Phoenix solves that case presently. It just publishes to available nodes, causing broadcasts not to be delivered across the partition.

To view this discussion on the web visit https://groups.google.com/d/msgid/phoenix-talk/3e659a0b-e672-479d-8dd1-1c0ba1fa13d0%40googlegroups.com.

[Esdras Mayrink]

I've been looking into this in the past weeks, I even posted a question on SO[1].

I'm assuming that you are talking about websockets because other scenarios are well known.

The short answer is: if you are using AWS, use the new ELB, the so called Application Load Balancer, it's cheap and you won't have to worry about anything.

If you have to build your own, I'd recommend nginx, but, and here is the catch, as far as I know, you'll only be able to connect at most 64k clients per phoenix server simultaneously, this is because, to every client the LB creates two connections, one from the client's machine to the LB server and other from the LB to the phoenix server. In short, every connection is stored in a unique four element tuple by the OS: (origin_ip, origin_port, target_ip, target_port). So for every connection to a upstream server, the OS uses one of the ephemeral ports[2], and, if you're talking about millions of connection per upstream server 64k ports won't be enough. The solution is to attach more than one ip to the LB machine and distribute outbound connections between them, you can do this easily on nginx using the split_clients directive, se and example here[3]

I'm assuming that the backend servers have static IPs and are of fixed numbers, if you are using a elastic infrastructure where phoenix machines come and go based on traffic and have random IPs, than I would use Openresty with the balancer_by_lua directive[4]. Openresty is a fantastic technology, worthy looking into if you're dealing with this stuff.

You can read more about load balancing WS here[5]

Also, you'll probably have to increase the ephemeral port range on the LB machine, among other things[6]

Good luck.

[1] http://stackoverflow.com/questions/39312706/how-to-distribute-persistent-connection-across-a-elixir-phoenix-cluster

[2] https://en.wikipedia.org/wiki/Ephemeral_port

[3] https://www.nginx.com/blog/overcoming-ephemeral-port-exhaustion-nginx-plus

[4] https://github.com/openresty/lua-nginx-module#balancer_by_lua_block
[5] https://deepstream.io/blog/load-balancing-websocket-connections

[6] https://mrotaru.wordpress.com/2013/10/10/scaling-to-12-million-concurrent-connections-how-migratorydata-did-it/