How to setup Nginx as a load balancer using the StrongLoop Nginx Controller

[Stack Overflow]

I'm attempting to setup Nginx as a load balancer using the StrongLoop Nginx Controller. Nginx will be acting as a load balancer for a StrongLoop LoopBack application hosted by the standalone StrongLoop Process Manager. However, I've been unsuccessful at making the Nginx deployment following [the official directions][1] from StrongLoop. Here are the steps I've taken:

**Step #1** -- My first step was to install Nginx and the StrongLoop Nginx Controller on an AWS EC2 instance. I launched an EC2 sever (Ubuntu 14.04) to host the load balancer, and attached an Elastic IP to the server. Then I executed the following commands:

    $ ssh -i ~/mykey.pem ubuntu@[nginx-ec2-ip-address]

    $ sudo apt-get update

    $ sudo apt-get install nginx

    $ sudo apt-get install build-essential

    $ curl -sL https://deb.nodesource.com/setup_0.12 | sudo bash -

    $ sudo apt-get install -y nodejs

    $ sudo npm install -g strong-nginx-controller 

    $ sudo sl-nginx-ctl-install -c 444

Then I opened up port 444 in the security group of the EC2 instance using a Custom TCP Rule.

**Step #2** -- My second step was to setup two Loopback application servers. To accomplish this I launched two more EC2 servers (both Ubuntu 14.04) for the application servers, and attached an Elastic IP to each server. Then I ran the following series of commands, once on each application server:

    $ ssh -i ~/mykey.pem ubuntu@[application-server-ec2-ip-address]

    $ sudo apt-get update

    $ sudo apt-get install build-essential

    $ curl -sL https://deb.nodesource.com/setup_0.12 | sudo bash -

    $ sudo apt-get install -y nodejs

    $ sudo npm install -g strong-pm

    $ sudo sl-pm-install

    $ sudo /sbin/initctl start strong-pm

**Step #3** -- My third step was to deploy the application to each of the application servers. For this I used StrongLoop Arc:

    $ cd /path/to/loopback-getting-started-intermediate # my application

    $ slc arc

Once in the StrongLoop Arc web console, I built a tar for the application, and deployed it to both application servers. Then in the Arc Process Manager, I connected to both application servers. Once connected, I clicked "load balancer," and entered the Nginx host and port into the form and pressed save. This caused a message to pop up saying "load balancer config saved."

*Something strange happened at this point:* The fields in StrongLoop Arc where I just typed the settings for the load balancer (host and port) reverted back to the original values the fields had before I started typing. (The original port value was 555 and the original value in the host field was the address of my second application server.)

**Don't know what to do next** -- This is where I really don't know what to do next. (I tried opening my web browser and navigating to the IP address of the Nginx load balancer, using several different port values. I tried 80, 8080, 3001, and 80, having opened up each in the security group, in an attempt to find the place to which I need to navigate in order to see "load balancing" in action. However, I saw nothing by navigating to each of these places, with the exception of port 80 which served up the "welcome to Nginx page," not what I'm looking for.)

How do I setup Nginx as a load balancer using the StrongLoop Nginx Controller? What's the next step in the process, assuming all of my steps listed are correct.

  [1]: http://docs.strongloop.com/display/SLC/Configuring+Nginx+load+balancer

[Krishna Raman]

On Thursday, June 11, 2015 at 7:24:40 PM UTC-7, Stack Overflow wrote:

I'm attempting to setup Nginx as a load balancer using the StrongLoop Nginx Controller. Nginx will be acting as a load balancer for a StrongLoop LoopBack application hosted by the standalone StrongLoop Process Manager. However, I've been unsuccessful at making the Nginx deployment following [the official directions][1] from StrongLoop. Here are the steps I've taken:

**Step #1** -- My first step was to install Nginx and the StrongLoop Nginx Controller on an AWS EC2 instance. I launched an EC2 sever (Ubuntu 14.04) to host the load balancer, and attached an Elastic IP to the server. Then I executed the following commands:

    $ ssh -i ~/mykey.pem ubuntu@[nginx-ec2-ip-address]

    $ sudo apt-get update

    $ sudo apt-get install nginx

    $ sudo apt-get install build-essential

    $ curl -sL https://deb.nodesource.com/setup_0.12 | sudo bash -

    $ sudo apt-get install -y nodejs

    $ sudo npm install -g strong-nginx-controller 

    $ sudo sl-nginx-ctl-install -c 444

Then I opened up port 444 in the security group of the EC2 instance using a Custom TCP Rule.

The "welcome to Nginx" page you mention is there because you installed the nginx package which is probably configured to listen on port 80 by default. You can stop the nginx service to get rid of this page.

At this point you have correctly configured nginx controller to listen on port 444 for control commands from Arc. Since you did not specify -l flag, it will default to port 8080 for application traffic.

One concern I have here is that ports less than 1024 (including 444) are privileged ports and the user that is created when you did sl-nginx-ctl-install may not have permission to listen on this port. This would cause the service to not start. Can you check the status of the nginx-controller service and also do a `ps` on the machine to see if its actually running sl-nginx-ctl and nginx server?

Also can you verify that you can access the nginx controller (port 444) from the machine you are running Arc on. 

HTH

Krishna

[Stack Overflow]

Hello Krishna (and everybody else), thank you for your guidance!

It doesn't look like the StrongLoop Nginx Controller is listening on the control port. I used the "lsof" command to list the open ports on the host, and the control port is not one of those listed. How do I get the StrongLoop Nginx Controller to listen on the control port? Here's a detailed break down:

I terminated all of my EC2 instances, and ran though all the steps described in my original post again, but this time instead of choosing a control port of 444, I picked a port above 1024 to address your concern about privileged ports:

$ sudo sl-nginx-ctl-install -c 4444

Interestingly, it wasn't until rebooting the instance that the StrongLoop Nginx Controller process became active. Before rebooting the instance, running "ps aux | grep nginx" returned nothing, but after the reboot it returns:

999       1905 82.0  6.8 653972 69956 ?        Rsl  14:36   0:00 /usr/bin/nodejs /usr/lib/node_modules/strong-nginx-controller/bin/sl-nginx-ctl.js --control 4444 --listen http://0.0.0.0:8080 --base /var/lib/strong-nginx-controller/.strong-nginx-controller

You mentioned that the welcome page is there because of the installation of the Nginx package, and recommended stoping the Nginx service to remove the page. Does this mean that I don't need to install the Nginx package to use the StrongLoop Nginx Controller? In other words, does the Nginx Controller come bundled with Nginx itself? To explore the possibility that the Nginx package many not be necessary, I installed the Nginx Controller without the Nginx package, and ran the following command:

$ sudo lsof -i -n -P

COMMAND    PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME

dhclient   595   root    5u  IPv4   7826      0t0  UDP *:68 

dhclient   595   root   20u  IPv4   7759      0t0  UDP *:16120 

dhclient   595   root   21u  IPv6   7760      0t0  UDP *:50048 

sshd       936   root    3u  IPv4   8731      0t0  TCP *:22 (LISTEN)

sshd       936   root    4u  IPv6   8733      0t0  TCP *:22 (LISTEN)

sshd     24223   root    3u  IPv4  60033      0t0  TCP [private ip of nginx host]:22->[my home ip]:62887 (ESTABLISHED)

sshd     24274 ubuntu    3u  IPv4  60033      0t0  TCP [private ip of nginx host]:22->[my home ip]:62887 (ESTABLISHED)

As you can see, the Nginx Controller is not listening on any ports, although the process is active, as you can see from the output above from when I ran "ps aux | grep nginx." Next, I installed the Nginx package, and ran the command to list open ports again:

$ sudo lsof -i -n -P

COMMAND    PID                    USER   FD   TYPE DEVICE SIZE/OFF NODE NAME

dhclient   595                    root    5u  IPv4   7826      0t0  UDP *:68 

dhclient   595                    root   20u  IPv4   7759      0t0  UDP *:16120 

dhclient   595                    root   21u  IPv6   7760      0t0  UDP *:50048 

sshd       936                    root    3u  IPv4   8731      0t0  TCP *:22 (LISTEN)

sshd       936                    root    4u  IPv6   8733      0t0  TCP *:22 (LISTEN)

sshd     24223                    root    3u  IPv4  60033      0t0  TCP [private ip of nginx host]:22->[my home ip]:62887 (ESTABLISHED)

sshd     24274                  ubuntu    3u  IPv4  60033      0t0  TCP [private ip of nginx host]:22->[my home ip]:62887 (ESTABLISHED)

nodejs   25152 strong-nginx-controller   10u  IPv4  61951      0t0  TCP *:3000 (LISTEN)

nginx    25193                    root    6u  IPv4  61933      0t0  TCP *:80 (LISTEN)

nginx    25193                    root    7u  IPv6  61934      0t0  TCP *:80 (LISTEN)

nginx    25194                www-data    6u  IPv4  61933      0t0  TCP *:80 (LISTEN)

nginx    25194                www-data    7u  IPv6  61934      0t0  TCP *:80 (LISTEN)

nginx    25195                www-data    6u  IPv4  61933      0t0  TCP *:80 (LISTEN)

nginx    25195                www-data    7u  IPv6  61934      0t0  TCP *:80 (LISTEN)

nginx    25196                www-data    6u  IPv4  61933      0t0  TCP *:80 (LISTEN)

nginx    25196                www-data    7u  IPv6  61934      0t0  TCP *:80 (LISTEN)

nginx    25197                www-data    6u  IPv4  61933      0t0  TCP *:80 (LISTEN)

nginx    25197                www-data    7u  IPv6  61934      0t0  TCP *:80 (LISTEN)

nginx    25212 strong-nginx-controller    5u  IPv4  61959      0t0  TCP *:8080 (LISTEN)

nginx    25213 strong-nginx-controller    5u  IPv4  61959      0t0  TCP *:8080 (LISTEN)

The Nginx service is now active. As you can see, not only is Nginx now listening on some ports, but the Nginx Controller is now listening on some ports. This leads me to conclude that the Nginx package is indeed necessary to use the Nginx Controller. What's important to note is that the Nginx Controller isn't listening on the control port, port 4444. How do I get the StrongLoop Nginx Controller to listen on the control port?

I tried navigating my web browser to port 8080 of the address of the server running Nginx, and received the following message:

http://[public ip of nginx host]:8080/

503: Service Unavailable

I was expecting that navigating to this address would allow me to see load balancing in action, but got this error message instead. The next thing I tried was to stop Nginx, and run the command which lists open ports again:

$ sudo service nginx stop

$ sudo lsof -i -n -P

COMMAND    PID                    USER   FD   TYPE DEVICE SIZE/OFF NODE NAME

dhclient   595                    root    5u  IPv4   7826      0t0  UDP *:68 

dhclient   595                    root   20u  IPv4   7759      0t0  UDP *:16120 

dhclient   595                    root   21u  IPv6   7760      0t0  UDP *:50048 

sshd       936                    root    3u  IPv4   8731      0t0  TCP *:22 (LISTEN)

sshd       936                    root    4u  IPv6   8733      0t0  TCP *:22 (LISTEN)

sshd     24223                    root    3u  IPv4  60033      0t0  TCP [private ip of nginx host]:22->[my home ip]:62887 (ESTABLISHED)

sshd     24274                  ubuntu    3u  IPv4  60033      0t0  TCP [private ip of nginx host]:22->[my home ip]:62887 (ESTABLISHED)

nodejs   25152 strong-nginx-controller   10u  IPv4  61951      0t0  TCP *:3000 (LISTEN)

nodejs   25152 strong-nginx-controller   11u  IPv4  62173      0t0  TCP [private ip of nginx host]:3000->[my home ip]:49591 (ESTABLISHED)

nginx    25212 strong-nginx-controller    5u  IPv4  61959      0t0  TCP *:8080 (LISTEN)

nginx    25213 strong-nginx-controller    5u  IPv4  61959      0t0  TCP *:8080 (LISTEN)

nginx    25213 strong-nginx-controller    6u  IPv4  62171      0t0  TCP [private ip of nginx host]:8080->[my home ip]:49471 (ESTABLISHED)

As you can see, the Nginx Controller is still listening on the same ports as before, but still not on the control port. It would seem that it's necessary to have the Nginx package installed to use the Nginx Controller, but that it's not necessary to have Nginx started to use the Nginx Controller. Is this correct?

Incidentally, in my original post I neglected to mention that my "step 2" involved opening ports 8701 and 3001 in the security group.

Thanks again.

[Krishna Raman]

Hi,

It doesn't look like the StrongLoop Nginx Controller is listening on the control port. I used the "lsof" command to list the open ports on the host, and the control port is not one of those listed. How do I get the StrongLoop Nginx Controller to listen on the control port? Here's a detailed break down:

As I suspected. The controller was unable to start since it was unable to use 444 (privileged port)

 

I terminated all of my EC2 instances, and ran though all the steps described in my original post again, but this time instead of choosing a control port of 444, I picked a port above 1024 to address your concern about privileged ports:

$ sudo sl-nginx-ctl-install -c 4444

Interestingly, it wasn't until rebooting the instance that the StrongLoop Nginx Controller process became active. Before rebooting the instance, running "ps aux | grep nginx" returned nothing, but after the reboot it returns:

999       1905 82.0  6.8 653972 69956 ?        Rsl  14:36   0:00 /usr/bin/nodejs /usr/lib/node_modules/strong-nginx-controller/bin/sl-nginx-ctl.js --control 4444 --listen http://0.0.0.0:8080 --base /var/lib/strong-nginx-controller/.strong-nginx-controller

Ok, looks like you got it running at this point. The control port is 4444, this is what you put into Arc. And you should be able to connect to 8080 to connect to your app.

 

You mentioned that the welcome page is there because of the installation of the Nginx package, and recommended stoping the Nginx service to remove the page. Does this mean that I don't need to install the Nginx package to use the StrongLoop Nginx Controller? In other words, does the Nginx Controller come bundled with Nginx itself? To explore the possibility that the Nginx package many not be necessary, I installed the Nginx Controller without the Nginx package, and ran the following command:

You need the Nginx package for the Nginx binaries however, you dont need the Nginx service to be started.

 

$ sudo lsof -i -n -P

COMMAND    PID                    USER   FD   TYPE DEVICE SIZE/OFF NODE NAME

 

nginx    25193                    root    6u  IPv4  61933      0t0  TCP *:80 (LISTEN)

nginx    25193                    root    7u  IPv6  61934      0t0  TCP *:80 (LISTEN)

nginx    25194                www-data    6u  IPv4  61933      0t0  TCP *:80 (LISTEN)

nginx    25194                www-data    7u  IPv6  61934      0t0  TCP *:80 (LISTEN)

nginx    25195                www-data    6u  IPv4  61933      0t0  TCP *:80 (LISTEN)

nginx    25195                www-data    7u  IPv6  61934      0t0  TCP *:80 (LISTEN)

nginx    25196                www-data    6u  IPv4  61933      0t0  TCP *:80 (LISTEN)

nginx    25196                www-data    7u  IPv6  61934      0t0  TCP *:80 (LISTEN)

nginx    25197                www-data    6u  IPv4  61933      0t0  TCP *:80 (LISTEN)

nginx    25197                www-data    7u  IPv6  61934      0t0  TCP *:80 (LISTEN)

^^^ This is the Nginx service which is not needed

 

nodejs   25152 strong-nginx-controller   10u  IPv4  61951      0t0  TCP *:3000 (LISTEN)

nginx    25212 strong-nginx-controller    5u  IPv4  61959      0t0  TCP *:8080 (LISTEN)

nginx    25213 strong-nginx-controller    5u  IPv4  61959      0t0  TCP *:8080 (LISTEN)

^^^ This is the nginx controller which is expected altough I am not sure yet why its listening to 3000 instead of 4444.

 

I tried navigating my web browser to port 8080 of the address of the server running Nginx, and received the following message:

http://[public ip of nginx host]:8080/

503: Service Unavailable

This is because Arc was most likely unable to connect to the Nginx controller to configure the backends where your app is running.

As you can see, the Nginx Controller is still listening on the same ports as before, but still not on the control port. It would seem that it's necessary to have the Nginx package installed to use the Nginx Controller, but that it's not necessary to have Nginx started to use the Nginx Controller. Is this correct?

Yes, exactly.

 

So the next question is why is it listening on 3000 instead of 4444. Based on the command line you have pasted above, it looks like you give it just "--control 4444" instead of "--control http://:4444" . Can you try again with "http://:4444" and let me know how that works for you.

HTH

Krishna

[Stack Overflow]

Good news, implementing your suggestion has brought me limited success. The Nginx Controller is now listening on port 4444 after using "--control http://:4444" on installation. However, I'm still not able to connect to my application on port 8080. When I navigate my web browser to port 8080 of the address of the server running Nginx, I still receive the following message:

http://[public ip of nginx host]:8080/

503: Service Unavailable

Why am I still seeing this error, considering that Arc should now be able to connect to the Nginx Controller? Shouldn't I be seeing my web application through the load balancer at this point?

To review, I'll summarize the current state of my deployment:

I've got two application servers running on AWS EC2 instances, each running Strong PM. In the security group of each instance, ports 8701, 22, and 3001 are open. I'm able to navigate to http://[public ip of app server]:3001 for each of my application servers, and see my web application, independent of the load balancer. The application was deployed to each application server using Arc.

I've got Nginx and the Nginx Controller installed on another EC2 instance with the security group providing access to open ports 8080, 22, and 4444. The Nginx Controller is listening on port 4444, as evidenced by:

$ sudo lsof -i -n -P | grep strong-nginx-controller

nodejs    886 strong-nginx-controller   10u  IPv4   9116      0t0  TCP *:4444 (LISTEN)

nginx    1116 strong-nginx-controller    5u  IPv4   9124      0t0  TCP *:8080 (LISTEN)

nginx    1117 strong-nginx-controller    5u  IPv4   9124      0t0  TCP *:8080 (LISTEN)

From the process manager screen in Arc, I'm able to add and activate each PM host successfully, using the IP address of each and port 8701. However, as described in my previous post, a strange thing happens after I enter the IP address for the load balancer and port 4444 into the Arc load balancer screen (within the process manager screen.) After pressing save, the "load balancer config saved" message pops up, but then the values in the fields for load balancer host and port revert back to their original values.

Is there anything I need to do next to get this deployment wired together and working? What's the cause of the "service unavailable" message now that the Nginx Controller has an open control port? Thank you!

[Sergi Kolesnik]

Same here. I can't figure out what's next? If Strong pm is listening on port 3000, where's proxy pass for nginx-controller to pass requests to that service? I'm buffled.

[Cloud Pilot]

hey, some impressions on my attempts whcih ended at the same point.

how would the Process Manager let the strong-nginx-controller know how to map Applications on port `3000 + service ID` ? 

i have been able to inspect the nginx config file is generated to run the `sl-nginx-ctl.js` and it looks like below, redirecting to 503 !?

```

  server {

    listen 0.0.0.0:80;

    root /usr/local/lib/node_modules/strong-nginx-controller/lib/html;

    error_page 503 /503.html;

    location / {

      return 503;

    }

    location = /503.html {

      internal;

    }

  }

```

```

slc pm

slc pm(13358): StrongLoop PM v5.2.0 (API v6.1.0) on port `8701`

slc pm(13358): Base folder `/home/user/.strong-pm`

slc pm(13358): Applications on port `3000 + service ID`

Browse your REST API at http://127.0.0.1:8701/explorer

```

[Rohit Raghuvanshi]

just access your api under http://<hostname>:<nginxxport>/<path-to-access-api> for me it was http://localhost:8088/api. Actually once you install strong-nginx-controller everything is controlled by strong-nginx-controller so the proxy pass.