Network Load Balancer- Instances are always unhealthy

2,959 views
Skip to first unread message

Jatan Ajmera

unread,
Sep 14, 2016, 2:49:13 PM9/14/16
to gce-discussion
Hi , 
I have a network load balancer on Port 5050, which has an target pool with 3 instances and a health check associated with it on the same port. My instances are perfectly healthy and running but it is showing unhealthy for the load balancer, so i cannot actually use the load balancer. I have tried everything mentioned on stackoverflow and this group, but have had no luck figuring out the actual problem.
I also checked for the google-address-manager, but that doesnt seem to exist on my instances. I am using CentOS 7. 

Thanks,
Jatan

Matt Hite

unread,
Sep 14, 2016, 6:00:25 PM9/14/16
to Jatan Ajmera, gce-discussion
Are you using a Google supplied CentOS image? 

AFAIK, if the google-address-manager doesn't exist on your system, you won't have the external address associated with the NLB automatically set up on the CentOS host and therefore will not be able to bind (or receive traffic destined to) the NLB address.

If you're going to supply your own image, you'll need to carefully follow these guidelines:



--
© 2016 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-discussion@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussion+unsubscribe@googlegroups.com.
To post to this group, send email to gce-discussion@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/aa09a4c5-384a-484a-95ea-f2666dea3f46%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Kamran (Google Cloud Support)

unread,
Sep 14, 2016, 6:00:46 PM9/14/16
to gce-dis...@googlegroups.com
Hello Jatan,

As described in this article:  

For a health check to be deemed successful, the instance must return a valid HTTP response with code 200 and close the connection normally within the timeoutSec period. If it does so, the health check service returns a status ofHEALTHY for that instance. 

So the first step for troubleshooting this issue is making sure that health check port is configured to send HTTP requests to the correct port which HTTP service on the servers is listening on and can respond with code 200. For example, if you are running multimedia conference tool on port 5050 and your health check is configured to send HTTP request on that port, the health check fails.

You can find more information on debugging health checks in load balancing on this GCP blog post.

Please let me know if you have further questions about HTTP health checks.

Sincerely,

Jatan Ajmera

unread,
Sep 14, 2016, 7:26:06 PM9/14/16
to gce-discussion
Hi, 
I have already been able to successfully curl to the instances in my pool as follows:
curl -IL ip.add.ress.10:5050/health and i got the following status:
HTTP/1.1 200 OK
Date: Wed, 14 Sep 2016 23:02:55 GMT
Content-Length: 0

I have tried this from both the within the google private network/across instances and from outside the network, and both have returned 200 OK. I've also double checked my firewall rules to include the CIDR blocks for the health checks as well.

Jatan Ajmera

unread,
Sep 14, 2016, 7:38:17 PM9/14/16
to gce-discussion
Hi,
I started a python server using the following command :

sudo python -m SimpleHTTPServer 80
aaand reconfigured the health check with port 80 and now it is showing up as healthy for all the instances.
So I am trying to figure out whether this is a bug or an issue of ACLS.

Thanks,
Jatan


Kamran (Google Cloud Support)

unread,
Sep 14, 2016, 7:46:37 PM9/14/16
to gce-discussion

If you were not able to figure out the cause of issue, you can email me your project ID and the name of LB, and I'll take a look at your health check settings.

Sincerely,

Костя Хомко

unread,
Jan 13, 2018, 4:01:46 PM1/13/18
to gce-discussion
Hello, 
please help me with this.

root@dev:~# curl -IL 35.***.**.***:8080/health
HTTP/1.1 404
Content-Type: text/html;charset=utf-8
Content-Language: en
Transfer-Encoding: chunked
Date: Sat, 13 Jan 2018 20:57:55 GMT

root@dev:~# curl -IL 35.***.**.***:80/health
HTTP/1.1 404 Not Found
Server: nginx/1.12.1 (Ubuntu)
Date: Sat, 13 Jan 2018 20:59:28 GMT
Content-Type: text/html
Content-Length: 178
Connection: keep-alive

Hot to pass the health check if i have nginx redirection rule enabled?

If to turn it off i simply get error code curl: (7) Failed to connect to 35.***.**.*** port 80: Connection refused
Reply all
Reply to author
Forward
0 new messages