Too many 502 from LoadBalance

[Anu V]

I'am experiencing too many 502 errors from Load Balance and the instances are working fine. (

@type: "type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry"
statusDetails: "failed_to_connect_to_backend"

I've also found that this happen on my few projects starting about 12 hours ago, I've checked https://status.cloud.google.com/ and it said all working fine.

I cannot upgrade support level to create the ticket, so I am posting it here to ask if anyone having the same problem as I am?

[Navi Aujla (Google Cloud Support)]

Hello Anu,

The "failed_to_connect_to_backend" error message indicates that load balancer is failing to connect to backends. There could be many possible reasons for it. For example: Unhealthy instance if reported by health check, incorrect URL mapping, high backend utilization causing backend not to respond to the load balancer requests. You can refer to this thread for more information and check instance system/error and access logs. 

[Anu V]

Backends are healthy, I found this 502 errors across my few projects using AppEngine.

All instances are able to access and work normally, I've tested using curl to send a request directly to instance's private ip and it works all the time.

[Navi Aujla (Google Cloud Support)]

Hello Anu

As mentioned in my last message there are multiple factors which may cause this error. However, If you believe this to an issue with the GCP load balancer, I would suggest you to open a private issue report using this link[1] providing the relevant detailed information for the investigation. 

[Anu V]

20 hours after created a ticket support and post in issuetracker, the issue has been solved, still no explanation though.

Not to mention that I cannot upgrade my support plan for about 20 hours earlier until I need to call support team and hours laters I finally can upgrade support plan.

My frustration is it took so long and so hard to convince your engineers that it's problem on your end, not to mention how long it've been solve.

And yeah, google cloud status shown all green.

It took about 36 hours before P1 issue get fixed, really unacceptable.

[Suvodeep Pyne]

We are also facing the same issue. The LB is throwing 502's intermittently. 

A simple parallel curl with 30 requests is able to choke up the LB 5 - 8 times. Our App servers are healthy and are not receiving the requests.

[Suvodeep Pyne]

+1 Anu

This is the second time in the past 4 months we faced a P1 issue which took 2 days to get resolved.

@Google team, this is perhaps not the best customer experience. I am aware of your support tiers but there should be a better interaction model with your customer. We are paying for the service and you are answerable to us in case of outages / service disruption.

[Jonas Klemming]

I have an open support case for this as well. It seems to be a known issue they are working on for 1+ months. Not sure when it will be fixed.

[Vinay Kumar Dahiya]

Exactly the same error been happening with us as well since yesterday.

And the log viewer doesn't really give a verbose options in detailing the error.

the logs don't show which ip was used to send the request with from the load balancer which makes it super difficult to debug any issues.