Load balancer drops backend while leaving frontend connected

12 views
Skip to first unread message

Tyler Johnson

unread,
Mar 29, 2018, 11:42:35 PM3/29/18
to Kubernetes user discussion and Q&A
Is it possible that an HTTP load balancer (auto-configured as part of an Ingress) could occasionally drop backend connections while leaving the frontend connected?

I'm running a websocket backend service (the backend-service timeout is high) and on very rare occasions I'll see the service pod log that the client dropped connection, while on the client side the HTTP connection is still ESTABLISHED. So I'm guessing it must be the LB.

Is there a recommended way to troubleshoot the LB?

Any other potential scenarios that could cause this problem?  

Tim Hockin

unread,
Mar 29, 2018, 11:56:53 PM3/29/18
to Kubernetes user discussion and Q&A
Which environment and which Ingress controller?

--
You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-use...@googlegroups.com.
To post to this group, send email to kubernet...@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

Tyler Johnson

unread,
Mar 30, 2018, 10:42:42 AM3/30/18
to kubernet...@googlegroups.com
Environment: GKE 1.7.12-gke.1
I believe the ingress controller is GLBC[1]


On Thu, Mar 29, 2018 at 11:56 PM, 'Tim Hockin' via Kubernetes user discussion and Q&A <kubernet...@googlegroups.com> wrote:
Which environment and which Ingress controller?

On Thu, Mar 29, 2018 at 8:42 PM Tyler Johnson <tyler....@risevision.com> wrote:
Is it possible that an HTTP load balancer (auto-configured as part of an Ingress) could occasionally drop backend connections while leaving the frontend connected?

I'm running a websocket backend service (the backend-service timeout is high) and on very rare occasions I'll see the service pod log that the client dropped connection, while on the client side the HTTP connection is still ESTABLISHED. So I'm guessing it must be the LB.

Is there a recommended way to troubleshoot the LB?

Any other potential scenarios that could cause this problem?  

--
You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-users/LQJCmd3SYDE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kubernetes-users+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.

Nicks

unread,
Apr 4, 2018, 11:39:46 AM4/4/18
to Kubernetes user discussion and Q&A
Did you remove a node from the cluster? Even if a node was in the cluster and didn't run any workload, it could still be used to proxy connections from the LB and the service pod. 


On Friday, March 30, 2018 at 7:42:42 AM UTC-7, Tyler Johnson wrote:
Environment: GKE 1.7.12-gke.1
I believe the ingress controller is GLBC[1]

On Thu, Mar 29, 2018 at 11:56 PM, 'Tim Hockin' via Kubernetes user discussion and Q&A <kubernet...@googlegroups.com> wrote:
Which environment and which Ingress controller?

On Thu, Mar 29, 2018 at 8:42 PM Tyler Johnson <tyler....@risevision.com> wrote:
Is it possible that an HTTP load balancer (auto-configured as part of an Ingress) could occasionally drop backend connections while leaving the frontend connected?

I'm running a websocket backend service (the backend-service timeout is high) and on very rare occasions I'll see the service pod log that the client dropped connection, while on the client side the HTTP connection is still ESTABLISHED. So I'm guessing it must be the LB.

Is there a recommended way to troubleshoot the LB?

Any other potential scenarios that could cause this problem?  

--
You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-use...@googlegroups.com.
To post to this group, send email to kubernet...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-users/LQJCmd3SYDE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kubernetes-use...@googlegroups.com.
To post to this group, send email to kubernet...@googlegroups.com.

Tyler Johnson

unread,
Apr 4, 2018, 12:56:17 PM4/4/18
to kubernet...@googlegroups.com
No this is during regular operation.

To unsubscribe from this group and all its topics, send an email to kubernetes-users+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages