Envoy xDS API to disable a backend host in a cluster

268 views
Skip to first unread message

feng...@airbnb.com

unread,
Apr 30, 2018, 6:50:32 PM4/30/18
to envoy-users
hi,

Is there an API such that the xDS service can explicitly ask Envoy to take a backend host out of a cluster?

Thanks
Feng

feng...@airbnb.com

unread,
Apr 30, 2018, 6:51:53 PM4/30/18
to envoy-users
When I remove a host in the return of the EDS service, Envoy may still keep it in load balance as long as its active health check still succeeds.
So I wonder if there is a way to allow EDS to ask Envoy to take a host out 

Feng

Matt Klein

unread,
May 1, 2018, 12:46:19 AM5/1/18
to feng...@airbnb.com, envoy-users
Not currently. It wouldn't be that hard to implement as an option. It's tracked here: https://github.com/envoyproxy/envoy/issues/440

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users+unsubscribe@googlegroups.com.
To post to this group, send email to envoy...@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/320f614e-7fe1-4a8b-b91a-ad3fb7eb66b8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--

Harvey Tuch

unread,
May 1, 2018, 9:54:37 AM5/1/18
to Matt Klein, feng...@airbnb.com, envoy-users

On Tue, May 1, 2018 at 12:46 AM Matt Klein <mkl...@lyft.com> wrote:
Not currently. It wouldn't be that hard to implement as an option. It's tracked here: https://github.com/envoyproxy/envoy/issues/440
On Tue, May 1, 2018 at 12:51 AM, feng.pan via envoy-users <envoy...@googlegroups.com> wrote:
When I remove a host in the return of the EDS service, Envoy may still keep it in load balance as long as its active health check still succeeds.
So I wonder if there is a way to allow EDS to ask Envoy to take a host out 

Feng


On Monday, April 30, 2018 at 3:50:32 PM UTC-7, feng...@airbnb.com wrote:
hi,

Is there an API such that the xDS service can explicitly ask Envoy to take a backend host out of a cluster?

Thanks
Feng

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/320f614e-7fe1-4a8b-b91a-ad3fb7eb66b8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
--

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/CA%2BOFKgtaxSP_cQEj6sKJh1Fob-MxDpjqFFiFaJHt%2BVyAsQvkuA%40mail.gmail.com.

Matt Klein

unread,
May 1, 2018, 10:32:37 AM5/1/18
to Harvey Tuch, feng...@airbnb.com, envoy-users
Ah good point. I wonder if that feature is good enough for this request and we can call that issue resolved? WDYT?

On Tue, May 1, 2018 at 3:54 PM, Harvey Tuch <ht...@google.com> wrote:

On Tue, May 1, 2018 at 12:46 AM Matt Klein <mkl...@lyft.com> wrote:
Not currently. It wouldn't be that hard to implement as an option. It's tracked here: https://github.com/envoyproxy/envoy/issues/440
On Tue, May 1, 2018 at 12:51 AM, feng.pan via envoy-users <envoy...@googlegroups.com> wrote:
When I remove a host in the return of the EDS service, Envoy may still keep it in load balance as long as its active health check still succeeds.
So I wonder if there is a way to allow EDS to ask Envoy to take a host out 

Feng


On Monday, April 30, 2018 at 3:50:32 PM UTC-7, feng...@airbnb.com wrote:
hi,

Is there an API such that the xDS service can explicitly ask Envoy to take a backend host out of a cluster?

Thanks
Feng

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users+unsubscribe@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.
--

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users+unsubscribe@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/CA%2BOFKgtaxSP_cQEj6sKJh1Fob-MxDpjqFFiFaJHt%2BVyAsQvkuA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Harvey Tuch

unread,
May 1, 2018, 10:59:44 AM5/1/18
to Matt Klein, feng...@airbnb.com, envoy-users
I think so; the EDS delivered health status when indicating failure will override any other health check.

On Tue, May 1, 2018 at 10:32 AM Matt Klein <mkl...@lyft.com> wrote:
Ah good point. I wonder if that feature is good enough for this request and we can call that issue resolved? WDYT?
On Tue, May 1, 2018 at 3:54 PM, Harvey Tuch <ht...@google.com> wrote:

On Tue, May 1, 2018 at 12:46 AM Matt Klein <mkl...@lyft.com> wrote:
Not currently. It wouldn't be that hard to implement as an option. It's tracked here: https://github.com/envoyproxy/envoy/issues/440
On Tue, May 1, 2018 at 12:51 AM, feng.pan via envoy-users <envoy...@googlegroups.com> wrote:
When I remove a host in the return of the EDS service, Envoy may still keep it in load balance as long as its active health check still succeeds.
So I wonder if there is a way to allow EDS to ask Envoy to take a host out 

Feng


On Monday, April 30, 2018 at 3:50:32 PM UTC-7, feng...@airbnb.com wrote:
hi,

Is there an API such that the xDS service can explicitly ask Envoy to take a backend host out of a cluster?

Thanks
Feng

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.
--

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/CA%2BOFKgtaxSP_cQEj6sKJh1Fob-MxDpjqFFiFaJHt%2BVyAsQvkuA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Matt Klein

unread,
May 1, 2018, 11:01:17 AM5/1/18
to Harvey Tuch, feng...@airbnb.com, envoy-users
OK. I will think about a small doc change to make it more clear that this feature exists in the arch docs.

On Tue, May 1, 2018 at 4:59 PM, Harvey Tuch <ht...@google.com> wrote:
I think so; the EDS delivered health status when indicating failure will override any other health check.
On Tue, May 1, 2018 at 10:32 AM Matt Klein <mkl...@lyft.com> wrote:
Ah good point. I wonder if that feature is good enough for this request and we can call that issue resolved? WDYT?
On Tue, May 1, 2018 at 3:54 PM, Harvey Tuch <ht...@google.com> wrote:
On Tue, May 1, 2018 at 12:46 AM Matt Klein <mkl...@lyft.com> wrote:
Not currently. It wouldn't be that hard to implement as an option. It's tracked here: https://github.com/envoyproxy/envoy/issues/440
On Tue, May 1, 2018 at 12:51 AM, feng.pan via envoy-users <envoy...@googlegroups.com> wrote:
When I remove a host in the return of the EDS service, Envoy may still keep it in load balance as long as its active health check still succeeds.
So I wonder if there is a way to allow EDS to ask Envoy to take a host out 

Feng


On Monday, April 30, 2018 at 3:50:32 PM UTC-7, feng...@airbnb.com wrote:
hi,

Is there an API such that the xDS service can explicitly ask Envoy to take a backend host out of a cluster?

Thanks
Feng

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users+unsubscribe@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.
--

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users+unsubscribe@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.

feng...@airbnb.com

unread,
May 1, 2018, 12:20:11 PM5/1/18
to envoy-users
This is helpful. Thank you!



On Tuesday, May 1, 2018 at 8:01:17 AM UTC-7, Matt Klein wrote:
OK. I will think about a small doc change to make it more clear that this feature exists in the arch docs.
On Tue, May 1, 2018 at 4:59 PM, Harvey Tuch <ht...@google.com> wrote:
I think so; the EDS delivered health status when indicating failure will override any other health check.
On Tue, May 1, 2018 at 10:32 AM Matt Klein <mkl...@lyft.com> wrote:
Ah good point. I wonder if that feature is good enough for this request and we can call that issue resolved? WDYT?
On Tue, May 1, 2018 at 3:54 PM, Harvey Tuch <ht...@google.com> wrote:
On Tue, May 1, 2018 at 12:46 AM Matt Klein <mkl...@lyft.com> wrote:
Not currently. It wouldn't be that hard to implement as an option. It's tracked here: https://github.com/envoyproxy/envoy/issues/440
On Tue, May 1, 2018 at 12:51 AM, feng.pan via envoy-users <envoy...@googlegroups.com> wrote:
When I remove a host in the return of the EDS service, Envoy may still keep it in load balance as long as its active health check still succeeds.
So I wonder if there is a way to allow EDS to ask Envoy to take a host out 

Feng


On Monday, April 30, 2018 at 3:50:32 PM UTC-7, feng...@airbnb.com wrote:
hi,

Is there an API such that the xDS service can explicitly ask Envoy to take a backend host out of a cluster?

Thanks
Feng

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.

Feng Pan

unread,
May 8, 2018, 2:25:12 AM5/8/18
to Matt Klein, Harvey Tuch, envoy-users
hi Harvey/Matt,

I tried the suggested method to let EDS to override the health status. It works fine. 

There seems to be one caveat though. As document in here, the only time that Envoy purges hosts is when they fail HC and also they are absent from xDS response. It seems not working when the healthy status is override by EDS. In my test, once I have EDS to return 'unheathy' status, I can see that the host is tagged as failed_eds_health in Envoy. However, it seems that the host is kept there even after it is removed from the response of EDS later. Is that expected behavior or a bug in this code path?

Thanks,
Feng
 

Harvey Tuch

unread,
May 8, 2018, 10:52:49 AM5/8/18
to Feng Pan, Matt Klein, envoy-users

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.

To post to this group, send email to envoy...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages