Endpoints and Endpoints Slices controllers and API deprecation
517 views
Skip to first unread message
Antonio Ojea
Sep 10, 2020, 5:42:24 AM9/10/20
to kubernetes-sig-network
Hi all,
I was asked today if the Endpoints controller will be running in future releases, I would say yes, but honestly, I don't have it clear, so I think that we should clarify this for endpoints users.
Reading the KEP from endpoint slices [1], it says:
> Endpoints Controller (classic)
> In order to ensure backwards compatibility for external consumer of the core/v1 Endpoints API, the existing Endpoints controller will keep running until the API is EOL. After EndpointSlices become GA, the Endpoints controller will gradually limit functionality.
> In order to ensure backwards compatibility for external consumer of the core/v1 Endpoints API, the existing Endpoints controller will keep running until the API is EOL. After EndpointSlices become GA, the Endpoints controller will gradually limit functionality.
However, the Endpoints API is GA, and I can't see how it will reach EOL based on [2]
Thoughts?
Rob Scott
Sep 10, 2020, 12:42:32 PM9/10/20
to Antonio Ojea, kubernetes-sig-network, John Belamaric
That's a great question, thanks for bringing it up! I had a similar conversation on a docs PR for my EndpointSlice blog pots. Although I don't know of any plans to deprecate the Endpoints API, the interpretation of the deprecation policy in that thread was that Endpoints (or any v1 API) theoretically could be deprecated in the future given enough advance notice.
--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-network" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-ne...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-network/CABhP%3Dtba8if0shcT5vyWM8nraapgRWz_kC1MpcbhnnjZu-aE-g%40mail.gmail.com.
Bowei Du
Sep 10, 2020, 12:46:06 PM9/10/20
to Rob Scott, Antonio Ojea, kubernetes-sig-network, John Belamaric
I think it might be the wording -- the idea is that new features will land on EndpointSlice not that existing features will somehow be reduced ("... limit functionality" part).
Should we update so there is no confusion?
Bowei
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-network/CAGY4dkmgCQbO52yG6J%2BiyLUqwp8CwURRmEy1oyRZeYzP5bcN%3Dw%40mail.gmail.com.
Tim Hockin
Sep 10, 2020, 12:50:18 PM9/10/20
to Rob Scott, Antonio Ojea, kubernetes-sig-network, John Belamaric
You can only deprecate a v1 API via a v2 group, of which we have no
plans, for now. You should assume Endpoints will continue to exist
and the controller will be available for the indefinite future. We
will be obligated to maintain it, though we may choose not to enhance
it.
We can discuss whether we should remove Endpoints support from
kube-proxy, which seems reasonable to me.
We can also discuss introducing more concrete limits in Endpoints,
e.g. setting a deterministic size limit, but that will have to be done
carefully.
We can also discuss offering a way to disable the Endpoints controller
for cluster admins who want to, though that seems less useful than
changing the biggest consumers (kube-proxy and Ingress controllers) to
use EndpointSlice instead.
On Thu, Sep 10, 2020 at 9:42 AM 'Rob Scott' via kubernetes-sig-network
<kubernetes-...@googlegroups.com> wrote:
>
> To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-network/CAGY4dkmgCQbO52yG6J%2BiyLUqwp8CwURRmEy1oyRZeYzP5bcN%3Dw%40mail.gmail.com.
plans, for now. You should assume Endpoints will continue to exist
and the controller will be available for the indefinite future. We
will be obligated to maintain it, though we may choose not to enhance
it.
We can discuss whether we should remove Endpoints support from
kube-proxy, which seems reasonable to me.
We can also discuss introducing more concrete limits in Endpoints,
e.g. setting a deterministic size limit, but that will have to be done
carefully.
We can also discuss offering a way to disable the Endpoints controller
for cluster admins who want to, though that seems less useful than
changing the biggest consumers (kube-proxy and Ingress controllers) to
use EndpointSlice instead.
On Thu, Sep 10, 2020 at 9:42 AM 'Rob Scott' via kubernetes-sig-network
<kubernetes-...@googlegroups.com> wrote:
>
Tim Hockin
Sep 10, 2020, 12:52:47 PM9/10/20
to Rob Scott, Antonio Ojea, kubernetes-sig-network, John Belamaric
Just to be clear - we can (and probably should) formally deprecate
Endpoints - as in "this API exists and works, but you'd be better off
using EndpointSlice". But that's more documentation than tech.
Also, we should go through docs (e.g. the "Debug Services" doc) and
add references to EPSlice, when we hit GA.
Endpoints - as in "this API exists and works, but you'd be better off
using EndpointSlice". But that's more documentation than tech.
Also, we should go through docs (e.g. the "Debug Services" doc) and
add references to EPSlice, when we hit GA.
Rob Scott
Sep 10, 2020, 1:00:22 PM9/10/20
to Tim Hockin, Antonio Ojea, kubernetes-sig-network, John Belamaric
Thanks for the clarification! I've submitted a PR to clear this up in the KEP as well.
Antonio Ojea
Sep 10, 2020, 1:16:53 PM9/10/20
to Tim Hockin, Rob Scott, kubernetes-sig-network, John Belamaric
On Thu, 10 Sep 2020 at 18:50, Tim Hockin <tho...@google.com> wrote:
You can only deprecate a v1 API via a v2 group, of which we have no
plans, for now. You should assume Endpoints will continue to exist
and the controller will be available for the indefinite future. We
will be obligated to maintain it, though we may choose not to enhance
it.
We can discuss whether we should remove Endpoints support from
kube-proxy, which seems reasonable to me.
We can also discuss introducing more concrete limits in Endpoints,
e.g. setting a deterministic size limit, but that will have to be done
carefully.
We can also discuss offering a way to disable the Endpoints controller
for cluster admins who want to, though that seems less useful than
changing the biggest consumers (kube-proxy and Ingress controllers) to
use EndpointSlice instead.
I think we should do this, the other day @aanm in slack (sorry I don't know his name :-) ), was explaining different possible attacks creating endpoints with a particular set of labels that will be mirrored by the endpoint slice mirroring, allowing to "hijack" endpoints slices to steal services traffic.
I don't know exactly how much impact this has, but just letting the user disable the legacy endpoint controller seems a nice thing to have in terms of security.
Rob Scott
Sep 10, 2020, 1:31:43 PM9/10/20
to Antonio Ojea, Tim Hockin, kubernetes-sig-network
The concern there was that the EndpointSliceMirroring controller would create EndpointSlices for any user that had access to create Endpoints and not EndpointSlices. So I don't think disabling the Endpoints controller would help with that. My response was that access to Endpoints should be seen as equivalent to access to EndpointSlices. It's similar to the problem that anyone with access to Deployments can create Pods even if they don't have access to Pods directly. We do need to document it better, but I do not see this as a serious issue.
Bowei Du
Sep 10, 2020, 2:27:27 PM9/10/20
to Rob Scott, Antonio Ojea, Tim Hockin, kubernetes-sig-network
Is there a place to document RBAC permissions for EPS and EP maybe?
That you should always have RBAC be the same for both resources.
Bowei
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-network/CAGY4dk%3D8W0u-aUm36P%3D8rAq0XvdxeH2KWi_p%3D%3DHgqSchc-ScDQ%40mail.gmail.com.
Sandor Szuecs
Sep 10, 2020, 3:31:54 PM9/10/20
to Tim Hockin, Rob Scott, Antonio Ojea, kubernetes-sig-network, John Belamaric
Hi!
On Thu, 10 Sep 2020 at 18:50, 'Tim Hockin' via kubernetes-sig-network <kubernetes-...@googlegroups.com> wrote:
You can only deprecate a v1 API via a v2 group, of which we have no
plans, for now. You should assume Endpoints will continue to exist
and the controller will be available for the indefinite future. We
will be obligated to maintain it, though we may choose not to enhance
it.
ACK
We can discuss whether we should remove Endpoints support from
kube-proxy, which seems reasonable to me.
I would not change kube-proxy, because tools already rely on it.
For example https://github.com/zalando/postgres-operator relies on it. It manages endpoints of services itself to make sure we switch the current writable DB and the read-followers when the operator decides it. Of course we can change it to something else if this works, but there will be other users that might notice when it breaks....
From a developer point of view I understand less code is better, but in this case we have to be really careful.
Best, sandor
--
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-network/CAO_RewZHYq5fjQjYD1jwZLSFgHGeqKWxi3J3sq%2BOR4x3d16_vw%40mail.gmail.com.
--
Sandor Szücs | 418 I'm a teapot
Tim Hockin
Sep 10, 2020, 3:36:26 PM9/10/20
to Sandor Szuecs, Rob Scott, Antonio Ojea, kubernetes-sig-network, John Belamaric
On Thu, Sep 10, 2020 at 12:31 PM Sandor Szuecs <sandor...@zalando.de> wrote:
>
> Hi!
>
> On Thu, 10 Sep 2020 at 18:50, 'Tim Hockin' via kubernetes-sig-network <kubernetes-...@googlegroups.com> wrote:
>>
>> You can only deprecate a v1 API via a v2 group, of which we have no
>> plans, for now. You should assume Endpoints will continue to exist
>> and the controller will be available for the indefinite future. We
>> will be obligated to maintain it, though we may choose not to enhance
>> it.
>
>
> ACK
>
>>
>> We can discuss whether we should remove Endpoints support from
>> kube-proxy, which seems reasonable to me.
>
>
> I would not change kube-proxy, because tools already rely on it.
> For example https://github.com/zalando/postgres-operator relies on it. It manages endpoints of services itself to make sure we switch the current writable DB and the read-followers when the operator decides it. Of course we can change it to something else if this works, but there will be other users that might notice when it breaks....
> From a developer point of view I understand less code is better, but in this case we have to be really careful.
Do you depend on kube-proxy's use of Endpoints vs. EndpointSlices?
>
> Hi!
>
> On Thu, 10 Sep 2020 at 18:50, 'Tim Hockin' via kubernetes-sig-network <kubernetes-...@googlegroups.com> wrote:
>>
>> You can only deprecate a v1 API via a v2 group, of which we have no
>> plans, for now. You should assume Endpoints will continue to exist
>> and the controller will be available for the indefinite future. We
>> will be obligated to maintain it, though we may choose not to enhance
>> it.
>
>
> ACK
>
>>
>> We can discuss whether we should remove Endpoints support from
>> kube-proxy, which seems reasonable to me.
>
>
> I would not change kube-proxy, because tools already rely on it.
> For example https://github.com/zalando/postgres-operator relies on it. It manages endpoints of services itself to make sure we switch the current writable DB and the read-followers when the operator decides it. Of course we can change it to something else if this works, but there will be other users that might notice when it breaks....
> From a developer point of view I understand less code is better, but in this case we have to be really careful.
How so? The EPSlice mirroring controller should handle apps that
manually manipulate Endpoints by converting to slices...
Sandor Szuecs
Sep 10, 2020, 5:00:58 PM9/10/20
to Tim Hockin, Rob Scott, Antonio Ojea, kubernetes-sig-network, John Belamaric
On Thu, 10 Sep 2020 at 21:36, Tim Hockin <tho...@google.com> wrote:
On Thu, Sep 10, 2020 at 12:31 PM Sandor Szuecs <sandor...@zalando.de> wrote:
>
> Hi!
>
> On Thu, 10 Sep 2020 at 18:50, 'Tim Hockin' via kubernetes-sig-network <kubernetes-...@googlegroups.com> wrote:
>>
>> You can only deprecate a v1 API via a v2 group, of which we have no
>> plans, for now. You should assume Endpoints will continue to exist
>> and the controller will be available for the indefinite future. We
>> will be obligated to maintain it, though we may choose not to enhance
>> it.
>
>
> ACK
>
>>
>> We can discuss whether we should remove Endpoints support from
>> kube-proxy, which seems reasonable to me.
>
>
> I would not change kube-proxy, because tools already rely on it.
> For example https://github.com/zalando/postgres-operator relies on it. It manages endpoints of services itself to make sure we switch the current writable DB and the read-followers when the operator decides it. Of course we can change it to something else if this works, but there will be other users that might notice when it breaks....
> From a developer point of view I understand less code is better, but in this case we have to be really careful.
Do you depend on kube-proxy's use of Endpoints vs. EndpointSlices?
I don't know the difference yet.
I think we rely on that a service IP:port is accessed and the postgres-operator controlled endpoint is the one kube-proxy adds (or better sets, because it deletes the other) to the service. Of course an EndpointSlice with size 1 could work, too.
How so? The EPSlice mirroring controller should handle apps that
manually manipulate Endpoints by converting to slices...
I would have to check it. Does it just reflect the same state to endpoints as part of a service?
I will have more time again in October (parental leave right now).
Best, sandor
--
Tim Hockin
Sep 10, 2020, 7:30:11 PM9/10/20
to Sandor Szuecs, Rob Scott, Antonio Ojea, kubernetes-sig-network, John Belamaric
On Thu, Sep 10, 2020 at 2:00 PM Sandor Szuecs <sandor...@zalando.de> wrote:
>
>
>
> On Thu, 10 Sep 2020 at 21:36, Tim Hockin <tho...@google.com> wrote:
>>
>> On Thu, Sep 10, 2020 at 12:31 PM Sandor Szuecs <sandor...@zalando.de> wrote:
>> >
>> > Hi!
>> >
>> > On Thu, 10 Sep 2020 at 18:50, 'Tim Hockin' via kubernetes-sig-network <kubernetes-...@googlegroups.com> wrote:
>> >>
>> >> You can only deprecate a v1 API via a v2 group, of which we have no
>> >> plans, for now. You should assume Endpoints will continue to exist
>> >> and the controller will be available for the indefinite future. We
>> >> will be obligated to maintain it, though we may choose not to enhance
>> >> it.
>> >
>> >
>> > ACK
>> >
>> >>
>> >> We can discuss whether we should remove Endpoints support from
>> >> kube-proxy, which seems reasonable to me.
>> >
>> >
>> > I would not change kube-proxy, because tools already rely on it.
>> > For example https://github.com/zalando/postgres-operator relies on it. It manages endpoints of services itself to make sure we switch the current writable DB and the read-followers when the operator decides it. Of course we can change it to something else if this works, but there will be other users that might notice when it breaks....
>> > From a developer point of view I understand less code is better, but in this case we have to be really careful.
>>
>> Do you depend on kube-proxy's use of Endpoints vs. EndpointSlices?
>
>
> I don't know the difference yet.
> I think we rely on that a service IP:port is accessed and the postgres-operator controlled endpoint is the one kube-proxy adds (or better sets, because it deletes the other) to the service. Of course an EndpointSlice with size 1 could work, too.
This should be unchanged. The difference between kube-proxy using
>
>
>
> On Thu, 10 Sep 2020 at 21:36, Tim Hockin <tho...@google.com> wrote:
>>
>> On Thu, Sep 10, 2020 at 12:31 PM Sandor Szuecs <sandor...@zalando.de> wrote:
>> >
>> > Hi!
>> >
>> > On Thu, 10 Sep 2020 at 18:50, 'Tim Hockin' via kubernetes-sig-network <kubernetes-...@googlegroups.com> wrote:
>> >>
>> >> You can only deprecate a v1 API via a v2 group, of which we have no
>> >> plans, for now. You should assume Endpoints will continue to exist
>> >> and the controller will be available for the indefinite future. We
>> >> will be obligated to maintain it, though we may choose not to enhance
>> >> it.
>> >
>> >
>> > ACK
>> >
>> >>
>> >> We can discuss whether we should remove Endpoints support from
>> >> kube-proxy, which seems reasonable to me.
>> >
>> >
>> > I would not change kube-proxy, because tools already rely on it.
>> > For example https://github.com/zalando/postgres-operator relies on it. It manages endpoints of services itself to make sure we switch the current writable DB and the read-followers when the operator decides it. Of course we can change it to something else if this works, but there will be other users that might notice when it breaks....
>> > From a developer point of view I understand less code is better, but in this case we have to be really careful.
>>
>> Do you depend on kube-proxy's use of Endpoints vs. EndpointSlices?
>
>
> I don't know the difference yet.
> I think we rely on that a service IP:port is accessed and the postgres-operator controlled endpoint is the one kube-proxy adds (or better sets, because it deletes the other) to the service. Of course an EndpointSlice with size 1 could work, too.
Endpoints and EndpointSlice should be invisible. The contents should
be synced, and the results should be the same.
Now, in the future it may get more fun, as we teach slices about
topology, but we have a strong commitment to making sure it still
works.
>> How so? The EPSlice mirroring controller should handle apps that
>> manually manipulate Endpoints by converting to slices...
>
>
> I would have to check it. Does it just reflect the same state to endpoints as part of a service?
> I will have more time again in October (parental leave right now).
automatically mirror that into an EndpointSlice.
Congrats on parental leave! :)
Sandor Szuecs
Sep 12, 2020, 3:36:13 PM9/12/20
to Tim Hockin, Antonio Ojea, John Belamaric, Rob Scott, kubernetes-sig-network
On Fri 11. Sep 2020 at 01:30 Tim Hockin <tho...@google.com> wrote:
On Thu, Sep 10, 2020 at 2:00 PM Sandor Szuecs <sandor...@zalando.de> wrote:
>
>
>
> On Thu, 10 Sep 2020 at 21:36, Tim Hockin <tho...@google.com> wrote:
>>
>> On Thu, Sep 10, 2020 at 12:31 PM Sandor Szuecs <sandor...@zalando.de> wrote:
>> >
>> > Hi!
>> >
>> > On Thu, 10 Sep 2020 at 18:50, 'Tim Hockin' via kubernetes-sig-network <kubernetes-...@googlegroups.com> wrote:
>> >> We can discuss whether we should remove Endpoints support from
>> >> kube-proxy, which seems reasonable to me.
>> >
>> >
>> > I would not change kube-proxy, because tools already rely on it.
>> > For example https://github.com/zalando/postgres-operator relies on it. It manages endpoints of services itself to make sure we switch the current writable DB and the read-followers when the operator decides it. Of course we can change it to something else if this works, but there will be other users that might notice when it breaks....
>> > From a developer point of view I understand less code is better, but in this case we have to be really careful.
>>
>> Do you depend on kube-proxy's use of Endpoints vs. EndpointSlices?
>
>
> I don't know the difference yet.
> I think we rely on that a service IP:port is accessed and the postgres-operator controlled endpoint is the one kube-proxy adds (or better sets, because it deletes the other) to the service. Of course an EndpointSlice with size 1 could work, too.
This should be unchanged. The difference between kube-proxy using
Endpoints and EndpointSlice should be invisible. The contents should
be synced, and the results should be the same.
Now, in the future it may get more fun, as we teach slices about
topology, but we have a strong commitment to making sure it still
works.
Then it’s fine to move forward.
>> How so? The EPSlice mirroring controller should handle apps that
>> manually manipulate Endpoints by converting to slices...
>
>
> I would have to check it. Does it just reflect the same state to endpoints as part of a service?
> I will have more time again in October (parental leave right now).
Yes. If your controller writes Endpoints directly, we will
automatically mirror that into an EndpointSlice.
This sounds great, so I have no doubt about it.
Congrats on parental leave! :)
Thanks :)
Best, sandor
—
Reply all
Reply to author
Forward
0 new messages