[code-organization] Extracting hyperkube

183 views
Skip to first unread message

Davanum Srinivas

unread,
Aug 9, 2019, 6:22:05 PM8/9/19
to kubernetes-si...@googlegroups.com
Team,

Do we know who is using hyperkube? We had an old issue where we were trying to transition everything to hyperkube[1], but i guess it feel out of favor and most folks just use the binaries. Right?

Can we get rid of hyperkube binary and image from release artifacts? 

Why am i asking this? when revisiting all the cloud provider dependencies that we drag in, it turns out we don't really need to ship cloud-controller-manager as we will have external providers that have their own controller manager. To remove cloud-controller-manager, we first have to remove it from hyperkube (deprecation in [2]). Then the question was, do we really need hyperkube?

Given that it's easy to just create a separate repo for hyperkube (prototype [3]). Do we have a set of folks who can sign up for this work? (Also there's k3s already! for the same use case).

Note that the only hiccup in this prototype was that we don't check in generated code for openapi which i have a fix we can debate on in [4].

WDYT? Can we give hyperkube images and binary the boot?

Thanks,
Dims



--
Davanum Srinivas :: https://twitter.com/dims

Stephen Augustus

unread,
Aug 9, 2019, 6:25:19 PM8/9/19
to Davanum Srinivas, kubernetes-sig-architecture, kubernetes-sig-release
(+ SIG Release for vis)

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/CANw6fcE%3DCyzCTRqA6LY6u-Ypoj%3DQLOho3_Kbc6n-7Nks7iwFPA%40mail.gmail.com.

Mike Spreitzer

unread,
Aug 9, 2019, 9:34:31 PM8/9/19
to Stephen Augustus, Davanum Srinivas, kubernetes-sig-architecture, kubernetes-sig-release
hack/local-up-cluster.sh uses hyperkube, right?

Regards,
Mike

kubernetes-si...@googlegroups.com wrote on 08/09/2019 06:25:05 PM:

> From: Stephen Augustus <Ste...@agst.us>

> d/msgid/kubernetes-sig-architecture/
> CAFQm5yRtb4djUPDMSTFbXn3Fju3EKHvWgkL1coakyTZfbq%2BY1w%40mail.gmail.com.

Davanum Srinivas

unread,
Aug 9, 2019, 9:50:37 PM8/9/19
to Mike Spreitzer, Stephen Augustus, kubernetes-sig-architecture, kubernetes-sig-release
yep. that could be easily modified to not use it.

Davanum Srinivas

unread,
Aug 11, 2019, 10:58:17 AM8/11/19
to Mike Spreitzer, Stephen Augustus, kubernetes-sig-architecture, kubernetes-sig-release

Lubomir I. Ivanov

unread,
Aug 11, 2019, 2:49:33 PM8/11/19
to Davanum Srinivas, kubernetes-sig-architecture, kubernetes-sig-cluster-lifecycle
AFAIK, SAP have claimed to use hyperkube in production.
should / can we mark the image as deprecated in 1.16 and possibly
remove in 1.17?

lubomir
--

Vallery Lancey

unread,
Aug 11, 2019, 3:37:21 PM8/11/19
to Lubomir I. Ivanov, Davanum Srinivas, kubernetes-sig-architecture, kubernetes-sig-cluster-lifecycle
Maybe mark as depreciated with tentative/gentle timeline, and try to gauge usage. I’m not sure what the cleanup benefit vs user cost is.

Davanum Srinivas

unread,
Aug 11, 2019, 5:16:39 PM8/11/19
to Vallery Lancey, Lubomir I. Ivanov, kubernetes-sig-architecture, kubernetes-sig-cluster-lifecycle
Vallery, Jason,

Please note that interested parties can maintain it in another repo. See prototype:


-- Dims

Davanum Srinivas

unread,
Aug 11, 2019, 5:55:38 PM8/11/19
to Andrew Rynhard, Lubomir I. Ivanov, kubernetes-sig-architecture, kubernetes-sig-cluster-lifecycle
Andrew,

Ack. If we move just hyperkube support to another repo, you can have custom Makefile/scripts for the musl use case as well :)

On Sun, Aug 11, 2019 at 2:53 PM Andrew Rynhard <and...@rynhard.io> wrote:
As Jason pointed out, Talos does indeed use hyperkube. That being said I have been looking for a good reason to get away from it. The main reason we do this is because we use musl instead of glibc and IIRC kubelet is linked against glibc.
> You received this message because you are subscribed to the Google Groups "kubernetes-sig-cluster-lifecycle" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-cluster...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-cluster-lifecycle/CAGDbWi_SqGqWk_GtusJbW6xn8sUbUaUXah%2BNg%2BEOXNC9AXdHtw%40mail.gmail.com.

Stephen Augustus

unread,
Aug 11, 2019, 5:57:52 PM8/11/19
to Davanum Srinivas, Andrew Rynhard, Lubomir I. Ivanov, kubernetes-sig-architecture, kubernetes-sig-cluster-lifecycle, kubernetes-sig-release
(looks like the threads forked off, so adding SIG Release back)

I'm in favor of moving hyperkube out of tree.

-- Stephen

David Eads

unread,
Aug 12, 2019, 7:59:13 AM8/12/19
to Stephen Augustus, Davanum Srinivas, Andrew Rynhard, Lubomir I. Ivanov, kubernetes-sig-architecture, kubernetes-sig-cluster-lifecycle, kubernetes-sig-release
I like the idea of moving away from hyperkube.  I'm less sure that an out of tree hyperkube is practical long term if the mono-repo is someday divided.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/CAFQm5ySX1pDOni2usf8Y--Po2o94VvK-5R9PaCGptAozc0FLwA%40mail.gmail.com.

Patrick Lang

unread,
Aug 12, 2019, 1:11:50 PM8/12/19
to Davanum Srinivas, Mike Spreitzer, Stephen Augustus, kubernetes-sig-architecture, kubernetes-sig-release, Lachie Evenson, Craig Peters

Is your goal to do this during 1.16 or 1.17?

 

The test passes running on Azure depend on this and build the hyperkube image from kubetest. Moving away from this would need a week or two so we can coordinate across the teams using it since they span US/China/EU. I think that’s too risky this close to feature freeze.

 

https://github.com/kubernetes/test-infra/blob/ee5d952705281807c0528babdffde85c0e9290d8/kubetest/azure.go#L628

Davanum Srinivas

unread,
Aug 12, 2019, 4:42:03 PM8/12/19
to Patrick Lang, Mike Spreitzer, Stephen Augustus, kubernetes-sig-architecture, kubernetes-sig-release, Lachie Evenson, Craig Peters
Patrick,

1.17 :) So no worries!

Dalton Hubble

unread,
Sep 10, 2019, 2:58:52 AM9/10/19
to kubernetes-sig-architecture
Hyperkube has been pretty valuable imo, especially on container optimized operating systems (downloading binaries is much less desired). On Container Linux, Flatcar Linux, and Fedora CoreOS and associated distributions there plenty of hyperkube users I'm aware of (who likely aren't aware of this discussion, someone just pointed me to it). Some tools favoring it:








On Monday, August 12, 2019 at 1:42:03 PM UTC-7, Davanum Srinivas wrote:
Patrick,

1.17 :) So no worries!


> To view this discussion on the web visit

https://groups.google.com/
> d/msgid/kubernetes-sig-architecture/CANw6fcE%3DCyzCTRqA6LY6u-
> Ypoj%3DQLOho3_Kbc6n-7Nks7iwFPA%40mail.gmail.com.
> --

> You received this message because you are subscribed to the Google
> Groups "kubernetes-sig-architecture" group.
> To unsubscribe from this group and stop receiving emails from it,


> To view this discussion on the web visit

https://groups.google.com/
> d/msgid/kubernetes-sig-architecture/
> CAFQm5yRtb4djUPDMSTFbXn3Fju3EKHvWgkL1coakyTZfbq%2BY1w%40mail.gmail.com.


 

--

Davanum Srinivas :: https://twitter.com/dims


 

--

Davanum Srinivas :: https://twitter.com/dims

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-architecture+unsub...@googlegroups.com.

Davanum Srinivas

unread,
Sep 10, 2019, 7:29:13 AM9/10/19
to Dalton Hubble, kubernetes-sig-architecture
Dalton,

Thanks for the feedback. 

- Looks like we cannot deprecate and drop it since a lot of folks are using it.
- We could move it to another repository and enable folks who are interested in this to help with its upkeep. I don't yet see any objections to that. There have been some folks who volunteered to help with it already, but definitely more the merrier. If any of those folks can help, please point them to this thread until i start a KEP when 1.16 winds down.

Thanks,
Dims


> To view this discussion on the web visit

https://groups.google.com/
> d/msgid/kubernetes-sig-architecture/CANw6fcE%3DCyzCTRqA6LY6u-
> Ypoj%3DQLOho3_Kbc6n-7Nks7iwFPA%40mail.gmail.com.
> --

> You received this message because you are subscribed to the Google
> Groups "kubernetes-sig-architecture" group.
> To unsubscribe from this group and stop receiving emails from it,


> To view this discussion on the web visit

https://groups.google.com/
> d/msgid/kubernetes-sig-architecture/
> CAFQm5yRtb4djUPDMSTFbXn3Fju3EKHvWgkL1coakyTZfbq%2BY1w%40mail.gmail.com.


 

--

Davanum Srinivas :: https://twitter.com/dims


 

--

Davanum Srinivas :: https://twitter.com/dims

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.



--
Davanum Srinivas :: https://twitter.com/dims

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-architecture" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-arch...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-architecture/22052ebf-1020-408d-a75d-84516009141a%40googlegroups.com.

Stephen Augustus

unread,
Oct 31, 2019, 3:47:45 AM10/31/19
to Davanum Srinivas, kubernetes-sig-architecture, kubernetes-sig-release
SIG Arch Chairs --

Can you approve this migration request: https://github.com/kubernetes/org/issues/1365

-- Stephen

On Sun, Aug 11, 2019 at 5:57 PM Stephen Augustus <Ste...@agst.us> wrote:
Reply all
Reply to author
Forward
0 new messages