No new cloud provider dependencies for Kube API Server

190 views
Skip to first unread message

Davanum Srinivas

unread,
Oct 18, 2017, 1:41:47 PM10/18/17
to K8s API Machinery SIG
Hi Team,

@lavalamp asked to bring this topic to the next meeting, so let me start this thread here before that so folks have a chance to dig into stuff:

Here are the latest comments from @lavalamp:

Here's the Issue:

Which is part of the larger effort to move things out of k/k:

For this group, the question is, Can we get rid of having to specify cloud-provider and cloud-config when running kube-apiserver?

Thanks,
Dims


Daniel Smith

unread,
Oct 18, 2017, 1:49:04 PM10/18/17
to Davanum Srinivas, K8s API Machinery SIG
Just to repeat myself, I think it is a great goal but not an immediately urgent one.

I don't expect us to work on this in Q4.

--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig-api-machinery@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/eac78014-5a03-497c-a57d-1ed18f020471%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Davanum Srinivas

unread,
Oct 18, 2017, 2:01:40 PM10/18/17
to Daniel Smith, K8s API Machinery SIG
Daniel,

I was told that there may not be a consensus on this goal, hence this thread.

Thanks,
Dims
>> email to kubernetes-sig-api-m...@googlegroups.com.
>> To post to this group, send email to
>> kubernetes-sig...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/eac78014-5a03-497c-a57d-1ed18f020471%40googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "K8s API Machinery SIG" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-sig-api-m...@googlegroups.com.
> To post to this group, send email to
> kubernetes-sig...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/CAB_J3bZaynfa7AjVVT52pc5pSidHJsG4xmKLuEA0UJODHdVO9A%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/d/optout.



--
Davanum Srinivas :: https://twitter.com/dims

de...@redhat.com

unread,
Oct 18, 2017, 2:15:05 PM10/18/17
to K8s API Machinery SIG
Part of this effort would remove the SSH tunnel from the kube-apiserver since that depends on the cloud provider and I don't think we would plan to replace it with anything.  If we agree to do that, I think we should announce the removal of SSH tunnel now, indicate that VPN (or similar) network config is the future and set a removal release date for the SSH tunnel.

Daniel Smith

unread,
Oct 18, 2017, 3:36:30 PM10/18/17
to David Eads, K8s API Machinery SIG
I think we still can't deprecate (start the clock) until a non-alpha alternative exists.

On Wed, Oct 18, 2017 at 11:15 AM, <de...@redhat.com> wrote:
Part of this effort would remove the SSH tunnel from the kube-apiserver since that depends on the cloud provider and I don't think we would plan to replace it with anything.  If we agree to do that, I think we should announce the removal of SSH tunnel now, indicate that VPN (or similar) network config is the future and set a removal release date for the SSH tunnel.

--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig-api-machinery@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/4f69b7f3-d471-4bb7-b78e-4e3a1d97a5f6%40googlegroups.com.

David Eads

unread,
Oct 18, 2017, 4:08:57 PM10/18/17
to Daniel Smith, Jordan Liggitt, K8s API Machinery SIG
For the SSH tunnel, I think most people agree that no in-process alternative should be developed.  Network configuration is outside the scope of kube-apiserver.

On Wed, Oct 18, 2017 at 3:36 PM, Daniel Smith <dbs...@google.com> wrote:
I think we still can't deprecate (start the clock) until a non-alpha alternative exists.
On Wed, Oct 18, 2017 at 11:15 AM, <de...@redhat.com> wrote:
Part of this effort would remove the SSH tunnel from the kube-apiserver since that depends on the cloud provider and I don't think we would plan to replace it with anything.  If we agree to do that, I think we should announce the removal of SSH tunnel now, indicate that VPN (or similar) network config is the future and set a removal release date for the SSH tunnel.

--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsubs...@googlegroups.com.

To post to this group, send email to kubernetes-sig-api-machinery@googlegroups.com.

Daniel Smith

unread,
Oct 18, 2017, 4:45:21 PM10/18/17
to David Eads, Jordan Liggitt, K8s API Machinery SIG
I tentatively agree that the replacement needn't be in-process. But it does need to exist.

Davanum Srinivas

unread,
Oct 18, 2017, 7:25:37 PM10/18/17
to Daniel Smith, David Eads, Jordan Liggitt, K8s API Machinery SIG
Daniel,

from [1] you mentioned, "The latter is not related to the reason why
we have the ssh tunnels". i was also looking at [2] and trying to
understand the code. Looks like there are SSH based health checks and
some communication seems to be tunneled over the ports established
over SSH. Can you please explain/expand on it a bit more when you get
a chance?

Thanks,
Dims

[1] https://github.com/kubernetes/kubernetes/pull/53912#issuecomment-337658778
[2] https://kubernetes.io/docs/concepts/architecture/master-node-communication/#ssh-tunnels

On Wed, Oct 18, 2017 at 4:45 PM, 'Daniel Smith' via K8s API Machinery
SIG <kubernetes-sig...@googlegroups.com> wrote:
> I tentatively agree that the replacement needn't be in-process. But it does
> need to exist.
>
> On Wed, Oct 18, 2017 at 1:08 PM, David Eads <de...@redhat.com> wrote:
>>
>> For the SSH tunnel, I think most people agree that no in-process
>> alternative should be developed. Network configuration is outside the scope
>> of kube-apiserver.
>>
>> On Wed, Oct 18, 2017 at 3:36 PM, Daniel Smith <dbs...@google.com> wrote:
>>>
>>> I think we still can't deprecate (start the clock) until a non-alpha
>>> alternative exists.
>>>
>>> On Wed, Oct 18, 2017 at 11:15 AM, <de...@redhat.com> wrote:
>>>>
>>>> Part of this effort would remove the SSH tunnel from the kube-apiserver
>>>> since that depends on the cloud provider and I don't think we would plan to
>>>> replace it with anything. If we agree to do that, I think we should
>>>> announce the removal of SSH tunnel now, indicate that VPN (or similar)
>>>> network config is the future and set a removal release date for the SSH
>>>> tunnel.
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "K8s API Machinery SIG" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to kubernetes-sig-api-m...@googlegroups.com.
>>>> To post to this group, send email to
>>>> kubernetes-sig...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/4f69b7f3-d471-4bb7-b78e-4e3a1d97a5f6%40googlegroups.com.
>>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "K8s API Machinery SIG" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-sig-api-m...@googlegroups.com.
> To post to this group, send email to
> kubernetes-sig...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/CAB_J3bafQC0A%2B%2BZUra8isfK%2Bj38Q3dA8vwQGzjBa5ed9Ym13Ag%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/d/optout.



Daniel Smith

unread,
Oct 18, 2017, 8:17:41 PM10/18/17
to Davanum Srinivas, David Eads, Jordan Liggitt, K8s API Machinery SIG
The documentation you linked to looks correct. What is unclear?


>>>> To post to this group, send email to

>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/4f69b7f3-d471-4bb7-b78e-4e3a1d97a5f6%40googlegroups.com.
>>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "K8s API Machinery SIG" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> To post to this group, send email to

> To view this discussion on the web visit

David Eads

unread,
Oct 19, 2017, 8:32:10 AM10/19/17
to Daniel Smith, Davanum Srinivas, Jordan Liggitt, K8s API Machinery SIG
Off the shelf VPN solutions already exist for joining multiple networks and making them routable.  Similarly DNS solutions exist for looking up names.

I think that this is a good case for deprecation.  We announce deprecation now and give consumers ample time to learn and deploy the VPN (or other networking solution) of their choice.  The solutions are fairly standard and generally understood, so the normal one year deprecation policy for GA features is reasonable for the SSH tunnel removal.

Daniel Smith

unread,
Oct 19, 2017, 1:45:52 PM10/19/17
to David Eads, Davanum Srinivas, Jordan Liggitt, K8s API Machinery SIG
That sounds like a super user-hostile replacement. Would you be happy if something you depended on deprecated like that?

We either should do better than that, or try to identify all actual users and make sure that they actually plan to stop using it in the timeframe.

lu...@luxaslabs.com

unread,
Oct 20, 2017, 11:52:46 AM10/20/17
to K8s API Machinery SIG
We have identified the actual user: GKE

One year deprecation period, starting now, for GKE to start its own VPN tunnels sounds reasonable to me.


Den torsdag 19 oktober 2017 kl. 18:45:52 UTC+1 skrev lavalamp:
That sounds like a super user-hostile replacement. Would you be happy if something you depended on deprecated like that?

We either should do better than that, or try to identify all actual users and make sure that they actually plan to stop using it in the timeframe.
On Thu, Oct 19, 2017 at 5:32 AM, David Eads <de...@redhat.com> wrote:
Off the shelf VPN solutions already exist for joining multiple networks and making them routable.  Similarly DNS solutions exist for looking up names.

I think that this is a good case for deprecation.  We announce deprecation now and give consumers ample time to learn and deploy the VPN (or other networking solution) of their choice.  The solutions are fairly standard and generally understood, so the normal one year deprecation policy for GA features is reasonable for the SSH tunnel removal.

On Wed, Oct 18, 2017 at 4:45 PM, Daniel Smith <dbs...@google.com> wrote:
I tentatively agree that the replacement needn't be in-process. But it does need to exist.
On Wed, Oct 18, 2017 at 1:08 PM, David Eads <de...@redhat.com> wrote:
For the SSH tunnel, I think most people agree that no in-process alternative should be developed.  Network configuration is outside the scope of kube-apiserver.
On Wed, Oct 18, 2017 at 3:36 PM, Daniel Smith <dbs...@google.com> wrote:
I think we still can't deprecate (start the clock) until a non-alpha alternative exists.
On Wed, Oct 18, 2017 at 11:15 AM, <de...@redhat.com> wrote:
Part of this effort would remove the SSH tunnel from the kube-apiserver since that depends on the cloud provider and I don't think we would plan to replace it with anything.  If we agree to do that, I think we should announce the removal of SSH tunnel now, indicate that VPN (or similar) network config is the future and set a removal release date for the SSH tunnel.

--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig...@googlegroups.com.

Daniel Smith

unread,
Oct 20, 2017, 1:01:26 PM10/20/17
to Lucas Käldström, K8s API Machinery SIG
We do not know that gke is the only user.

On Fri, Oct 20, 2017 at 8:52 AM, <lu...@luxaslabs.com> wrote:
We have identified the actual user: GKE

One year deprecation period, starting now, for GKE to start its own VPN tunnels sounds reasonable to me.

Den torsdag 19 oktober 2017 kl. 18:45:52 UTC+1 skrev lavalamp:
That sounds like a super user-hostile replacement. Would you be happy if something you depended on deprecated like that?

We either should do better than that, or try to identify all actual users and make sure that they actually plan to stop using it in the timeframe.
On Thu, Oct 19, 2017 at 5:32 AM, David Eads <de...@redhat.com> wrote:
Off the shelf VPN solutions already exist for joining multiple networks and making them routable.  Similarly DNS solutions exist for looking up names.

I think that this is a good case for deprecation.  We announce deprecation now and give consumers ample time to learn and deploy the VPN (or other networking solution) of their choice.  The solutions are fairly standard and generally understood, so the normal one year deprecation policy for GA features is reasonable for the SSH tunnel removal.

On Wed, Oct 18, 2017 at 4:45 PM, Daniel Smith <dbs...@google.com> wrote:
I tentatively agree that the replacement needn't be in-process. But it does need to exist.
On Wed, Oct 18, 2017 at 1:08 PM, David Eads <de...@redhat.com> wrote:
For the SSH tunnel, I think most people agree that no in-process alternative should be developed.  Network configuration is outside the scope of kube-apiserver.
On Wed, Oct 18, 2017 at 3:36 PM, Daniel Smith <dbs...@google.com> wrote:
I think we still can't deprecate (start the clock) until a non-alpha alternative exists.
On Wed, Oct 18, 2017 at 11:15 AM, <de...@redhat.com> wrote:
Part of this effort would remove the SSH tunnel from the kube-apiserver since that depends on the cloud provider and I don't think we would plan to replace it with anything.  If we agree to do that, I think we should announce the removal of SSH tunnel now, indicate that VPN (or similar) network config is the future and set a removal release date for the SSH tunnel.

--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsubs...@googlegroups.com.
To post to this group, send email to kubernetes-sig...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig-api-machinery@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/cdefd51c-fd35-4c7d-802a-29b2194c6b35%40googlegroups.com.

Davanum Srinivas

unread,
Oct 20, 2017, 1:36:44 PM10/20/17
to Daniel Smith, Lucas Käldström, K8s API Machinery SIG
So the procedure for finding that out is the deprecation process
right? If we find out there are going to be serious issues then we
drag out the deprecation period longer.

We could even throw in a command line flag to get folks to make a
choice for the moment when they try a new release out and then tell us
not to go through with the deprecation. Right?

Thanks,
Dims
>>>>>>>> send an email to kubernetes-sig-api-m...@googlegroups.com.
>>>>>>>> To post to this group, send email to
>>>>>>>> kubernetes-sig...@googlegroups.com.
>>>>>>>> To view this discussion on the web visit
>>>>>>>> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/4f69b7f3-d471-4bb7-b78e-4e3a1d97a5f6%40googlegroups.com.
>>>>>>>>
>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "K8s API Machinery SIG" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to kubernetes-sig-api-m...@googlegroups.com.
>> To post to this group, send email to
>> kubernetes-sig...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/cdefd51c-fd35-4c7d-802a-29b2194c6b35%40googlegroups.com.
>>
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "K8s API Machinery SIG" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-sig-api-m...@googlegroups.com.
> To post to this group, send email to
> kubernetes-sig...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/CAB_J3bb09VGSaKkyiP_Uj3qAXQz-Tk73cJodugfuvwqUhd0HTw%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/d/optout.



--
Davanum Srinivas :: https://twitter.com/dims

Jordan Liggitt

unread,
Oct 23, 2017, 12:42:41 AM10/23/17
to Daniel Smith, Lucas Käldström, K8s API Machinery SIG
As described in https://github.com/kubernetes/kubernetes/issues/54076, only the GCE cloudprovider has ever supported distributing keys to nodes. That means the tunneling feature has only ever functioned standalone within GCE.

Further, 1.2 was the only release where this feature functioned correctly outside of GCE (though even that still required an external process to distribute keys to nodes):
  • In 1.0 and 1.1, SSH tunneling broke proxy subresources in all environments (including GCE/GKE)
  • Since 1.3, using SSH tunneling outside a GCE/GKE cloudprovider environment causes API servers to go unhealthy after 10 minutes (due to key distribution health checks that fail everywhere but GCE)
I am strongly in favor of deprecating and removing this feature from the apiserver:
  • only GCE/GKE benefit from it
  • all other deployments are responsible for forming their network outside the apiserver process
  • it makes the apiserver responsible for maintaining network topology, which seems well outside the apiserver's scope
  • it has consistently increased complexity for new features within the apiserver (subresource proxying, aggregation, webhook admission, etc)




On Fri, Oct 20, 2017 at 1:01 PM, 'Daniel Smith' via K8s API Machinery SIG <kubernetes-sig...@googlegroups.com> wrote:
To post to this group, send email to kubernetes-sig-api-machinery@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig-api-machinery@googlegroups.com.

hen...@loodse.com

unread,
Oct 23, 2017, 2:39:02 AM10/23/17
to K8s API Machinery SIG
As described in https://github.com/kubernetes/kubernetes/issues/54076, only the GCE cloudprovider has ever supported distributing keys to nodes.
Yes.
 
That means the tunneling feature has only ever functioned standalone within GCE.
No. The distribution of keys is a neat feature for GKE users, but it does not restrict the usage of the ssh-tunneling to GKE.
You can simply deploy the publickey of the specified "--ssh-keyfile" onto the nodes and it works.
 
We at Loodse use this feature. 
We run kubernetes-on-kubernetes. So we host the master-components in a namespace and let the worker nodes exist at some cloud-provider.
The ssh-tunnel enables us to let the apiserver get into the pod-networking. So the user can use whatever pod-networking he wants.
 
Further, 1.2 was the only release where this feature functioned correctly outside of GCE (though even that still required an external process to distribute keys to nodes):
  • In 1.0 and 1.1, SSH tunneling broke proxy subresources in all environments (including GCE/GKE)
  • Since 1.3, using SSH tunneling outside a GCE/GKE cloudprovider environment causes API servers to go unhealthy after 10 minutes (due to key distribution health checks that fail everywhere but GCE)
Regarding the /health endpoint, yes. I opened a pr: https://github.com/kubernetes/kubernetes/pull/52645




To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig...@googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig...@googlegroups.com.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "K8s API Machinery SIG" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-api-machinery+unsub...@googlegroups.com.
To post to this group, send email to kubernetes-sig...@googlegroups.com.

Davanum Srinivas

unread,
Oct 23, 2017, 6:22:32 AM10/23/17
to hen...@loodse.com, K8s API Machinery SIG
Henrik,

So ... you would be ok if this functionality moves to another
stand-alone GCE specific process and not part of kube-apiserver.
Right? (As the exercise is here is Trying to get rid of
--cloudprovider from kube api server)

Thanks,
Dims

On Mon, Oct 23, 2017 at 2:39 AM, <hen...@loodse.com> wrote:
>
>> As described in https://github.com/kubernetes/kubernetes/issues/54076,
>> only the GCE cloudprovider has ever supported distributing keys to nodes.
>
>>>>>>>>>> send an email to kubernetes-sig-api-m...@googlegroups.com.
>>>>>>>>>> To post to this group, send email to
>>>>>>>>>> kubernetes-sig...@googlegroups.com.
>>>>>>>>>> To view this discussion on the web visit
>>>>>>>>>> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/4f69b7f3-d471-4bb7-b78e-4e3a1d97a5f6%40googlegroups.com.
>>>>>>>>>>
>>>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "K8s API Machinery SIG" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to kubernetes-sig-api-m...@googlegroups.com.
>>>> To post to this group, send email to kubernetes-sig...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/cdefd51c-fd35-4c7d-802a-29b2194c6b35%40googlegroups.com.
>>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "K8s API Machinery SIG" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to kubernetes-sig-api-m...@googlegroups.com.
>>>>>>>>>> send an email to kubernetes-sig-api-m...@googlegroups.com.
>>>>>>>>>> To post to this group, send email to
>>>>>>>>>> kubernetes-sig...@googlegroups.com.
>>>>>>>>>> To view this discussion on the web visit
>>>>>>>>>> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/4f69b7f3-d471-4bb7-b78e-4e3a1d97a5f6%40googlegroups.com.
>>>>>>>>>>
>>>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "K8s API Machinery SIG" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to kubernetes-sig-api-m...@googlegroups.com.
>>>> To post to this group, send email to kubernetes-sig...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/cdefd51c-fd35-4c7d-802a-29b2194c6b35%40googlegroups.com.
>>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "K8s API Machinery SIG" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to kubernetes-sig-api-m...@googlegroups.com.
>>>>>>>>>> send an email to kubernetes-sig-api-m...@googlegroups.com.
>>>>>>>>>> To post to this group, send email to
>>>>>>>>>> kubernetes-sig...@googlegroups.com.
>>>>>>>>>> To view this discussion on the web visit
>>>>>>>>>> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/4f69b7f3-d471-4bb7-b78e-4e3a1d97a5f6%40googlegroups.com.
>>>>>>>>>>
>>>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "K8s API Machinery SIG" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to kubernetes-sig-api-m...@googlegroups.com.
>>>> To post to this group, send email to kubernetes-sig...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/cdefd51c-fd35-4c7d-802a-29b2194c6b35%40googlegroups.com.
>>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "K8s API Machinery SIG" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to kubernetes-sig-api-m...@googlegroups.com.
>>> To post to this group, send email to kubernetes-sig...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/CAB_J3bb09VGSaKkyiP_Uj3qAXQz-Tk73cJodugfuvwqUhd0HTw%40mail.gmail.com.
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "K8s API Machinery SIG" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-sig-api-m...@googlegroups.com.
> To post to this group, send email to
> kubernetes-sig...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubernetes-sig-api-machinery/b5e1a600-6bdb-47e4-888f-31b750c909eb%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.



--
Davanum Srinivas :: https://twitter.com/dims

Henrik

unread,
Oct 23, 2017, 6:27:31 AM10/23/17
to Davanum Srinivas, K8s API Machinery SIG
I'm ok with removing AddSSHKeyToAllInstances but not with the removal of
the ssh-tunnel feature as this is not cloud-provider specific.

Davanum Srinivas

unread,
Oct 23, 2017, 6:48:41 AM10/23/17
to Henrik, K8s API Machinery SIG
Henrik,

Can you please expand a bit more on why it *has* to be in the same
process as the api-server?

Thanks,
Dims

Henrik

unread,
Oct 23, 2017, 11:33:00 AM10/23/17
to Davanum Srinivas, K8s API Machinery SIG
Hey Dims,

misunderstanding, the tunnel does not have to be in the same process.


Technical (i hope not too off-topic):

I created a ssh-tunnel via
"ssh -D 8123 -C -N user@node-ip"

Set a env var on the apiserver to HTTPS_PROXY=socks5://127.0.0.1:8123

Shouldn't that theoretically do the same thing?
Because
kubectl proxy
curl
http://localhost:8001/api/v1/namespaces/kube-system/services/kubernetes-dashboard/proxy/

Returns:
Error: 'proxyconnect tcp: dial tcp: lookup socks5 on 10.47.240.10:53: no
such host'
Trying to reach: 'http://172.25.0.3:9090/'

Thanks
Henrik

Daniel Smith

unread,
Oct 23, 2017, 1:27:49 PM10/23/17
to Jordan Liggitt, Lucas Käldström, K8s API Machinery SIG
On Sun, Oct 22, 2017 at 9:42 PM, Jordan Liggitt <jlig...@redhat.com> wrote:
As described in https://github.com/kubernetes/kubernetes/issues/54076, only the GCE cloudprovider has ever supported distributing keys to nodes. That means the tunneling feature has only ever functioned standalone within GCE.

Ah, thanks, that addresses my objection. Deprecating with a 1 year timeframe seems reasonable.

Further, 1.2 was the only release where this feature functioned correctly outside of GCE (though even that still required an external process to distribute keys to nodes):
  • In 1.0 and 1.1, SSH tunneling broke proxy subresources in all environments (including GCE/GKE)
  • Since 1.3, using SSH tunneling outside a GCE/GKE cloudprovider environment causes API servers to go unhealthy after 10 minutes (due to key distribution health checks that fail everywhere but GCE)
I am strongly in favor of deprecating and removing this feature from the apiserver:
  • only GCE/GKE benefit from it
  • all other deployments are responsible for forming their network outside the apiserver process
  • it makes the apiserver responsible for maintaining network topology, which seems well outside the apiserver's scope
  • it has consistently increased complexity for new features within the apiserver (subresource proxying, aggregation, webhook admission, etc)
apiserver still needs to do service resolution to avoid layering violations-- this only removes the proxy dialer component.

 




Reply all
Reply to author
Forward
0 new messages