RBAC rules for heapster via proxy

[Chris Jones]

Hi All,

What access do I need to grant my users so that they can run "kubectl top pod"? When a user runs that command, I see this in the logs:

GET /api/v1/proxy/namespaces/kube-system/services/http:heapster:/apis/metrics/v1alpha1/namespaces/my-namespace/pods?labelSelector=: (1.193183ms) 403

I wasn't able to find any examples of configuring access for this sort of thing.

Chris

[Eric Chiang]

Chris,

The following RBAC role/binding should grant access (have not test
this). Importantly you need to grant access to the "proxy"
subresource.

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: service-proxier
rules:
- apiGroups: [""]
resources: ["services/proxy"]
verbs: ["get"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: proxy-services
namespace: kube-system
subjects:
- kind: User
name: ( USERNAME )
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: service-proxier
apiGroup: rbac.authorization.k8s.io

The default cluster role "edit" also encompasses this power.

It seems like we expand the subresources RBAC docs. "status" is pretty
intuitive, but proxy, logs, etc probably need to be mentioned and
ctrl-f'able.

Eric

> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes developer/contributor discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-de...@googlegroups.com.
> To post to this group, send email to kuberne...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubernetes-dev/607a022f-859f-4f8e-a38e-b0455ad9c3f0%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Clayton Coleman]

Please also note that /proxy is incredibly powerful - it allows you
to do many things to the underlying node, up to accessing all
container data and potentially host logs

> To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-dev/CAA8S17GP9B3ejYmMPYvRnWjy1Tq9qJAXDQiqthhcjuXRE7wKbg%40mail.gmail.com.

[Daniel Smith]

Came here to say what Clayton said--proxy to all services in kube-system is probably root on the cluster.

>> email to kubernetes-dev+unsubscribe@googlegroups.com.
>> To post to this group, send email to kubernetes-dev@googlegroups.com.

>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/kubernetes-dev/607a022f-859f-4f8e-a38e-b0455ad9c3f0%40googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "Kubernetes developer/contributor discussion" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-dev+unsubscribe@googlegroups.com.
> To post to this group, send email to kubernetes-dev@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-dev/CAA8S17GP9B3ejYmMPYvRnWjy1Tq9qJAXDQiqthhcjuXRE7wKbg%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Kubernetes developer/contributor discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-dev+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-dev@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-dev/-8256276560176115382%40unknownmsgid.

[Chris Jones]

Thanks for the reply, Eric.

So this allows the user to proxy any service in any namespace? It seems overly broad; could I use resourceNames to allow access to only certain services? If so, what would that look like? Ideally I'd like to allow the user to proxy only for heapster, and only for containers in a particular namespace.

Chris

[Jordan Liggitt]

In 1.6, `kubectl top` switched to use the proxy subresource so you can create a role that just allows services/proxy on just the heapster service in the kube-system namespace

Previous versions used the top level proxy verb, which is difficult to grant subdivided access to. 

To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-de...@googlegroups.com.
To post to this group, send email to kuberne...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-dev/CAB_J3bYsVk5HSdJx_A%3DKx8iWdapc2NnsUnsv6n9QorhvRKi2Nw%40mail.gmail.com.

[Eric Chiang]

Yeah, you should be able something like

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: service-proxier
rules:
- apiGroups: [""]
resources: ["services/proxy"]

resourceNames: ["heapster"]
verbs: ["get"]

I've opened https://github.com/kubernetes/kubernetes.github.io/issues/3850
to follow up on the RBAC docs.

Eric

[Chris Jones]

Nice. That's still going to allow the user to gather heapster statistics for every pod in the cluster; but at least it won't allow access to every service in every namespace. Thanks for the detailed reply.

Chris

[Chris Jones]

...so building on what Eric suggested, I would create a Role, not a ClusterRole, in the kube-system namespace?

kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: service-proxier
  namespace: kube-system
rules:
- apiGroups: [""]
  resources: ["services/proxy"]
  resourceNames: ["heapster"]
  verbs: ["get"]

[Eric Chiang]

Bindings determine the context of a role, not the role itself.

By referring to a ClusterRole with a RoleBinding you're saying "grant
this ClusterRole in this namespace." So you can use a ClusterRole or a
Role, and get the same results. The only difference is a Role can't be
referenced from a different namespace, which it seems like you
wouldn't want to do with this one, so yes, a Role might be more
appropriate.

>> >> email to kubernetes-de...@googlegroups.com.
>> >> To post to this group, send email to kuberne...@googlegroups.com.
>> >> To view this discussion on the web visit
>> >>

>> >> https://groups.google.com/d/msgid/kubernetes-dev/607a022f-859f-4f8e-a38e-b0455ad9c3f0%40googlegroups.com.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups "Kubernetes developer/contributor discussion" group.
>> > To unsubscribe from this group and stop receiving emails from it, send

>> > an email to kubernetes-de...@googlegroups.com.
>> > To post to this group, send email to kuberne...@googlegroups.com.
>> > To view this discussion on the web visit

>> > https://groups.google.com/d/msgid/kubernetes-dev/CAA8S17GP9B3ejYmMPYvRnWjy1Tq9qJAXDQiqthhcjuXRE7wKbg%40mail.gmail.com.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Kubernetes developer/contributor discussion" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

[Piotr Szczesniak]

Also there is an effort to introduce metrics server features#271 which will make metrics available via kube-aggregator and resolve this issue.

Thanks,
Piotr

>> >> email to kubernetes-dev+unsubscribe@googlegroups.com.
>> >> To post to this group, send email to kubernetes-dev@googlegroups.com.

>> >> To view this discussion on the web visit
>> >>
>> >> https://groups.google.com/d/msgid/kubernetes-dev/607a022f-859f-4f8e-a38e-b0455ad9c3f0%40googlegroups.com.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups "Kubernetes developer/contributor discussion" group.
>> > To unsubscribe from this group and stop receiving emails from it, send

>> > an email to kubernetes-dev+unsubscribe@googlegroups.com.
>> > To post to this group, send email to kubernetes-dev@googlegroups.com.

>> > To view this discussion on the web visit
>> > https://groups.google.com/d/msgid/kubernetes-dev/CAA8S17GP9B3ejYmMPYvRnWjy1Tq9qJAXDQiqthhcjuXRE7wKbg%40mail.gmail.com.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Kubernetes developer/contributor discussion" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to kubernetes-dev+unsubscribe@googlegroups.com.
>> To post to this group, send email to kubernetes-dev@googlegroups.com.

>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/kubernetes-dev/-8256276560176115382%40unknownmsgid.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes developer/contributor discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to kubernetes-dev+unsubscribe@googlegroups.com.
> To post to this group, send email to kubernetes-dev@googlegroups.com.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kubernetes-dev/CAB_J3bYsVk5HSdJx_A%3DKx8iWdapc2NnsUnsv6n9QorhvRKi2Nw%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.
>
>

--
You received this message because you are subscribed to the Google Groups "Kubernetes developer/contributor discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-dev+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-dev@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-dev/CAA8S17E_-w8KVUoSXcD7oiLraWetdJDWk%2BVBpWKoZpFAfcjDzA%40mail.gmail.com.

[Mohammad Mahzoun]

Is there a solution to isolate metrics of each namespace for its users?

[Solly Ross]

> Is there a solution to isolate metrics of each namespace for its users?

No, not with Heapster. If you want that capability, you'll need to use metrics-server,
and the resource metrics API, which is a proper Kubernetes style API. Then you can create
RBAC policies as if the metrics were normal Kubernetes objects.

Best Regards,
Solly Ross