Is there any reason prometheus should have nodes/proxy permissions in ClusterRole for kubernetes?

[Jesse Simpson]

Hey all,

I'm investigating a security vulnerability reported by my company's security scanning software.  We were scanning a helm chart that we make use out of that has a prometheus server pod in it.

The threat is that a pod with node/proxy permission is vulnerable to privilege escalation.

https://blog.aquasec.com/privilege-escalation-kubernetes-rbac

As part of my investigation, I tried removing this nodes/proxy permission, and checked a number of prometheus metrics to see if they report different data, or no data when there previously was data.  But so far, I can't see any negative side effect to removing the nodes/proxy permission.

I've contacted the developers of the helm chart we scanned, and they cannot justify their need for this permission and insist that we do not remove it.

Is there a reason you all can think of that this permission might be required for prometheus to function?

Thanks,

Jesse

[Ben Kochie]

It would help if you linked the specific helm chart and issue you filed. There are a lot of different charts out there maintained by different people.

But just a guess, you're talking about the prometheus-community/prometheus chart[0].

IIRC in some configurations the Prometheus server needs access to the proxy in order to scrape data from the kubelet. I think this may be a legacy mode of operation, but it used to be the default.

[0]: https://github.com/prometheus-community/helm-charts/blob/0b928f341240c76d8513534035a825686ed28a4b/charts/prometheus/templates/clusterrole.yaml#L23

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/29001ead-8ac6-414a-9d0c-76631c253acen%40googlegroups.com.

[Jesse Simpson]

Hey Ben,

Sorry for not initially specifying the helm chart, I was under the impression that the repo was private and found out recently that it's public.

repo url: https://kubecost.github.io/cost-analyzer/

repo name: kubecost

chart version: 1.99.0

The link to the cluster role definition is here: https://github.com/kubecost/cost-analyzer-helm-chart/blob/v1.100/cost-analyzer/charts/prometheus/templates/server-clusterrole.yaml

And I think that prometheus chart inside kubecost may have been copied from the prometheus-community helm chart you linked.  They seem similar enough.

The security tool that reported the vulnerability is Trivy, so other users of Trivy probably report the same vulnerability.

Your insight into the historical use of prometheus scraping data from kubelet is helpful. If this is no longer required, perhaps I can suggest removing this dependency in prometheus-community/helm-charts and request that the kubecost maintainers update their version of this helm chart.

Jesse

[Ben Kochie]

Please be aware, security scanners are highly prone to false positives. You need to verify there is an actual exploitable path here before worrying too much. Don't blindly believe security scanners.

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/7d5ba8ac-54d8-48d4-b85e-2606ef7af399n%40googlegroups.com.

[Ben Kochie]

Complete: https://github.com/prometheus-community/helm-charts/pull/3077

[Jesse Simpson]

Thank you for the PR!

Jesse