prometheus visit k8s x509 certificate signed by unknown authority
alex he
I can use curl to visit k8s apiserver api:
curl https://10.10.10.68:6443/api/v1/nodes --cacert kube-ca.pem --cert kube-node.pem --key kube-node-key.pem|head -n 20
"kind": "NodeList",
"apiVersion": "v1",
"metadata": {
"selfLink": "/api/v1/nodes",
"resourceVersion": "67299229"
},
"items": [
{
"metadata": {
"name": "k8smaster12",
"selfLink": "/api/v1/nodes/k8smaster12",
"uid": "060be972-6346-11ea-a193-00155d0a3a00",
"resourceVersion": "67299092",
"creationTimestamp": "2020-03-11T03:11:38Z",
"labels": {
"beta.kubernetes.io/arch": "amd64",
"beta.kubernetes.io/os": "linux",
"kubernetes.io/arch": "amd64",
"kubernetes.io/hostname": "k8smaster12",
but I can't use prometheus to visit k8s.
this is my prometheus.yml:
root@alextest-55c44cddc8-gqcdt:~/prometheus-2.23.0.linux-amd64# cat prometheus.yml
global:
scrape_interval: 15s
evaluation_interval: 15s
alerting:
alertmanagers:
- static_configs:
- targets:
rule_files:
scrape_configs:
- job_name: "alexk8s-apiserver"
kubernetes_sd_configs:- role: endpoints
api_server: 'https://10.10.10.68:6443'
scheme: https
tls_config:
insecure_skip_verify: true
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
cert_file: /root/ssl/kube-node.pem
key_file: /root/ssl/kube-node-key.pem
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
relabel_configs: - action: labelmap
regex: _meta_kubernetes_node_label(.+)
- role: endpoints
when I start prometheus, it reports:
root@alextest-gqcdt:~/prometheus-2.23.0.linux-amd64# ./prometheus
level=info ts=2020-12-22T08:39:27.185Z caller=main.go:322 msg="No time or size retention was set so using the default time retention" duration=15d level=info ts=2020-12-22T08:39:27.185Z caller=main.go:360 msg="Starting Prometheus" version="(version=2.23.0, branch=HEAD, revision=26d89b4b0776fe4cd5a3656dfa520f119a375273)" level=info ts=2020-12-22T08:39:27.185Z caller=main.go:365 build_context="(go=go1.15.5, user=root@37609b3a0a21, date=20201126-10:56:17)" level=info ts=2020-12-22T08:39:27.185Z caller=main.go:366 host_details="(Linux 4.15.0-123-generic #126-Ubuntu SMP Wed Oct 21 09:40:11 UTC 2020 x86_64 alextest-55c44cddc8-gqcdt (none))" level=info ts=2020-12-22T08:39:27.186Z caller=main.go:367 fd_limits="(soft=1048576, hard=1048576)" level=info ts=2020-12-22T08:39:27.186Z caller=main.go:368 vm_limits="(soft=unlimited, hard=unlimited)" level=info ts=2020-12-22T08:39:27.188Z caller=main.go:722 msg="Starting TSDB ..." level=info ts=2020-12-22T08:39:27.188Z caller=web.go:528 component=web msg="Start listening for connections" address=0.0.0.0:9090 level=info ts=2020-12-22T08:39:27.193Z caller=head.go:645 component=tsdb msg="Replaying on-disk memory mappable chunks if any" level=info ts=2020-12-22T08:39:27.193Z caller=head.go:659 component=tsdb msg="On-disk memory mappable chunks replay completed" duration=4.9µs level=info ts=2020-12-22T08:39:27.193Z caller=head.go:665 component=tsdb msg="Replaying WAL, this may take a while" level=info ts=2020-12-22T08:39:27.193Z caller=head.go:717 component=tsdb msg="WAL segment loaded" segment=0 maxSegment=7 level=info ts=2020-12-22T08:39:27.194Z caller=head.go:717 component=tsdb msg="WAL segment loaded" segment=1 maxSegment=7 level=info ts=2020-12-22T08:39:27.195Z caller=head.go:717 component=tsdb msg="WAL segment loaded" segment=2 maxSegment=7 level=info ts=2020-12-22T08:39:27.197Z caller=head.go:717 component=tsdb msg="WAL segment loaded" segment=3 maxSegment=7 level=info ts=2020-12-22T08:39:27.198Z caller=head.go:717 component=tsdb msg="WAL segment loaded" segment=4 maxSegment=7 level=info ts=2020-12-22T08:39:27.199Z caller=head.go:717 component=tsdb msg="WAL segment loaded" segment=5 maxSegment=7 level=info ts=2020-12-22T08:39:27.200Z caller=head.go:717 component=tsdb msg="WAL segment loaded" segment=6 maxSegment=7 level=info ts=2020-12-22T08:39:27.200Z caller=head.go:717 component=tsdb msg="WAL segment loaded" segment=7 maxSegment=7 level=info ts=2020-12-22T08:39:27.200Z caller=head.go:722 component=tsdb msg="WAL replay completed" checkpoint_replay_duration=102.209µs wal_replay_duration=7.33696ms total_replay_duration=7.495874ms level=info ts=2020-12-22T08:39:27.203Z caller=main.go:742 fs_type=794c7630 level=info ts=2020-12-22T08:39:27.203Z caller=main.go:745 msg="TSDB started" level=info ts=2020-12-22T08:39:27.203Z caller=main.go:871 msg="Loading configuration file" filename=prometheus.yml level=info ts=2020-12-22T08:39:27.204Z caller=main.go:902 msg="Completed loading of configuration file" filename=prometheus.yml totalDuration=1.170705ms remote_storage=2µs web_handler=500ns query_engine=1.5µs scrape=252.623µs scrape_sd=336.23µs notify=17.502µs notify_sd=18.502µs rules=1.5µs level=info ts=2020-12-22T08:39:27.204Z caller=main.go:694 msg="Server is ready to receive web requests." level=error ts=2020-12-22T08:39:27.253Z caller=klog.go:96 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:514: Failed to watch *v1.Node: failed to list *v1.Node: Get \"https://10.10.10.68:6443/api/v1/nodes?limit=500&resourceVersion=0\": x509: certificate signed by unknown authority" level=error ts=2020-12-22T08:39:28.554Z caller=klog.go:96 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:514: Failed to watch *v1.Node: failed to list *v1.Node: Get \"https://10.10.10.68:6443/api/v1/nodes?limit=500&resourceVersion=0\": x509: certificate signed by unknown authority" level=error ts=2020-12-22T08:39:31.675Z caller=klog.go:96 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:514: Failed to watch *v1.Node: failed to list *v1.Node: Get \"https://10.10.10.68:6443/api/v1/nodes?limit=500&resourceVersion=0\": x509: certificate signed by unknown authority" level=error ts=2020-12-22T08:39:37.017Z caller=klog.go:96 component=k8s_client_runtime func=ErrorDepth msg="/app/discovery/kubernetes/kubernetes.go:514: Failed to watch *v1.Node: failed to list *v1.Node: Get \"https://10.10.10.68:6443/api/v1/nodes?limit=500&resourceVersion=0\": x509: certificate signed by unknown authority"
Matthias Peng
--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/9caa85ba-8aee-48df-9fae-ef4078a3d1c7n%40googlegroups.com.
alex he
alex he
Matthias Peng
-k, --insecure Allow insecure server connections when using SSL
regards.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/d28aa7b0-52c3-4c96-81cf-c2f9b90d13b9n%40googlegroups.com.
