Remote Write Kuberenetes Mixin Recorded Rules

[rksakt...@gmail.com]

Hi All,

I'm trying to remote write kubernetes Mixin Recorded K8's rules and additional cAdvisor metrics exposed by prometheus operator. The config reload happens properly and I can see the below metric names in prometheus. The prometheus logs says Done replaying WAL but i don't see the metric on my remote write end. I even tried to reduce the capacity and shards to force the remote write but it didn't help. Can you help to check if my regex has any issue. I don't see any error log in the prometheus end as well.

If I remove the writeRelabelConfig I can see the metrics immediately populating at the remoteWrite end. 

remoteWrite:

        - url: "http://*******/api/v1/write"

          name: test

          queue_config:

            capacity: 100

            max_shards: 5

            min_shards: 1

          writeRelabelConfigs:

            - source_labels: [__name__]

              separator: ;

              regex: node_namespace_pod_container:container_cpu_usage_seconds_total:sum_rate|node_namespace_pod_container:container_memory_working_set_bytes|node_namespace_pod_container:container_memory_rss|node_namespace_pod_container:container_memory_cache|node_namespace_pod_container:container_memory_swap|namespace_memory:kube_pod_container_resource_requests:sum|namespace_cpu:kube_pod_container_resource_requests:sum|namespace_workload_pod:kube_pod_owner:relabel|container_memory_working_set_bytes|container_memory_rss|container_memory_cache|container_memory_swap|container_network_receive_bytes_total|container_network_transmit_bytes_total|container_network_receive_packets_total|container_network_transmit_packets_total|container_network_receive_packets_dropped_total|container_network_transmit_packets_dropped_total|container_cpu_cfs_throttled_periods_total|container_cpu_cfs_periods_total

              replacement: $1

              action: keep

Thank You

Regards

Sakthi