Errors in rules file - mapping values are not allowed in this context

[Ed Barc]

Hi all

Setting up my first Prometheus instance.

When adding this rule as  a test I get an error when i run  (indentation is ok in yaml)

- alert: PrometheusAllTargetsMissing
    expr: count by (job) (up) == 0
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: Prometheus all targets missing (instance {{ $labels.instance }})
      description: A Prometheus job does not have living target anymore.\n  VALUE = {{ $value }}\n  LABELS: {{ $labels }}

promtool check config /etc/prometheus/rules/alert.rules.yml

Checking /etc/prometheus/prometheus.yml
  SUCCESS: 1 rule files found

Checking /etc/prometheus/rules/alert.rules.yml
  FAILED: [yaml: line 1: mapping values are not allowed in this context]

Any advice on the mapping values error would be most welcome

Thanks

[Brian Candler]

The top level of a rules file must be rule groups.

groups:

- name: MiscellaneousRules

  rules:

    - alert: PrometheusAllTargetsMissing
      expr: count by (job) (up) == 0

      ... etc

Documentation references:

https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/

https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/

Note: I don't think that expression does what you want.  If there are no targets for job "foo" then there will be no metrics with 'up{job="foo"}', hence there will be no output from that 'count by (job)' expression with label job="foo"; *not* a metric with a value of zero.

I can't think of a way to find configured jobs which have no instances.  You may have to use absent() or unless() with an offset to compare current scrapes with previous ones.

[Ed Barc]

Hi Brain

Thanks for the reply

I have created another rule with more straightforward logic and I get the same sort of error.

groups:
- name: MiscellaneousRules
- alert: HostOutOfMemory
    expr: node_memory_MemAvailable_bytes{job=”node_exporter”} / node_memory_MemTotal_bytes{job=”node_exporter”} * 100 < 10
    for: 5m
    labels:
      severity: warning
    annotations:
    summary: "Host out of memory (instance {{ $labels.instance }})"
    description: "Node memory is filling up"

FAILED: [yaml: line 3: mapping values are not allowed in this context]

What I am trying to undertstand is exactly what is being mapped to where.

Thanks

[Brian Candler]

You didn't copy what I wrote.  You missed out the line "rules:", and also the indentation is significant:

groups:

  - name: MiscellaneousRules

    rules:

      - alert: PrometheusAllTargetsMissing
        expr: ...

(for a given rule group, "name:" and "rules:" must be indented at the same level; and each rule is a bullet underneath that)

Note that you don't have to indent sequences beyond the parent key, so the following is also accepted:

groups:

- name: MiscellaneousRules

  rules:

  - alert: PrometheusAllTargetsMissing
    expr: ....

> What I am trying to undertstand is exactly what is being mapped to where.

A "map" in YAML (and in Go) is what you might call a "dict" in Python or a "Hash" in Ruby or Perl: a mapping from key to value.

In YAML, a map can look like this:

key1: value1

key2: value2

key3: value3

although it can also be represented more compactly as

{ key1: value1, key2: value2, key3: value3 }

There are also "sequences" (= "list" or "array" in other languages).  Those are

- value1

- value2

- value3

or 

[ value1, value2, value3 ]

Both sequences and maps can contain other sequences and/or maps, by nesting.

The error message is saying you tried to put a map value in a place where one was not expected by prometheus.  That is: a file can be structurally valid YAML, but still invalid input to prometheus, because it has the wrong types of object in the wrong places.