What empty spaces means in promethues graphs?

[stav alfi]

Intro:

I have total_betslips_requests_to_placing_manager metric that counts (prometheus-counter) the amount of incoming requets to specific route in my service.

Each point in the graph has 2 addtional custom labels: frequency (what type of request is it) and request_source (where did the request came from).

The following graph shows a simple query which I can't understand: The amount of requests for every frequency and every request_source.

Question

Why each sub-graph (color) doesn't start from the buttom? for example, the red line is a stright line that starts in (14:21:50,4) and ends in (6:40:45,4). How can it be that the red line doesn't start from (some-date,0)?

Addtional Info:

I'm using a counter, my client-lib is prom-client for node.js. 

--------------------------------

Thanks,

Stav Alfi

[Brian Candler]

> Why each sub-graph (color) doesn't start from the buttom? for example, the red line is a stright line that starts in (14:21:50,4) and ends in (6:40:45,4). How can it be that the red line doesn't start from (some-date,0)?

The reason the line doesn't start from zero is because you didn't scrape a zero value for that timeseries.  The first point shown is the first value scraped.

In prometheus, every unique combination of metric_name + labels defines a new timeseries.  It starts when the first data point is scraped.  If you stop providing data then it goes "stale" - either when the scrape job which had previously scraped that timeseries does another scrape and that timeseries is not present, or after 5 minutes (although it can come back to life again if the exact same metric_name+labels combination appears in a subsequent scrape)