Re: [prometheus-users] Count unique values for a metric

[Julius Volz]

Hi,

So Prometheus doesn't work in an event-based way, but only collects the current *state* (value) of a series every N seconds. Meaning you can't really do what you want with Prometheus, unless you implement something like HyperLogLog (https://en.wikipedia.org/wiki/HyperLogLog) for stream cardinality estimation on the client side (in your app) and then expose the result (approximate count of unique users) as a single metric value.

I assume your number of user IDs can be very large, and is not something small and well-bounded?

Julius

On Wed, Aug 2, 2017 at 11:36 PM, MARSHALL HAVARD FISHER <mfi...@berkeley.edu> wrote:

Hello,

I've been working for the last day to accomplish a simple task: count the number of unique users that call a particular method. 

The user-id is passed as a parameter each time the method is called, so I figured my solution would be: create a gauge that is set to the user-id on each call, and then aggregate the number of unique values in the gauge for a given time period.

I have struggled with how to do this accurately. From the documentation it seems that count_over_time() is my best bet, but I can't tell if "the count of all unique values in the specified interval" implies unique values. My testing with this function has been very inconclusive; I'm very fuzzy on how that aggregation function operates it.

Could someone provide a recommendation how to tackle this problem or is count_over_time() the correct approach?

Thank you very much

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/03461611-f94e-4c27-a277-7b42fb91fed5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[marshal...@climate.com]

Thank you for your prompt help! Yes, your assumption is incorrect. One alternative I'm considering as an approximation is the https://prometheus.io/docs/querying/functions/#changes function, which seems like it would give the correct number if I set the range to my scrape interval and am willing to assume that each user only uses the feature once per scrape. Do you have thoughts on this?

On Wednesday, August 2, 2017 at 2:50:52 PM UTC-7, Julius Volz wrote:

Hi,

So Prometheus doesn't work in an event-based way, but only collects the current *state* (value) of a series every N seconds. Meaning you can't really do what you want with Prometheus, unless you implement something like HyperLogLog (https://en.wikipedia.org/wiki/HyperLogLog) for stream cardinality estimation on the client side (in your app) and then expose the result (approximate count of unique users) as a single metric value.

I assume your number of user IDs can be very large, and is not something small and well-bounded?

Julius

On Wed, Aug 2, 2017 at 11:36 PM, MARSHALL HAVARD FISHER <mfi...@berkeley.edu> wrote:

Hello,

I've been working for the last day to accomplish a simple task: count the number of unique users that call a particular method. 

The user-id is passed as a parameter each time the method is called, so I figured my solution would be: create a gauge that is set to the user-id on each call, and then aggregate the number of unique values in the gauge for a given time period.

I have struggled with how to do this accurately. From the documentation it seems that count_over_time() is my best bet, but I can't tell if "the count of all unique values in the specified interval" implies unique values. My testing with this function has been very inconclusive; I'm very fuzzy on how that aggregation function operates it.

Could someone provide a recommendation how to tackle this problem or is count_over_time() the correct approach?

Thank you very much

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.