Grouping by high cardinality metrics?

[Ryan B]

Hi all! First off, thanks for building and maintaining GCM/Stackdriver, Google people. It's great!

I have a log-based metric with a high cardinality label. When I try to graph it over a time period with ~5-10k unique values, Monitoring struggles to include them all individually:

Is there any way to break out just the top N values and collapse the rest? Preprocessing function, group by parameter, anything else? Happy to use MQL if necessary.

(I know this is a nontrivial problem in the general case, streaming algorithms etc, but I'm fine with any approximation that works and doesn't hang my browser for 10s of seconds at a time. 😁)

[Mary Koes]

Hi Ryan!

Thanks for reaching out with your challenge. Would outlier mode work for your use case?

@Lee Yanco and @Joy Wang may have better suggestions...

Thanks again,

Mary

--
© 2021 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdr...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.
---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-stackdriver-discussion/ddbda970-aff8-4b3e-bcad-999a790ee2c5n%40googlegroups.com.

[Igor Peshansky]

Are you looking for top?

        Igor

On Tue, Jul 13, 2021 at 11:57 PM Ryan B <goo...@ryanb.org> wrote:

--

[Lee Yanco]

Hey Ryan,

There are also the top and top_by functions in MQL: https://cloud.google.com/monitoring/mql/reference?hl=en#top-tabop https://cloud.google.com/monitoring/mql/reference?hl=en#top_by-tabop

These should filter out your chart to just the highest X time series.

--

Lee Yanco Product Manager lya...@google.com

[Ryan B]

Oh, awesome! Apologies, no clue how I missed that. This is perfect, thanks!