Why DELTA not supported in monitoring custom metrics

[Terry Guo]

Hello, I'm trying to use stackdriver custom metrics to monitoring qps of a http server on gce. Reading from the doc, custom metrics support GAUGE and CUMULATIVE, but not support DELTA. Does anyone know why the limitation exists? In this case, how do I define qps like metrics?

Maybe I can try CUMULATIVE, but the doc says "Cumulative metrics cannot be used for charting or alerting in Stackdriver Monitoring.".

Does anyone have success experience with custom metrics, or am I misunderstanding something here?

Thanks,

Terry

[Carlos (Cloud Platform Support)]

Hi Terry,

I am not sure what is the cause of the Delta limitation for custom metrics. Nevertheless, if you are using an HTTP server like Apache or NGINX, the way to go is to configure the monitoring plugin. It will report the number of requests per second the server receives.

[Terry Guo]

Thanks Carlos. I'm using a hand made golang HTTP server so I cannot use those plugins. Also, I want to have more control on metrics so I can add custom labels.

Agent is basically a collectd, so does it support adding my own plugin? I didn't find any documentation about that.

[Carlos (Cloud Platform Support)]

Hi Terry,

Indeed while Stackdriver agent uses collectd custom metrics does not. I am not sure you would be able to make it work using your own plugin. I have seen cases where a NoSQL database plugin using collectd was not supported. I am trying to find additional information, if I come with something useful I will post it.

[Carlos (Cloud Platform Support)]

Hi Terry, 

After further discussion, I believe the best approach for your use case would be to use Gauge Custom Metrics.

[Terry Guo]

Could you elaborate how does this work? What's the value of the Gauge metric? 

On Thu, Feb 16, 2017 at 7:17 AM, 'Carlos (Cloud Platform Support)' via gce-discussion <gce-dis...@googlegroups.com> wrote:

Hi Terry, 

After further discussion, I believe the best approach for your use case would be to use Gauge Custom Metrics.

--
© 2017 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-discussion@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to a topic in the Google Groups "gce-discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gce-discussion/7ra5WJbmKY0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gce-discussion+unsubscribe@googlegroups.com.
To post to this group, send email to gce-discussion@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/b22c2090-87cc-4ddb-9456-c2a0115fb615%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Carlos (Cloud Platform Support)]

There are some specific examples on how to use a gauge custom metric on this document. I believe you can keep count on the number of request per seconds on your Server and parse it to Stackdriver.

[Andrew van Rooyen]

Surely that's just a temporary workaround until we get delta for custom metrics? I mean, delta is as least (if not more) important as gauge for taking general measurements.

Of course, I could keep track of my metric by doing all the aggregation on my end, then send the result of that to Stackdriver as a gauge metric. But what value is Stackdriver providing in this scenario?

I understand that Stackdriver is being actively worked on, and certain features aren't available. But gauge metrics are not a replacement for delta - there is a very clear difference.

So instead of the "just use gauge" response, it would be great to get some insight into if and when delta will be made available for custom metrics. 

[Terry Guo]

+1 for Andrew's opinion. I'd also like to know if you guys have solid plan to improve this, so I could make a better decision about whether to use Stackdriver or build something on my own.

--

© 2017 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-discussion@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---
You received this message because you are subscribed to a topic in the Google Groups "gce-discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/gce-discussion/7ra5WJbmKY0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to gce-discussion+unsubscribe@googlegroups.com.
To post to this group, send email to gce-discussion@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/4a694f33-056e-4dfb-8dee-39c2cfaf5364%40googlegroups.com.

[Carlos (Cloud Platform Support)]

Hi Terry and Andrew,

Thank you for your feedback. Sometimes Google's Product Managers watch these discussions and are able to provide and receive feedback from the community. As you mentioned there are many features that are being worked on. I do not have a way to find out if this is in the road map. Nevertheless I would suggest filing a feature request. Your feedback is greatly appreciated.

[ki...@thirdfloorventures.com]

Bump.

What's the status on Delta custom metrics? It's kind of shocking that it's still not there a year later...

[Digil (Google Cloud Platform Support)]

Which exact metrics are you talking about under DELTA kind?. I can see some of them with DELTA kind, under various GCP metrics list.

[Kiran Ryali]

I was referring to custom metric support for DELTA metrics. See this above thread.

On Wed, Aug 8, 2018 at 6:22 PM, 'Digil (Google Cloud Platform Support)' via gce-discussion <gce-dis...@googlegroups.com> wrote:

Which exact metrics are you talking about under DELTA kind?. I can see some of them with DELTA kind, under various GCP metrics list.

--
© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043

 
Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-discussion@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.
---

You received this message because you are subscribed to the Google Groups "gce-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussion+unsubscribe@googlegroups.com.

To post to this group, send email to gce-discussion@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/b2a8136a-b754-49f0-8042-98287b218e0e%40googlegroups.com.

[Digil (Google Cloud Platform Support)]

Hello Kiran,

I have created a feature request to support DELTA kind on monitoring  Stackdriver customer metrics. I would highly recommend you to follow this issue-report link for any updates regarding this. 

I further request you to provide your suggestions, comments, any use-case scenario and also vote for this feature to be implemented on the Google Cloud Platform using the provided link.

[Mary Koes]

Thanks so much for the feedback, Kiran.  I've added my colleagues on the monitoring side for visibility here as well.  You should be able to declare it as a CUMULATIVE metric and write delta points to it (with a new reset timestamp for every point) as a workaround. 

Best,

Mary

Product manager, Stackdriver Logging

--

© 2018 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 

Email preferences: You received this email because you signed up for the Google Compute Engine Discussion Google Group (gce-dis...@googlegroups.com) to participate in discussions with other members of the Google Compute Engine community and the Google Compute Engine Team.

---
You received this message because you are subscribed to the Google Groups "gce-discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gce-discussio...@googlegroups.com.
To post to this group, send email to gce-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gce-discussion/d4e061e4-84ad-40af-8b3b-c40ac4ba1054%40googlegroups.com.