How to monitor number of pods or containers?

305 views
Skip to first unread message

Vitaly Karasik

unread,
May 12, 2019, 6:42:58 AM5/12/19
to Google Stackdriver Discussion Forum
I use Kubernetes Horizontal Pod Autoscaling (HPA) and I'd like to build a graph with the number of pods (or containers) in this HPA configuration. 
Is there a proper way to build such metric?
The only workaround I found so far is using "count" aggregation for 'kubernetes containers' resource,  'uptime' metric. Graph based on this metric provides realistic values on small time intervals, while uses 'mean over 1 minute window'.  But value is wrong when it switches to 'mean over 1 hour window'.

TIA,
Vitaly

Shoucong Chen

unread,
May 13, 2019, 12:00:06 PM5/13/19
to Google Stackdriver Discussion Forum
Hi Vitaly,

Using "count" aggregation over an always-existed container / pod metric is still the recommended way to generate charts for number of containers / pods.
I can reproduce the wrong chart when "mean over 1 hour window" on "uptime" metric. It seems a bug and we will investigate. Thanks for notifying us!
In the meantime, you can probably choose a different metric (like "cpu usage time") to achieve your goal.
Please let me know if you have additional questions!

Best,
Shoucong

linux.il

unread,
May 13, 2019, 1:43:15 PM5/13/19
to Shoucong Chen, Google Stackdriver Discussion Forum
>Using "count" aggregation over an always-existed container / pod metric is still the recommended way 
I suspect it doesn't work by design for *not*  always-existed container / pod, and this is exactly situation with autoscaling. For instance, if we have my autoscaling fluctuating between 9 and 10 pods every  5 minutes, we'll see 9 'permanent' and 6 'temporary' pods during 1 hour, i.e. metric will be 15 instead of 9.5.

--
© 2016 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdr...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.
---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-d...@googlegroups.com.
To post to this group, send email to google-stackdr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-stackdriver-discussion/62778034-6d15-4039-b128-80a224a8cc21%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages