Using Snapshot to ask about a time interval.

[Benson Margulies]

I'd like to use a Histogram to implement some adaptive behavior. The question I want to ask is, what's the mean over the last minute? The javadoc for 'Snapshot' doesn't seem to tell me: is that going to be since the beginning of time or since the last snapshot? Or something else?

[Marshall Pierce]

On Jan 20, 2016, at 6:17 AM, Benson Margulies <ben...@basistech.com> wrote:
>
> I'd like to use a Histogram to implement some adaptive behavior. The question I want to ask is, what's the mean over the last minute? The javadoc for 'Snapshot' doesn't seem to tell me: is that going to be since the beginning of time or since the last snapshot? Or something else?

This isn’t really well-defined right now. Different tools want it done different ways, and metrics tries to play nice, but we don’t have API to express the differences very well (yet).

In my custom reservoir backed by hdrhistogram (https://bitbucket.org/marshallpierce/hdrhistogram-metrics-reservoir) there’s another reservoir impl that resets its state every time you ask for a snapshot for exactly uses like this. This will give you the behavior you want: get a snapshot once per minute and it will include only the last minute’s data.

As a side note, be sure that the thing you’re measuring has a mean that’s useful (i.e. is normally distributed). Latency generally is not normal, so the mean isn’t very useful.

[Benson Margulies]

To be clear, I don't care about a histogram at all. What I need is the average of a queue depth over the last N minutes. If that suggests anything even better than your resevoir, please let me know.

--
You received this message because you are subscribed to the Google Groups "metrics-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to metrics-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Benson Margulies]

So maybe I just need SlidingTimeWindowReservoir.

[Marshall Pierce]

It depends; do you want a sliding window, or a sequence of N-minute buckets? Those are somewhat different numbers. Are you sampling the queue depth at regular intervals, or how are you capturing those measurements?

[Benson Margulies]

I am obtaining a queue size at regular intervals (say, 30 seconds). I
want to make a flow control decision every so often (say, every minute
or few).

RIght now, the code I wrote just uses the SlidingWindow behind the
ordinary histogram, calls update when I make a measurement, and calls
getSnapshot().getMean() when considering a decision.

[Marshall Pierce]

OK, makes sense. If the semantics of the sliding window reservoir work well for your use case, then I think you’re doing exactly what you need to be doing.

-Marshall