Web service monitoring in real-time

[Bryan Baugher]

Hi everyone,

I'm wanting to monitor a web service in real-time (seconds). The metrics that are produced seem to be setup for windowing over minute or more and are not very reactive. I would love to get things like the mean/rate/95% values for each reporting period (seconds) so that when I send the data off to my time-series db I can produce graphs that reflect the current state at that time.

The easiest way to achieve this seems to be to write a reporter to clear metrics after reporting but thats clearly bad and also not available for meters. Another option seems like I could create my own timers/histograms with a SlidingTimeWindowReservoir, though applying it seems difficult especially for @Timed annotations. 

Is there another way to do this?

Bryan

[Marshall Pierce]

Ah, the perennial “how do I reset my Timers/Histograms/etc” question… There isn’t a great universal way to do this right now. One partial solution is to use HdrHistogramResetOnSnapshotReservoir from https://bitbucket.org/marshallpierce/hdrhistogram-metrics-reservoir.

As far as customizing @Timed goes, depends on what you’re doing. If you’re using metrics-guice, I maintain that, and I’d be open to a PR that allows you to customize how timers, etc. are created. If you’re using the jersey integration, try https://bitbucket.org/marshallpierce/jersey2-metrics which already lets you customize how its timers are created. If you’re doing something else with @Timed, I’ve got nothing for you. :(

-Marshall

> --
> You received this message because you are subscribed to the Google Groups "metrics-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to metrics-user...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Bryan Baugher]

Wow this was exactly what I was looking for, thanks! So if I were to use the reset on snapshot reservoir would that mess up if I were using multiple reporters (i.e. graphite and jmx)? Unfortunately I am on the jersey 1.x train but your code seems easy to follow so hopefully I can probably find a similar way if need be.

[Marshall Pierce]

For jersey 1, try https://github.com/palominolabs/jersey-metrics-filter
(also by me, so I'm biased).

It probably won't do what you want if you register multiple reporters
via the usual way, so I'd suggest registering *one* reporter with
Metrics that then wraps all your other reporters.

On 04/08/2015 07:52 AM, Bryan Baugher wrote:
> Wow this was exactly what I was looking for, thanks! So if I were to use
> the reset on snapshot reservoir would that mess up if I were using
> multiple reporters (i.e. graphite and jmx)? Unfortunately I am on the
> jersey 1.x train but your code seems easy to follow so hopefully I can
> probably find a similar way if need be.
>
> On Tuesday, April 7, 2015 at 9:06:04 PM UTC-5, Marshall Pierce wrote:
>
> Ah, the perennial “how do I reset my Timers/Histograms/etc”
> question… There isn’t a great universal way to do this right now.
> One partial solution is to use HdrHistogramResetOnSnapshotReservoir
> from
> https://bitbucket.org/marshallpierce/hdrhistogram-metrics-reservoir

> <https://bitbucket.org/marshallpierce/hdrhistogram-metrics-reservoir>.

>
> As far as customizing @Timed goes, depends on what you’re doing. If
> you’re using metrics-guice, I maintain that, and I’d be open to a PR
> that allows you to customize how timers, etc. are created. If you’re
> using the jersey integration, try
> https://bitbucket.org/marshallpierce/jersey2-metrics
> <https://bitbucket.org/marshallpierce/jersey2-metrics> which already
> lets you customize how its timers are created. If you’re doing
> something else with @Timed, I’ve got nothing for you. :(
>
> -Marshall
>
> > On Apr 7, 2015, at 12:45 PM, Bryan Baugher <bjb...@gmail.com

> <javascript:>> wrote:
> >
> > Hi everyone,
> >
> > I'm wanting to monitor a web service in real-time (seconds). The
> metrics that are produced seem to be setup for windowing over minute
> or more and are not very reactive. I would love to get things like
> the mean/rate/95% values for each reporting period (seconds) so that
> when I send the data off to my time-series db I can produce graphs
> that reflect the current state at that time.
> >
> > The easiest way to achieve this seems to be to write a reporter
> to clear metrics after reporting but thats clearly bad and also not
> available for meters. Another option seems like I could create my
> own timers/histograms with a SlidingTimeWindowReservoir, though
> applying it seems difficult especially for @Timed annotations.
> >
> > Is there another way to do this?
> >
> > Bryan
> >
> > --
> > You received this message because you are subscribed to the
> Google Groups "metrics-user" group.
> > To unsubscribe from this group and stop receiving emails from it,

> send an email to metrics-user...@googlegroups.com <javascript:>.

> > For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "metrics-user" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to metrics-user...@googlegroups.com

> <mailto:metrics-user...@googlegroups.com>.