Discuss the adoption of OpenTelemetry Metrics

[Emily Jiang]

As you might recall, Pavol started the thread on adopting OpenTelemetry tracing in MicroProfile and it was in general agreement that MP will adopt OpenTelemetry Tracing.  Pavol and Felix from MP Open Tracing are currently working on the prototype of adopting the tracing portion of OpenTelemetry, which is great.

I have been wondering the metrics part of OpenTelemetry. Today I took a look at the current status of OpenTelemetry, which is still under active development and the current release is 0.4.0. The metrics APIs on OpenTelemetry look quite simple, having meter and counter concept. It could be easily wrapped up by MP Metrics.

I am wondering what your thoughts are on the adoption of the whole OpenTelemetry (metrics and tracing) instead of just the tracing bit.

If we don't, our end users will be quite confused with the partial adoption. Since OpenTelemetry is the most recent effort under CNCF, it could be very likely to be adopted by many others, which then becomes mainstream.

If we do want to adopt OpenTelemetry metrics, MP Metrics team should participate on OpenTelemetry to define some useful APIs based on their past experience.

Thoughts?

Emily

  

[Jan Martiska]

Recently I've been hearing voices that we should adopt Micrometer instead, and wrap that to be used in one of the future MP Metrics releases. Don Bourne and David Chan can probably share more about this effort. The justification being that Micrometer is a mature, popular and widely used library with a rich API.

So now we have three approaches competing for being "the" MP Metrics in the future:
- Micrometer
- OpenTelemetry
- keeping and further evolving the old MP Metrics

I don't currently have a strong preference for any of these sides, but I think that if we decide to switch somewhere, it should be done with as little pain as possible, so embracing a new API by integrating/wrapping it into our current APIs would probably sound better than completely ditching our current API and switching to something else.
On the other hand, we don't want our API to bloat too much, and supporting two distinct APIs does sound like bloating.
So, I don't really know. This does need a wide discussion.

[Emily Jiang]

Hi Jan,

Don and David talked to me regarding micrometer. I did a twitter pool in order to find out how popular micrometer is. Here is the twitter poll. It still has one day to vote. Please help with retweet.

Here is the poll result so far.

I am pleased to know that there are around 30 customers use MP Metrics. The poll result might be biased due to the nature of my followers might be MP followers. I am not sure how popular Micrometer is. Once OpenTelemetry is out, people might adopt the new CNCF metrics framework just being new and modern, as mentioned earlier.

I am not suggesting ditching the current APIs. I think reworking the current APIs or introducing more APIs to wrap against the new shining mainstram is a good approach. I think we are in agreement here. We can then deprecate the unused APIs gradually so that MP Metrics can always offer useful APIs. Besides, the aim of MicroProfile is to hide the third party libs so that our customers don't need to worry about migration.

Thanks

Emily

[Emily Jiang]

For some reason, I could post the result. Let me try again. If it can't be displayed, below is the summary.

I don't use metrics: 30.9%

Micrometer: 29.6%

MicroProfile Metrics: 39.5%

Voters: 81. One day left for voting.

[Werner Keil]

You should have mentioned Dropwizard Metrics.

I assume multiple choices were possible, because it's over 100%? ;-)

Micrometer is still a relatively new project, it even started after MicroProfile Metrics or at least MP as a whole, so those who started using Spring Boot 1 or Dropwizard a long time ago probably have no reason to switch just because Spring Boot 2 is available.

Werner

[Emily Jiang]

You should have mentioned Dropwizard Metrics.

I didn't mention Dropwizard because MP Metrics adopts Dropwizard. If someone is on Dropwizard, they can easily move to MP Metrics if they seek for a defactor standard of doing metrics.

Thanks

Emily

[Ken Finnigan]

What MP Metrics should or shouldn't change from an API perspective shouldn't be discussed until there are actual implementations coming forward with ideas as to how the API should be adjusted.

Theorizing doesn't get MP to where it needs to be

--
You received this message because you are subscribed to the Google Groups "Eclipse MicroProfile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/2046a4f5-4570-4f95-8cf4-6b6c4fb8fe98%40googlegroups.com.

[Werner Keil]

Sure you got the right thread here?

The survey Emily mentioned did not mention any API changes, it was purely about usage of 2 libraries, with quite some ambiguity on the "adoption" or I'd rather call it "inspiration" by Dropwizard, but if some who actually use Dropwizard e.g. via Spring Boot and answered "MP Metrics" here that would explain the percentage ;-)

Am Dienstag, 26. Mai 2020 20:54:50 UTC+2 schrieb Ken Finnigan:

What MP Metrics should or shouldn't change from an API perspective shouldn't be discussed until there are actual implementations coming forward with ideas as to how the API should be adjusted.

Theorizing doesn't get MP to where it needs to be

On Tue, May 26, 2020 at 6:27 AM 'Emily Jiang' via Eclipse MicroProfile <microp...@googlegroups.com> wrote:

You should have mentioned Dropwizard Metrics.

I didn't mention Dropwizard because MP Metrics adopts Dropwizard. If someone is on Dropwizard, they can easily move to MP Metrics if they seek for a defactor standard of doing metrics.

Thanks

Emily

--
You received this message because you are subscribed to the Google Groups "Eclipse MicroProfile" group.

To unsubscribe from this group and stop receiving emails from it, send an email to microp...@googlegroups.com.

[Ken Finnigan]

Granted, I presume trying to "plug in" other metrics implementations will lead to API changes.

Maybe it won't, which is great. My concern is design before implementation.

To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/dfecb931-a7e9-481a-9d85-87967f806cd3%40googlegroups.com.

[Werner Keil]

De facto standard sounds a little too optimistic and through very rosy glasses, taking something like Prometheus it could be seen as the de facto standard because all the major collecting frameworks and projects like Dropwizard, MP Metrics or Micrometer and many others deliver metrics in its format.

OpenTelemetry is still in a very early stage, also sandbox at CNCF, but it looks promising. And respects most of the design principles that also went into MP Metrics including proper semantics via UoM.

Werner

[Werner Keil]

We already "plug in" heavily into Prometheus and its format now O;-)

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/dfecb931-a7e9-481a-9d85-87967f806cd3%40googlegroups.com.

[Erin Schnabel]

For some reason, a few people poked me about this thread, as I have been stirring the pot re: micrometer.

Micrometer is in use by a fairly significant community, and it isn’t only about Spring Boot. While Pivotal/VMWare sponsors the project, some of the core committers do not work there. If you want download/usage statistics, I can get them for you, but I would not use relative youth against that project against it. The people that built it have deep SRE experience, and it shows.

Both micrometer and OpenTelemetry metrics have similar abstraction patterns that separate the metrics used by the application from the exporter/provider backend. I would be be happy to walk anyone through the quarkus micrometer extension I’ve written, so you can understand the value the separation between MeterRegistry, MeterFilter and MeterBinder abstractions brings. Jan, I’ve been meaning to get back to you specifically to walk through what I did. I chased it mostly as proof of concept, but I think it does a lot to explain the limitations of MP Metrics as it exists right now.

I’ll be terribly honest: Dropwizard was great several years ago when I hit Mr. Bourne over the head with it. But things have moved on quite a bit since then, and prometheus is less than happy with some of the things dropwizard likes to do. Micrometer does have an active community, with _users_ of the library contributing binders for cloud services that weren’t already there. It is going to be hard for MP Metrics, as a spec, to keep up with that.

IMO, MP Metrics should be about “how does xyz library get injected in a consistent way using MP Config and CDI”. That would allow MP to be much more nimble in terms of supporting whatever the next new fangled industry hot thing is. Whether OpenTelemetry metrics or Micrometer or the thing we haven’t heard of before, MP will be much quicker to embrace industry trends if it stays focused on how that technology should be enabled and configured using MP Config and CDI. 

I could keep going and going (as both Mr. Bourne and Ms. Jiang will attest), but I’ll put the soapbox away for now. ;)

[Emily Jiang]

Thank you Erin for sharing the insightful view on Micrometer!

Both micrometer and OpenTelemetry metrics have similar abstraction patterns that separate the metrics used by the application from the exporter/provider backend.

Maybe OpenTelemetry metrics was inspired by the work from micrometer. Since OpenTelemetry is under CNCF and is establishing a language agnostic metrics model, personally I think it is very likely to set a standard for metrics. Besides, judging from the OpenTelemetry doc, it seems the analytics firms such as Datadog, Synatrace, Splunk, LightStep, New Relic are involved in the OpenTelemetry, which might influence other analytics firms to adopt this. I am also wondering whether micrometer will adopt OpenTelemetry when it releases since they are quite similar. It is difficult for us to decide right now. Maybe, we should wait for OpenTelemetry to release and see what will happen. However, I think it might be useful to contribute towards OpenTelemetry.

IMO, MP Metrics should be about “how does xyz library get injected in a consistent way using MP Config and CDI”. That would allow MP to be much more nimble in terms of supporting whatever the next new fangled industry hot thing is. Whether OpenTelemetry metrics or Micrometer or the thing we haven’t heard of before, MP will be much quicker to embrace industry trends if it stays focused on how that technology should be enabled and configured using MP Config and CDI.

 

I like the above statement! If MP Metrics can just support dropwizard, micrometer, OpenTelemetry via a config setting without much API changes, it will be awesome. Anyway, I think some investigation/involvement on micrometer and OpenTelemetry metrics are essential for evolving MP Metrics forward.

Jan/Don, what do you think?

Thanks

Emily

[Don Bourne]

To Erin's point on nimbleness, I agree that MP should take a much looser approach to libraries where we are not the main creators of those libraries.  For example, for JAX-RS or CDI we just point to those specs and describe what it means for an MP-compliant runtime to support them.  I think a similar approach could be taken for things like metrics and logging, where there are APIs in existence that people already use frequently.  Having consistent configuration and a clear TCK would ensure compatibility between MP implementations without us needing to fork or change the underlying metrics APIs we could support.

We are taking a close look at both Micrometer and OpenTelemetry metrics libraries.  I was hoping to get a better understanding of how complete those libraries are (so far it seems Micrometer is considerably richer in its capabilities than OpenTelemetry metrics), and form an opinion on how we could incorporate one or both of them into MP.  We're still gathering more info on OpenTelemetry (David Chan is digging...).  One of the big questions is adoption as there's no point in adopting another metrics API unless adoption of that API is widespread.  From Emily's poll, it seems like both MP Metrics and Micrometer have many users -- though I realize that poll result may be somewhat biased towards MP's API since Emily's contacts on Twitter are knowledgeable about MP.  

I don't think it should be considered a "done deal" that we need to change our metrics API, but I'd like to find a way to ensure MP metrics stays relevant -- even if other metrics libraries in the industry gain widespread adoption.  That could be accomplished by having Micrometer or OpenTelemetry metrics be able to plug-in behind the MP metrics API, or it could mean supporting Micrometer and/or OpenTelemetry metrics APIs for direct use by MP applications.  To Ken's point, before we consider changing the spec I think we should take a stance on what might work well and try it out in our implementation(s).

Don

[Werner Keil]

A looser coupling to certain libraries would benefit not just Metrics, the same goes for Fault Tolerance which is even more heavily inspired by Failsafe than Metrics was by Dropwizard.

Maybe there are not so many open standards or efforts to create one as OpenAPI or OpenTelemetry, but where these arise, it's a good idea to respect them.

Werner

[Jan Martiska]

+1 that before standardizing, we need to make sure that it will work reasonably well. Especially since this would probably be quite a big jump.

One thing that worries me is that, AFAIU, both OpenTelemetry Metrics and Micrometer have a very different programming model compared to ours - we have been basing it heavily on annotations. While Micrometer seems to support some annotated metrics too, they do not seem to be at the heart of the API, at least that's my impression (I just read the docs, I'm not super familiar with Micrometer, so I might be just misunderstanding stuff).

TBH I don't right now have a clear idea how integrating a new API while keeping the old one would technically be accomplished without turning everything into a bit of a mess, and I hope we're not planning to drop the current API, as that would, of course, completely break all current applications and force all our users to rewrite everything.

If we turn both APIs (MP Metrics and something else) into first-class citizens then we have something which will be confusing to users as they would probably be forced to choose which one to use.

If we just somehow "hide" the new library behind our current API, that doesn't seem to bring much benefit to anyone, because users would be unable (or it would be cumbersome) to make use of the goodies from Micrometer/OpenTelemetry API, because as was already mentioned, the APIs are quite rich.

Jan

[Erin Schnabel]

You’re right that micrometer does not rely on annotations. In my experience, relying completely (or even mostly) on annotations for measuring things is pretty odd. They are a nice accompaniment, but otherwise unnecessary. 

Most of what Micrometer does is accomplished via bindings (stuff is just measured as the binder does the magic to bridge the thing to be measured with the library) and filters (use properties to enable/disable or modify how a particular meter or group of meters behaves). OpenTelemetry behaves similarly. IMO, this is a +1 for the abstraction pattern. I agree that causes a problem for MP Metrics as it exists now. 

There are a few different ways that could be used to resolve this. One would be a breaking version of the spec (next major), while changes continue to be made to enhance the current version. That is probably the easiest: the base/vendor naming requirements are really weird compared to what other metrics libraries do. 

IMO, API/SPI choice (OpenTelemetry Metrics, Micrometer, existing MP API) should be ok. None of these libraries are tied to a specific vendor in the way MP is designed to avoid. If MP can outline how an MP runtime supports those libraries, that allows applications to be portable across MP runtimes (which is the end goal, as I understand it). MP could look at defining what metrics should be created for various MP components, but I’m not sure I would recommend that approach without a lot more input from SRE experts for what useful metrics are. This is an area where I think “cross-MP-runtime” consistency may be less important than other system/environment/SRE concerns.

--
You received this message because you are subscribed to a topic in the Google Groups "Eclipse MicroProfile" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/JnvkKjgkJ7U/unsubscribe.
To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/5e372d64-9ff0-46b1-97ab-4bcc2ce3face%40googlegroups.com.

--

Sent from Gmail Mobile

[Werner Keil]

Well the annotations are at least in a separate package, so factoring them out into a separate (Jigsaw/OSGi) module seems relatively easy and would be quite logical, so using only the API elements without annotations or together with them are both possible.

Werner

To unsubscribe from this group and all its topics, send an email to microp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/5e372d64-9ff0-46b1-97ab-4bcc2ce3face%40googlegroups.com.