MP Metrics future

[Ken Finnigan]

Hey everyone,

I appreciate this post might be “out of the blue”, or even a shock, but I wanted to raise the concern given the past couple of months I’ve had working on metrics related tasks for Quarkus.

First up, the shock, I’d like to recommend removing MP Metrics 3.0 from the MP 4.0 platform release.

Before I get trampled, let me explain why.

In late 2016, early 2017, when metrics were first being discussed for MicroProfile, DropWizard was the only solution for Java applications at the time. Even Spring Boot 1.x had its own way of capturing and presenting metrics from an application.

The world has changed a lot since then. Site Reliability Engineer (SRE) is a defined role that is a critical part of operations, and it’s the SRE role that is key when it comes to monitoring solutions.

In 2017 Jon Schneider, former Netflix SRE, founded the Micrometer project to provide a solution for metrics in Java frameworks and applications but designed around the needs of an SRE. Micrometer approaches the problem from a different perspective, by acknowledging that there are many different monitoring solutions available (Prometheus, Datadog, Stackdriver, etc) with each taking slightly different approaches to how they want metrics. That is the core reason why Micrometer treats each monitoring solution, or registry, as implementation and why there isn’t a single implementation.

So, why do I think we should remove MP Metrics from the platform?

Over the last couple of months, I’ve been exposed to how Micrometer handles metrics and have had discussions with Jon about the different approaches between Micrometer and MP Metrics. That is one of the reasons we had a call with Jon a few weeks back to discuss the two and how they could better align.

I’ve come to realize that what is currently defined for MP Metrics is not aligned with the SRE view of the world, and in many cases poses a hindrance to the adoption of MicroProfile runtimes in environments with other Java runtimes. You might wonder why that is the case. Micrometer is used within Spring Boot, Micronaut, Vert.x, and many other frameworks and libraries in the Java ecosystem.

Here are some key differences between MP Metrics and Micrometer:

• Hierarchical metric names
• For MP Metrics that’s the only option
• With Micrometer it’s possible to export hierarchical names based on the registry being used but is not the default
• Metric scopes
• MP Metrics prefixes all metrics with “base”, “vendor”, or “application”.
• Micrometer doesn’t have differentiation by metric type in that manner.
• Output formats
• MP Metrics supports Prometheus exposition and JSON formats, which to be honest I’m not sure why JSON is there.
• Micrometer supports exporting to Azure Monitor, Datadog, Dynatrace, Graphite, JMX, New Relic, Prometheus, Stackdriver, Statsd, and many more
• Naming conventions
• MP Metrics output is:
• application_io_quarkus_app_MyResource_any_seconds_count 1.0
• Micrometer output is:
  
• http_server_requests_seconds_count{method="GET",outcome="SUCCESS",status="200",uri="/app/any",} 1.0
• Metric units
  
• MP Metrics requires each metric being created define it’s unit of measure
  
• Micrometer leaves that up to the registry implementation as some monitoring solutions use different metric units.
  
• Automatic instrumentation
  
• MP Metrics defines optional REST endpoint metric output in most recent spec release. Prior to that, getting REST endpoint metrics required developers to add the metric annotations themselves.
  
• Micrometer has many implementations of its MeterBinder from libraries like Vert.x, Hibernate, Kafka, Hystrix, Tomcat, and more, which by their presence in an application with Micrometer result in defined metrics being produced without any effort from a developer.
  

Ok, so do these differences matter? Here is a list of problems experienced because the two are so different:

• Metrics output by an application using MP Metrics and another application using Micrometer are not compatible and cannot be collated together easily. Due to the metric scopes and naming conventions being completely different.
• MP Metrics use of package and class name in the metric makes it difficult to aggregate data across all endpoints within a single application, and even worse across many applications. When monitoring queries rely on specific naming of the metric to aggregate information and the class or package name changes, monitoring breaks! Also, an SRE doesn’t care about the name of a package or class, they just want to see the failure rate of endpoints to know when a problem is occurring.
• Micrometer includes a lot more export options than just Prometheus. Restricting MicroProfile to only Prometheus, while simplifying the specification, means that we’re incompatible with any environment that doesn’t use Prometheus.
• MP Metrics use with many popular Java libraries and frameworks doesn’t result in new metrics appearing. Each framework would need to be instrumented for MP Metrics, or the application developer specifically codes ways to capture metrics from them.

There are quite likely more issues that I’ve forgotten about or just not realized, but I wanted to note as many as I could.

I know in MicroProfile we like to specify APIs, but this is a case where I believe we need to acknowledge that the monitoring and SRE world has moved in a different direction to where MP Metrics has been heading, and we need to course-correct sooner rather than later.
Could MP Metrics create an API on Micrometer? Yes, but I would ask the question why? Micrometer is already an abstraction over the different registries for monitoring, why add another abstraction on top just to have an API for a specification?

Now, could MP Metrics define some type of CDI “glue” for Micrometer, as it’s a pure Java API? Maybe. But reaching consensus as to what “glue” might be needed will take time. Right now my main concern is continuing to promote an approach to metrics in MP that flies in the face of all other Java metrics solutions today. MP Metrics is very much “out on its own” in how it does things.

What happens if we take out MP Metrics?

I see two solutions at present, but there may be more:

1. MP platform spec states that runtimes should utilize Micrometer for metrics.
2. MP platform spec doesn’t indicate anything about metrics, but in the future, we create a new spec that focuses on monitoring output. Note, this option might better fit with OpenTelemetry metrics from a Java ecosystem perspective.

Once again, apologies for this rather large, and late, discussion for MP 4.0. However, it was only in the last few days that I had the realization that the MP platform should be corrected with regard to metrics before we go even further down a divergent path.

Thanks for reading, and I hope we can reach some consensus on this quickly.

Regards
Ken

[Ken Finnigan]

I knew there would be more solutions!

Another solution for what to do with the removal of MP Metrics is to utilize Micrometer (same as option 1), but then provide, or define, a shim that means MP Metrics APIs from the last version will be registered into Micrometer.

This would mean that existing applications don't stop working if they use MP Metrics annotations and APIs with MP 4.0, but we put forth the intent they won't be around forever.

[Don Bourne]

Hi Ken,

It's a holiday in Canada today (and I'm prepping for a conference talk I'm doing tomorrow).  I'll respond back with my thoughts later tomorrow.

Regards,

Don

[Bruno Baptista]

Hi All,

I agree that we should move towards the needs of the SRE world... This comment is also valid for open-telemetry instead of open-tracing.

If we decide to move on, we need a deprecation plan:

• Drop MP Metrics altogether. Maybe the work of using the MP Metrics annotations to wrap Micrometer might not be worth it because of difference in adoption.
  
• We can support MP Metrics and "MP Micrometer" annotations at the same time. We can mark the first ones as deprecated. I like this one.
  
• We can create a migration tool or we can create documentation with examples on how to migrate things. The first one is risky and the last is not very user friendly.
  
• In any case we need to give time for the different MP projects to do the switch. Supporting both annotations for a while would smooth things.
  

If we do this I don't see it as a failure of MP, on the contrary. I learned a lot with MP Metrics.

Also, we can use this case to settle the policy on how to deprecate APIs. What users should expect of a deprecation plan.

Cheers.

Bruno Baptista
https://twitter.com/brunobat_

--
You received this message because you are subscribed to the Google Groups "MicroProfile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CAKeeVe704JVjY4rWArS7CJ9%2BDwypSiMro34ibCC6jkHmquKvSw%40mail.gmail.com.

[Kevin Sutter]

Hi Everyone,

I've been in various discussions about the future of MP Metrics and how it relates (or doesn't relate) to Micrometer, or OpenTelemetry, or ...  My concern with dropping Metrics completely from the MP 4.0 Platform release is that we don't have any defined roadmap or alternative.  At least we don't have a direction in hand that lines up with our 4.0 release plans.  But, that doesn't mean we can't state our intentions by deprecating MP Metrics in 4.0.  This way, we would continue to ship MP Metrics in its current form.  But, it would be marked Deprecated to indicate that it's EOL is on the horizon.  Then, when we do define an MP Micrometer, or MP OpenTelemetry, or ..., we'll have a defined roadmap before dropping Metrics completely from the Platform.

(I should clarify that by including MP Metrics in MP 4.0, we would still move up to the Jakarta EE 8 dependencies.  We just wouldn't enhance the MP Metrics API in 4.0.)

-- Kevin

To unsubscribe from this group and stop receiving emails from it, send an email to microprofile+unsubscribe@googlegroups.com.

[Erin Schnabel]

My proposal would be: 

1. Deprecate MP Metrics in favor of Micrometer for MP 4.0

2. Provide an (optional) shim for mapping MP Metrics onto Micrometer (note that micrometer is not annotation based, if there is anything we could keep long-term, it is the convenience annotations, but we should have different conversation about whether or not they're doing what we intend them to do). This shim is optional because I doubt vendors would use it verbatim. The extension I have for Quarkus, e.g., already maps the Micrometer API.

One thing that should be clear is any expectation of naming consistency. The MP Annotations (outside of absolute names) are oriented around hierarchical naming conventions and are super concerned about uniqueness. With dimensional time series, you more often want overlapping things to make aggregation (and subsequent analysis based on selected tags) possible.  I've deliberately headed off in a different direction regarding names in the quarkus micrometer extension, but then Quarkus will have the smallrye extension around for naming compatibility.

To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.

[Don Bourne]

My thoughts roughly line up with Kevin's.  Essentially I think we could do both of the following:

- include MP Metrics 3.0 in the MP 4.0 platform (as initially planned)

- provide a deprecation statement to indicate we're looking at how to exploit Micrometer in future releases and that people instrumenting new apps should consider using Micrometer API

My rationale for not removing Metrics from the MP 4.0 platform is that we don't have a clear alternative yet

- at minimum I think we'd want to define Micrometer API + some set of expected metrics to be output from each compliant server in addition to those metrics coming from Micrometer metric bindings

- we may want to specify which version of Micrometer API is expected to be supported with each MP release if we want to ensure compatibility (as is done for MP Open Tracing today)

- we need to address how we expect to handle cross-spec metrics, such as Fault Tolerance metrics

- we may want to continue to have annotations support

- we may want a shim layer to help people migrate from MP Metrics to MP Micrometer (or whatever it might be called)

ps. to avoid giving people the wrong impression, MP Metrics naming convention is basically the same as Micrometer's -- both have the ability to specify the exact name you want to use and tags you want to apply for each metric.  MP Metrics does add prefixes to metric names (base / application / vendor), which I think a number of us agree we'd like to do away with.

Regards,

Don

On Monday, August 3, 2020 at 10:39:34 AM UTC-4, Ken Finnigan wrote:

[Jan Martiška]

+1 to Don and Kevin.
Removing MP Metrics from the platform at this point just because we're preparing something new, that sounds a bit too crazy to me.
I have some doubts we have enough time to prepare a full-fledged alternative before MP 4.0 goes out.

I'd say let's issue some sort of proper deprecation notice along with Metrics 3.0 and continue working on an alternative, and include that when it's clearly defined and considered stable enough. Ideally when it's also proven by having received bake time in an implementation.

[Ken Finnigan]

I'll respond to various comments and then provide some additional thoughts.

Bruno, thanks for chiming in with your thoughts, appreciate external contributors providing their views. With respect to your preferred option of supporting both annotations from MP Metrics and Micrometer at the same time, I don't know if that fits as Micrometer has a few annotations, but I don't think it has them for everything as it takes a more Java API approach instead of annotations.

I also agree with your statements around this not being seen as a failure. We can't learn what is right or wrong if we do nothing, and at the time MP Metrics was created there really wasn't anything akin to Micrometer for us to see as an alternative.

Kevin, I'm not sure I understand your comments around not having a defined roadmap?

The roadmap is to remove MP Metrics and say "use Micrometer". There would be no MP Micrometer, it would just be Micrometer. MicroProfile needs to move away from the thinking that for us to include or use something it needs to have wrappers or an abstraction that MP provides.

Erin, I agree that your suggestion is the clearest and straight forward approach to moving MP forward in relation to Metrics. I would prefer MP Metrics to be completely removed for MP 4.0 but appreciate the APIs are still used in other MP platform specs.

Don, is your "deprecation statement" actually deprecating MP Metrics, or only stating there's an intention to in the future?

I'm also not sure how users can begin using Micrometer APIs in their new apps without it being part of MP 4.0? If they can, would they? If MP only says "we're investigating", I don't think many people would jump to start using it.

I don't agree there isn't a clear alternative. Why can't "use Micrometer" be a clear alternative? Do we need to define everything in detail? If all runtimes use Micrometer, isn't the output identical? I would also caution against defining specific Micrometer versions that are "compatible" with MP platforms, as it ties MP into specific versions.

From my perspective, all your comments on what we might want to do are accurate but don't stop us from deprecating MP Metrics and switching to Micrometer today. Adding some or all of those other pieces can happen over time.

Jan, we're not really preparing something new. I'm saying "use Micrometer", which is not new. With regards to your "bake time" comment, Micrometer has been around for 3 years or so now, so I'm not sure what kind of bake time you're referring to.

So, some additional comments from me.

Many in the MP community say it needs to innovate, move fast, fail-fast, and break things when needed. Most of the comments so far are leaning in the other direction of, take it slow, let's wait, we need everything lined up before taking a step.

We can't have it both ways. Either we need to innovate fast and accept that there are times where we need to adjust course quickly, or we may as well fold into Jakarta.

If we took the approach of only deprecating MP Metrics, or only stating our intention to deprecate, without making Micrometer a part of MP 4.0, it then means for the next 10 months we're still pushing MP Metrics as "the way" to do metrics even though we know it doesn't align with SRE requirements or any other Java runtime.

If we included Micrometer has part of MP 4.0, and MP Metrics was present but deprecated, then users can begin modifying their apps to use Micrometer when they update to MP 4.0.

If we only deprecate MP Metrics but don't include Micrometer, then users still need to keep developing new applications to use MP Metrics until at least June 2021 and MP 5.0, which is really July or August once there's a runtime released that has updated. I really don't see having users stay on MP Metrics for nearly a year as being a good option for MP or it's users.

Ken

--

You received this message because you are subscribed to the Google Groups "MicroProfile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/314740ec-c5d2-4c41-8a6d-f792ebb7fd72n%40googlegroups.com.

[Emily Jiang]

My suggestion is to continue the release of MP Metrics 3.0 and get it included in MP 4.0 release. I am not sure why we issue a deprecation notice at all before we have any clear view on whether to align with OpenTelemetry or Micrometer. However, I think the next priority is to discuss the direction of MP Metrics by seriously looking at Micrometer and OpenTelemtry, which should has its 1.0 released soon.

By the way, the big chunk of the email is about the difference between Micrometer and MP Metrics. I don't think it is legimitate to deprecate MP Metrics just because it is different from Micrometer. As said earlier, MP has not decided whether to adopt OpenTelemetry or Micrometer as yet. Before we do anything too ratical against MP Metrics, we should continue the conversation on the thread of adopting micrometer or OpenTelemetry.

Thanks

Emily

To unsubscribe from this group and stop receiving emails from it, send an email to microp...@googlegroups.com.

[Jan Martiška]

On Tuesday, August 4, 2020 at 12:51:31 PM UTC+2 Ken Finnigan wrote:

I'll respond to various comments and then provide some additional thoughts.

Bruno, thanks for chiming in with your thoughts, appreciate external contributors providing their views. With respect to your preferred option of supporting both annotations from MP Metrics and Micrometer at the same time, I don't know if that fits as Micrometer has a few annotations, but I don't think it has them for everything as it takes a more Java API approach instead of annotations.

I also agree with your statements around this not being seen as a failure. We can't learn what is right or wrong if we do nothing, and at the time MP Metrics was created there really wasn't anything akin to Micrometer for us to see as an alternative.

Kevin, I'm not sure I understand your comments around not having a defined roadmap?

The roadmap is to remove MP Metrics and say "use Micrometer". There would be no MP Micrometer, it would just be Micrometer. MicroProfile needs to move away from the thinking that for us to include or use something it needs to have wrappers or an abstraction that MP provides.

Erin, I agree that your suggestion is the clearest and straight forward approach to moving MP forward in relation to Metrics. I would prefer MP Metrics to be completely removed for MP 4.0 but appreciate the APIs are still used in other MP platform specs.

Don, is your "deprecation statement" actually deprecating MP Metrics, or only stating there's an intention to in the future?

I'm also not sure how users can begin using Micrometer APIs in their new apps without it being part of MP 4.0? If they can, would they? If MP only says "we're investigating", I don't think many people would jump to start using it.

I don't agree there isn't a clear alternative. Why can't "use Micrometer" be a clear alternative? Do we need to define everything in detail? If all runtimes use Micrometer, isn't the output identical? I would also caution against defining specific Micrometer versions that are "compatible" with MP platforms, as it ties MP into specific versions.

From my perspective, all your comments on what we might want to do are accurate but don't stop us from deprecating MP Metrics and switching to Micrometer today. Adding some or all of those other pieces can happen over time.

Jan, we're not really preparing something new. I'm saying "use Micrometer", which is not new. With regards to your "bake time" comment, Micrometer has been around for 3 years or so now, so I'm not sure what kind of bake time you're referring to.

In the past weeks you have been mentioning working on a "stripped-down MP Metrics spec and API" that would focus on the nomenclature of metrics exposed from various frameworks - that's basically what I was referring to by the "something new". Is that effort still ongoing?

[Erin Schnabel]

The OpenTelemetry or Micrometer debate can be settled later. I am working with the OpenTelemetry team to sort out how the two can work together. For the rest of this year and next, micrometer is the way to go. While OTel metrics has a GA in the plan, real readiness for use looks questionable. there will be a lot missing.

My concern echoes Ken’s... I would like MP Metrics to stop receiving a blessing for use... that means deprecation and alternate. I have one working mp metrics -> micrometer shim, and I am happy to share it sooner rather than later. We can keep the annotations if that is what people feel strongly about (though they impose restrictions on the use of labels/tags). The annotations are the easiest to shim. The other changes to the registry API also help in this regard. 

To summarize, mp metrics -> micrometer (OTel metrics). Ken and I are messing with the latter (with ppl from both micrometer and OTel metrics). Micrometer has a lot if use, there are lots of reasons to make them work together and not treat it as an either/or situation.

You received this message because you are subscribed to a topic in the Google Groups "MicroProfile" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/mMvGzLq0Zkc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/79437ae5-626f-46a5-a655-f88c7608b352o%40googlegroups.com.

--

Sent from Gmail Mobile

[Ken Finnigan]

On Tue, Aug 4, 2020 at 7:28 AM Jan Martiška <wray...@gmail.com> wrote:

On Tuesday, August 4, 2020 at 12:51:31 PM UTC+2 Ken Finnigan wrote:

I'll respond to various comments and then provide some additional thoughts.

Bruno, thanks for chiming in with your thoughts, appreciate external contributors providing their views. With respect to your preferred option of supporting both annotations from MP Metrics and Micrometer at the same time, I don't know if that fits as Micrometer has a few annotations, but I don't think it has them for everything as it takes a more Java API approach instead of annotations.

I also agree with your statements around this not being seen as a failure. We can't learn what is right or wrong if we do nothing, and at the time MP Metrics was created there really wasn't anything akin to Micrometer for us to see as an alternative.

Kevin, I'm not sure I understand your comments around not having a defined roadmap?

The roadmap is to remove MP Metrics and say "use Micrometer". There would be no MP Micrometer, it would just be Micrometer. MicroProfile needs to move away from the thinking that for us to include or use something it needs to have wrappers or an abstraction that MP provides.

Erin, I agree that your suggestion is the clearest and straight forward approach to moving MP forward in relation to Metrics. I would prefer MP Metrics to be completely removed for MP 4.0 but appreciate the APIs are still used in other MP platform specs.

Don, is your "deprecation statement" actually deprecating MP Metrics, or only stating there's an intention to in the future?

I'm also not sure how users can begin using Micrometer APIs in their new apps without it being part of MP 4.0? If they can, would they? If MP only says "we're investigating", I don't think many people would jump to start using it.

I don't agree there isn't a clear alternative. Why can't "use Micrometer" be a clear alternative? Do we need to define everything in detail? If all runtimes use Micrometer, isn't the output identical? I would also caution against defining specific Micrometer versions that are "compatible" with MP platforms, as it ties MP into specific versions.

From my perspective, all your comments on what we might want to do are accurate but don't stop us from deprecating MP Metrics and switching to Micrometer today. Adding some or all of those other pieces can happen over time.

Jan, we're not really preparing something new. I'm saying "use Micrometer", which is not new. With regards to your "bake time" comment, Micrometer has been around for 3 years or so now, so I'm not sure what kind of bake time you're referring to.

In the past weeks you have been mentioning working on a "stripped-down MP Metrics spec and API" that would focus on the nomenclature of metrics exposed from various frameworks - that's basically what I was referring to by the "something new". Is that effort still ongoing?

Ah ok, right now my preferred approach is to just use Micrometer "as is" and worry about any additional layering and definition later.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/4645316e-8d84-44b8-a518-9807d8679e2an%40googlegroups.com.

[Ken Finnigan]

Indeed, OpenTelemetry is a long way from being "usable", especially for metrics.

Emily, though my original post talked at length about the differences between MP Metrics and Micrometer, it's because Micrometer focuses on the requirements of an SRE, and MP Metrics doesn't. Yes, it's a difference, but in the context of the monitoring problem we're trying to solve, it's a very large one.

There's also the consideration that MP Metrics is only used by MP runtimes. Micrometer, as mentioned, is used by Spring, Micronaut, Vert.x, and many others. Right now, MP is the outlier in the Java ecosystem by not using Micrometer. This fact causes real issues for SREs in trying to provide dashboards that span different Java runtimes.

You mention not wanting to be too radical, but isn't that the point with MP? Be radical, try something out, and if it doesn't work change direction.

In my view, MP Metrics has "lost" the metrics battle, we're on our own in continuing to use it. If we continue to push it as "the way" for another year, it further harms MP and makes it even harder to change direction in the future.

As I said before, if we accept MP is about innovation, we need to equally accept that doing so means breaking things when we've innovated in a direction that needs to change.

Ken

To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CADJ_gaTqjG2CX_dkUQVJb92if_6QFY5ROn1J%2BWZ11VDyn5zTHw%40mail.gmail.com.

[Ladislav Thon]

út 4. 8. 2020 v 12:51 odesílatel Ken Finnigan <k...@kenfinnigan.me> napsal:

Many in the MP community say it needs to innovate, move fast, fail-fast, and break things when needed. Most of the comments so far are leaning in the other direction of, take it slow, let's wait, we need everything lined up before taking a step.

That's probably because there are two vocal minorities, and the usual silent majority.

 

We can't have it both ways. Either we need to innovate fast and accept that there are times where we need to adjust course quickly, or we may as well fold into Jakarta.

And this is just a false dilemma. Let's not do this. Let's figure our own way, but let's not throw the baby out with the bathwater.

My perspective, as someone who is merely a user of MP Metrics:

Saying "MP Metrics are gone, use Micrometer" is simply not how API evolution should work. Deprecation periods are a standard thing and we should use them. No, the period doesn't have to be multiple years. It doesn't have to be even one year. 4 to 6 months sounds perfectly reasonable to me. If I remember correctly, the possibility of multiple major MP releases per year has not been strictly ruled out, and if the whole community agrees that removing a spec needs to happen, then that's a good reason to do potentially more than one major release in a given year, in my book.

Also, the proposal doesn't address the situation of other MP specifications that integrate with MP Metrics. What are they supposed to do in MP 4.0? Keep MP Metrics integration, probably as an optional part of the spec, even though MP Metrics are no longer in the platform? (Would MP Metrics even release their 3.0?) Remove MP Metrics integration as well, leaving metrics implementation-defined? Replace the MP Metrics integration with Micrometer integration in a few weeks? (I don't think so.)

The deprecation period I suggested above would give the necessary time not to just users, but also other MP specifications.

LT

P.S.: could someone expand on "SRE requirements"? I've heard this argument mentioned several times in this discussion, but it's very opaque to me. What is it that Micrometer does and MP Metrics doesn't? The only real difference I've been able to figure out is collecting metrics. MP Metrics specify a pull model biased towards Prometheus, while other metric aggregation platforms may want a push model and/or different data granularity. But surely there's more?

BTW as much as I don't like the MP Metrics annotations (why is there so much attributes I need to fill?), I'd certainly prefer them over the Micrometer plain Java API.

[Ken Finnigan]

As mentioned in one of my follow up responses, I'm good with marking MP Metrics 3.0 deprecated in MP 4.0, if we also say "use Micrometer" for metrics with MP 4.0 platform.

As such, for MP 4.0 any specification that uses MP Metrics would continue to do so, but future versions would change to use Micrometer instead.

With respect to "SRE Requirements", there are several aspects to it:

• Metric names containing package and class name, in addition to the metric scope prefix, cause metrics to be brittle to name changes, in terms of what's in a dashboard, but more critically they prevent an SRE from aggregating metrics across many metrics of the same type. For example, instead of each JAX-RS endpoint having a custom metric name prefix for each one, there would be a metric prefix of "http_server_requests" with tags to identify the URL, etc. This offers the ability to drill down to a specific endpoint or provide an aggregated view across all JAX-RS endpoints in an application, or even across applications.
• Not all environments use Prometheus. Only offering Prometheus as an output format means MP Metrics is only useful when Prometheus is also present. While it is certainly popular, it's not the only monitoring solution available.

These are just some of the SRE needs that MP Metrics doesn't meet, but Micrometer does. I'm sure there are others as well.

Ken

--

You received this message because you are subscribed to the Google Groups "MicroProfile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CALbocOkbRzih4VD3vH9xLzrAZoQPEvMx2HVA0fXbG_TO1kriNQ%40mail.gmail.com.

[Emily Jiang]

+1 LT!

You mention not wanting to be too radical, but isn't that the point with MP? Be radical, try something out, and if it doesn't work change direction.

In my view, MP Metrics has "lost" the metrics battle, we're on our own in continuing to use it. If we continue to push it as "the way" for another year, it further harms MP and makes it even harder to change direction in the future.

As I said before, if we accept MP is about innovation, we need to equally accept that doing so means breaking things when we've innovated in a direction that needs to change.

 

 Ken, I am not objecting trying something out and change direction. However, we need time to figure out how to join the dots and tell a straight story to our end users. At the moment, I don't think we have worked out what MP can offer about Metrics if we get rid of MP Metrics. As LT pointed out, what do we do for the specs that integrates with MP Metrics. We need to work this things out for our end users. Once we worked out the path, we then do it. I don't think we should destroy the current house without a shelter is provided.

As for OpenTelemetry, even though it has some way to go, we should not ignore it because it defines a consistent view across programming languages. It is also strange we take Tracing part not Metrics part. I think we should not limit ourselves but keep open minded so that we could potentially work with either OpenTelemetry and Micrometer.

Thanks

Emily

[Erin Schnabel]

On Tue, Aug 4, 2020 at 10:49 AM 'Emily Jiang' via MicroProfile <microp...@googlegroups.com> wrote:

+1 LT!

At this point, I think what is on the table is marking MP Metrics 3.0 deprecated, with micrometer as the recommended replacement. Creating a shim from the MP Metrics 3.0 API to micrometer isn't difficult, but it would be up to MP vendors to provide this shim.

You mention not wanting to be too radical, but isn't that the point with MP? Be radical, try something out, and if it doesn't work change direction.

In my view, MP Metrics has "lost" the metrics battle, we're on our own in continuing to use it. If we continue to push it as "the way" for another year, it further harms MP and makes it even harder to change direction in the future.

As I said before, if we accept MP is about innovation, we need to equally accept that doing so means breaking things when we've innovated in a direction that needs to change.

 

 Ken, I am not objecting trying something out and change direction. However, we need time to figure out how to join the dots and tell a straight story to our end users. At the moment, I don't think we have worked out what MP can offer about Metrics if we get rid of MP Metrics. As LT pointed out, what do we do for the specs that integrates with MP Metrics. We need to work this things out for our end users. Once we worked out the path, we then do it. I don't think we should destroy the current house without a shelter is provided.

As for OpenTelemetry, even though it has some way to go, we should not ignore it because it defines a consistent view across programming languages. It is also strange we take Tracing part not Metrics part. I think we should not limit ourselves but keep open minded so that we could potentially work with either OpenTelemetry and Micrometer.

In working with the OTel Java SDK team, they acknowledge that the metrics bit of the spec is lagging behind. There are a lot of questions around how metrics should be gathered because of the tie to span contexts, etc. To re-emphasize, there is every expectation that micrometer and OTel metrics will work together. This is not an either/or conversation that requires a delay.

Thanks

Emily

On Tuesday, August 4, 2020 at 2:22:01 PM UTC+1, Ladislav Thon wrote:

út 4. 8. 2020 v 12:51 odesílatel Ken Finnigan <k...@kenfinnigan.me> napsal:

Many in the MP community say it needs to innovate, move fast, fail-fast, and break things when needed. Most of the comments so far are leaning in the other direction of, take it slow, let's wait, we need everything lined up before taking a step.

That's probably because there are two vocal minorities, and the usual silent majority.

 

We can't have it both ways. Either we need to innovate fast and accept that there are times where we need to adjust course quickly, or we may as well fold into Jakarta.

And this is just a false dilemma. Let's not do this. Let's figure our own way, but let's not throw the baby out with the bathwater.

My perspective, as someone who is merely a user of MP Metrics:

Saying "MP Metrics are gone, use Micrometer" is simply not how API evolution should work. Deprecation periods are a standard thing and we should use them. No, the period doesn't have to be multiple years. It doesn't have to be even one year. 4 to 6 months sounds perfectly reasonable to me. If I remember correctly, the possibility of multiple major MP releases per year has not been strictly ruled out, and if the whole community agrees that removing a spec needs to happen, then that's a good reason to do potentially more than one major release in a given year, in my book.

Also, the proposal doesn't address the situation of other MP specifications that integrate with MP Metrics. What are they supposed to do in MP 4.0? Keep MP Metrics integration, probably as an optional part of the spec, even though MP Metrics are no longer in the platform? (Would MP Metrics even release their 3.0?) Remove MP Metrics integration as well, leaving metrics implementation-defined? Replace the MP Metrics integration with Micrometer integration in a few weeks? (I don't think so.)

The deprecation period I suggested above would give the necessary time not to just users, but also other MP specifications.

LT

P.S.: could someone expand on "SRE requirements"? I've heard this argument mentioned several times in this discussion, but it's very opaque to me. What is it that Micrometer does and MP Metrics doesn't? The only real difference I've been able to figure out is collecting metrics. MP Metrics specify a pull model biased towards Prometheus, while other metric aggregation platforms may want a push model and/or different data granularity. But surely there's more?

There are a few things re: SRE requirements: 

As Ken said, one is a pattern/recommendation for naming metrics that encourages overlapping names, with better use of tags and labels to be able to slice/dice/analyze different subsets of data. There is a definite line here around what is too specific and what is too general, but the annotations (in particular) encourage some bad practices, even if they are cool.

Micrometer has a composition and configuration model that supports separation of concerns. Libraries can ship their own instrumentation with MeterBinders that can use distribution summaries (histograms) or timers in the lightest possible way. Config can be added later (for a particular registry/exporter, or for metrics with a particular name e.g.) to enable calculation of histogram buckets or percentiles in some cases, or to completely suppress use of the metric altogether. This allows rich declaration of metrics (process, application, library), while then allowing selective export: suppress jvm metrics going to stackdriver, only send pre-calculated histograms to datadog, e.g. That is in addition to general differences in aggregation behavior between registry types (prometheus wants raw data, jmx requires pre-calculation because it has no capability to construct or query values later), etc.

BTW as much as I don't like the MP Metrics annotations (why is there so much attributes I need to fill?), I'd certainly prefer them over the Micrometer plain Java API.

Once you start using late-binding labels (e.g. label that include the result of operations), you start to prefer the programmatic API over annotations. The OTel Metrics Java API is also programmatic and not annotation based for similar reasons, and, IMO, it is semantically worse! But, you know, opinions. =)

Some of the patterns micrometer uses to construct metrics have been added to the MP Metrics 3.0 spec to facilitate a shim between the two systems, but some of the mapping requirements are less than ideal (creation of extra objects, etc.)

 

--
You received this message because you are subscribed to a topic in the Google Groups "MicroProfile" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/mMvGzLq0Zkc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/5a76bceb-5c8f-4fff-8931-a99fe96b5ce5o%40googlegroups.com.

--

An eye for eye only ends up making the whole world blind.  ~Mahatma Gandhi

[Don Bourne]

We had a good discussion in the MP Metrics community call this morning about this.  Ken and Erin joined the call this morning, and as usual Jan, David Chan and I were there.

We discussed a bunch of ideas, but I think there was some agreement around this approach:

- GA the MP Metrics 3.0, as planned

- include MP Metrics 3.0 in the MP 4.0 platform, as planned

- include a statement indicating that for MP Metrics we are seriously looking at using Micrometer for future releases.  This could be called a deprecation statement or a statement of intent or ...?  We discussed that it's a bit weird to have a deprecation statement without an alternative, but it may still fit.

@ LT, your point about how the cross-spec dependencies would cope, as well as what would happen if a user switched immediately to micrometer (resulting in a server without the same expected base set of metrics) were a couple of the reasons why I think it's not simple to just switch overnight.

I'm not sure anyone is ready to call this a plan yet -- people are still absorbing and thinking of the possible consequences of different paths.

Regards,

Don

On Monday, August 3, 2020 at 10:39:34 AM UTC-4, Ken Finnigan wrote:

[Werner Keil]

All,

It almost sounds a bit like raising the white flag or (something I kept mentioning in several threads even though some thought they needed to bash me for it) that certain features may not have the same resources and momentum as others, both inside the Eclipse, Jakarta EE and MP community and outside it.

I wasn't aware, the unit in MP Metrics was mandatory or non-null or e.g. could not be left blank, is that the case now?

If not, there seems little difference here. The naming baseUnit sounds rather weird, because base units are usually some units like METRE or KILOGRAM, but if it suggests some kind of transformation or conversion (e.g. m to km or byte to kilobyte or Megabyte) then I would say it makes some sense calling it that way.

It is certainly better than Dropwizard because it never had a clear notion of units not even optional, the only way was "abusing" some kind of tag or label.

Especially with the whole WG being formed and compatibility claims or logos I wonder how users or members would react if they have a "MP compatible" product and then suddenly there is no MP Metrics compatibility while there is no noticeable TCK except some unit tests.

The micrometer.io site claims it's "vendor-neutral" but the only sponsor at the moment seems to be VMware, do vendors involved here intend to change that? At the moment it all seems very much at the mercy of VMWare and it is quite clear, Dell wants to sell it as soon as they get a good offer. who knows, what a new owner does or where the priorities lie?

We saw this happen with most of the Netflix cloud solutions like Hystrix which became de-facto standards in many stacks including Spring or Spring Boot, but at some point they may have to find a replacement there, too. 

So if the same fate for Micrometer should be avoided, I guess contributors from different companies could not hurt. Interestingly it seems Jonathan, the main committer worked at Netflix before joining Pivotal, but he left and no longer seems to contribute much.

The 120+ total contributors are a bit misleading, because only two of them really contributed multiple Ten Thousands lines of code, while most others may only have contributed a few lines or a few dozens, thus the activity looks not so different from MP for most parts.

MP including Metrics has a strong dependency on CDI everywhere, while Micrometer has no trace of it in its otherwise gigantic dependency graph: https://github.com/micrometer-metrics/micrometer/blob/master/micrometer-core/build.gradle 

Most are optional, but still seems overwhelming with everything from Spring to Guice, MongoDB or Kafka and even Hystrix just in the core.

No Failsafe or whatever MP Fault Tolerance relies on, so that could also be problematic.

The core has a depencency to AspectJ, hopefully not a mandatory runtime necessity, otherwise it could become quite heavy.

Not sure why they baked AspectJ into the core instead of a separate yet small micrometer-aop module, see all the dependencies the "core" sounds rather big.

My two or three cents, a somewhat smoother migration path trying to create e.g. CDI support or find ways to support FaultTolerance, etc. seems helpful. Otherwise there is a risk of redundancy and duplication, e.g. having to use both Guice and CDI or similar overlaps.

MP 4 should embrace Jakarta EE 8, right? Another thing to watch because although they still use the same "javax" package, Micrometer does not even use that, it's all Java EE 7 or 8 at most, not Jakarta EE 8.

Werner

[Ladislav Thon]

út 4. 8. 2020 v 16:26 odesílatel Ken Finnigan <k...@kenfinnigan.me> napsal:

As mentioned in one of my follow up responses, I'm good with marking MP Metrics 3.0 deprecated in MP 4.0, if we also say "use Micrometer" for metrics with MP 4.0 platform.

Continuing the "we might as well fold into Jakarta" exaggeration / logical fallacy, why don't we just replace the entire MP Platform with "use Spring"? :-)

I'm quite wary of promoting a particular implementation as a standard. We did that with OpenTracing (and that was just an API, not an implementation), and now it's gone. As mentioned in this thread (or some other recent thread about MP Metrics), we essentially did that with Dropwizard metrics, and now we're attempting to get rid of it. I can only ask how long OpenTelemetry or Micrometer are going to stay.

At the same time, I of course understand that this is a turbulent area and changes are inevitable. So if Micrometer is the de facto standard today, sure, let's promote it, but we should also strive for at least 2 things on top:

1. Preserve the MicroProfile programming model.

2. Preserve user's knowledge investment.

Do you guys agree / already plan to do something along these lines?

In my opinion, both can largely be satisfied by creating a declarative approach for registering metrics. That's something I really liked about MP Metrics. (To Erin's concern about using inputs and outputs in metric tags/labels/whatever, I think that's quite possible to do declaratively, but is also a bit of an advanced use case.)

 

As such, for MP 4.0 any specification that uses MP Metrics would continue to do so, but future versions would change to use Micrometer instead.

With respect to "SRE Requirements", there are several aspects to it:

• Metric names containing package and class name, in addition to the metric scope prefix, cause metrics to be brittle to name changes, in terms of what's in a dashboard, but more critically they prevent an SRE from aggregating metrics across many metrics of the same type. For example, instead of each JAX-RS endpoint having a custom metric name prefix for each one, there would be a metric prefix of "http_server_requests" with tags to identify the URL, etc. This offers the ability to drill down to a specific endpoint or provide an aggregated view across all JAX-RS endpoints in an application, or even across applications.
• Not all environments use Prometheus. Only offering Prometheus as an output format means MP Metrics is only useful when Prometheus is also present. While it is certainly popular, it's not the only monitoring solution available.

These are just some of the SRE needs that MP Metrics doesn't meet, but Micrometer does. I'm sure there are others as well.

OK, right. I think the "SRE" term here is just a red herring. This is simply about MP Metrics catering for developers and mostly ignoring operational concerns. I think we talked about that already, and I don't think it's exclusive to Metrics. But I certainly agree. I myself have almost zero experience with running software in production, and it's been a long time, so naturally I look at everything through the prism of a developer. I don't know what I can do to change that :-)

LT

 

Ken

On Tue, Aug 4, 2020 at 9:22 AM Ladislav Thon <lad...@gmail.com> wrote:

út 4. 8. 2020 v 12:51 odesílatel Ken Finnigan <k...@kenfinnigan.me> napsal:

Many in the MP community say it needs to innovate, move fast, fail-fast, and break things when needed. Most of the comments so far are leaning in the other direction of, take it slow, let's wait, we need everything lined up before taking a step.

That's probably because there are two vocal minorities, and the usual silent majority.

 

We can't have it both ways. Either we need to innovate fast and accept that there are times where we need to adjust course quickly, or we may as well fold into Jakarta.

And this is just a false dilemma. Let's not do this. Let's figure our own way, but let's not throw the baby out with the bathwater.

My perspective, as someone who is merely a user of MP Metrics:

Saying "MP Metrics are gone, use Micrometer" is simply not how API evolution should work. Deprecation periods are a standard thing and we should use them. No, the period doesn't have to be multiple years. It doesn't have to be even one year. 4 to 6 months sounds perfectly reasonable to me. If I remember correctly, the possibility of multiple major MP releases per year has not been strictly ruled out, and if the whole community agrees that removing a spec needs to happen, then that's a good reason to do potentially more than one major release in a given year, in my book.

Also, the proposal doesn't address the situation of other MP specifications that integrate with MP Metrics. What are they supposed to do in MP 4.0? Keep MP Metrics integration, probably as an optional part of the spec, even though MP Metrics are no longer in the platform? (Would MP Metrics even release their 3.0?) Remove MP Metrics integration as well, leaving metrics implementation-defined? Replace the MP Metrics integration with Micrometer integration in a few weeks? (I don't think so.)

The deprecation period I suggested above would give the necessary time not to just users, but also other MP specifications.

LT

P.S.: could someone expand on "SRE requirements"? I've heard this argument mentioned several times in this discussion, but it's very opaque to me. What is it that Micrometer does and MP Metrics doesn't? The only real difference I've been able to figure out is collecting metrics. MP Metrics specify a pull model biased towards Prometheus, while other metric aggregation platforms may want a push model and/or different data granularity. But surely there's more?

BTW as much as I don't like the MP Metrics annotations (why is there so much attributes I need to fill?), I'd certainly prefer them over the Micrometer plain Java API.

--
You received this message because you are subscribed to the Google Groups "MicroProfile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CALbocOkbRzih4VD3vH9xLzrAZoQPEvMx2HVA0fXbG_TO1kriNQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "MicroProfile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CAKeeVe6K57xyxjOquZ_wKS9js6xA%2Bm7QkxNJbx2qw9MTuWS0_A%40mail.gmail.com.

[Erin Schnabel]

Committers for Micrometer are already beyond Pivotal, and yes, I am in talks with them to take it to a foundation (eclipse and apache both mandate package renames, so CNCF is most likely). I have stated, and firmly believe, that OTel Metrics and Micrometer will work together. 

It is true that micrometer itself does not require a DI system, but that does not mean it is not compatible with one. It is used by multiple vendors that have different DI systems (Spring DI or CDI).  The Quarkus micrometer extension, e.g., integrates CDI and micrometer without an issue. If the MP Metrics spec wants to piece together how Micrometer should work with CDI, that’s goodness. I am happy to contribute what I have working to make this happen.

You received this message because you are subscribed to a topic in the Google Groups "MicroProfile" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/mMvGzLq0Zkc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CALbocOnNEZ2dwiBMTbKR0BOge%2BbrQMNz%2BymdENQZwSGDiTpM0w%40mail.gmail.com.

[David Blevins]

On Aug 5, 2020, at 5:28 AM, Erin Schnabel <erin.s...@gmail.com> wrote:

Committers for Micrometer are already beyond Pivotal, and yes, I am in talks with them to take it to a foundation (eclipse and apache both mandate package renames, so CNCF is most likely)

Apache does require a package rename, yes.  On the Eclipse side of things there are projects, such as Vertx, that do not use "org.eclipse"

We discussed using "io.microprofile" as our package name, but chose to use "org.eclipse.microprofile."  On the Jakarta side we went with "jakarta.*"

I think there's a preference, but not a hard rule.

-David

[Erin Schnabel]

Oh, and I should add that keeping MP annotations is a doable thing (though I am not sure it is a recommended thing). OTel Metrics is also not annotation based. Programmatic determination of tag values reduces the appeal of annotated metrics once you start doing things in anger.

[Reza Rahman]

Against my best judgment, I will chime into this seemingly somewhat overheated thread.

To me this entire subject matter basically proves the point of why you will need certain things in MicroProfile and not in Jakarta EE. In order to meet the needs of a domain that is inherently faster moving and more volatile than enterprise Java in the past decades, you need something that blurs the lines between what the broader set of people would consider an open standard vs. something that provides some level of vendor-neutrality through a set of collaborative lightweight specification artifacts.

More to the core point of what to do with MicroProfile Metrics, continuing status quo is clearly not a good idea. While it is clear to me the Kubernetes ecosystem is betting on Open Telemetry at the moment, that effort still has quite a ways to go before it is anything near a sure thing. I think adopting Micrometer for now is OK provided there is sufficient resourcing and the technology is sufficiently vendor neutral and has enough of a shelf life (rather big questions that probably no one can in good faith provide definitive answers to). The nice thing to do is to be able to reuse as much as of the API design work as possible already done in MicroProfile Metrics to easily evolve to Micrometer,  Open Telemetry or whatever else is next. Some reasonable measure of being a "future proof abstraction" could possibly be a core value proposition for MicroProfile - although I think trying to maintain 100% backwards compatibility probably isn't too pragmatic for the domain.

Hope this helps. I will admit I am very far from a subject matter on this particular topic (indeed I suspect few people probably are owing to the aforementioned velocity and volatility).

Reza Rahman
Jakarta EE Ambassador, Author, Speaker, Blogger

Please note views expressed here are my own as an individual community member and do not reflect the views of my employer.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CADJ_gaQ72S192K0O0Q3POP_OCd-m8eSf7kqmw55miC88vSXmhQ%40mail.gmail.com.

[Erin Schnabel]

Thanks for that clarification, David. That is good to know. There are a few other reasons I think CNCF is a good fit, but we’ll see how it plays out. I can make suggestions, and can help once something is decided, but it isn’t my decision to make.

:)

On Wed, Aug 5, 2020 at 1:20 PM David Blevins <dble...@tomitribe.com> wrote:

On Aug 5, 2020, at 5:28 AM, Erin Schnabel <erin.s...@gmail.com> wrote:

Committers for Micrometer are already beyond Pivotal, and yes, I am in talks with them to take it to a foundation (eclipse and apache both mandate package renames, so CNCF is most likely)

Apache does require a package rename, yes.  On the Eclipse side of things there are projects, such as Vertx, that do not use "org.eclips

We discussed using "io.microprofile" as our package name, but chose to use "org.eclipse.microprofile."  On the Jakarta side we went with "jakarta.*"

I think there's a preference, but not a hard rule.

-David

--
You received this message because you are subscribed to a topic in the Google Groups "MicroProfile" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/mMvGzLq0Zkc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/FAC1817F-EDF1-40A0-9B30-09EC7C5041DF%40tomitribe.com.

[Erin Schnabel]

Thanks for chiming in Reza. You bring up a point I have heard a few times that I have tried to address, but it seems to not be clicking, so I will try again.

Micrometer is a metrics library. It is dependency-injection-system neutral, and it is vendor neutral, and it has its own tck. The tck is for the vendors, and the vendors are registry implementations (where a registry in micrometer parlance is an exporter). So vendor neutrality is between stackdriver, datadog , prometheus, etc. DI-independence means it works as well with CDI as it does with Spring DI. 

Our traditional view of what an API needs to be, and who needs to implement it is not helping us here.

Your comment about currency and investment is 💯.. and I will suggest that we’ll get better mileage if we all work together on micrometer (and get that working with OTel metrics) than if we keep trying to carry our own.

As a technologist, the abstractions in micrometer make sense. It supports separation of concerns between dev and ops, and allows specialization of aggregation to happen where needed (in the registry). 

I am happy to sit down with anyone and walk them through how this works. I am confident that if you give it more than a cursory glance, you will come away with the same conclusion Ken and I have. Power outages aside, I am not hard to find. Just ask.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/6c34db1b-ec23-ee39-fd81-3f72d3e8bdc2%40gmail.com.

[Emily Jiang]

Hi Erin,

We talked a lot on this subject in various calls and 121s :o. To move this conversation forward, I think if you map the MP Metrics directly towards Micrometer APIs, it will help the community and our end users to understand, migration their apps towards Micrometer direction. I think in some situations as you told me, the end users don't need to use Micrometer APIs in their app and they will get the metrics for free. Only under some scenarios you will need to call the Micrometer APIs directly in your app, which might be your argument on not creating an abstract layer over and above Micrometer. I am interested in seeing this kind of mapping and when you need to use Micrometer APIs directly.

As for the debate on whether to adopt Micrometer APIs directly, a reason is needed on why MP adopts the micrometer APIs without an abstract layer while all other specs provide an abstract layer. As pointed by a few others in this thread, if MP Metrics can provide an abstract layer (either old or new), MP specs can stay where they are while the underline implementation can change. Take MicroProfile Fault Tolerance for an example, there is no impact whatsoever when Hysterix went to maintenance mode as MP FT has no direct link to any libraries but borrowed some commonsense capabilities. It is a pity that MP Metrics is too close to Dropwizard. Can we learn the mistakes and design a shim layer so that in the future we can work with Micrometer and OpenTel or whatever becoming the new mainstream?

Thanks

Emily

On Wednesday, August 5, 2020 at 11:48:32 PM UTC+1, Erin Schnabel wrote:

Thanks for chiming in Reza. You bring up a point I have heard a few times that I have tried to address, but it seems to not be clicking, so I will try again.

Micrometer is a metrics library. It is dependency-injection-system neutral, and it is vendor neutral, and it has its own tck. The tck is for the vendors, and the vendors are registry implementations (where a registry in micrometer parlance is an exporter). So vendor neutrality is between stackdriver, datadog , prometheus, etc. DI-independence means it works as well with CDI as it does with Spring DI. 

Our traditional view of what an API needs to be, and who needs to implement it is not helping us here.

Your comment about currency and investment is 💯.. and I will suggest that we’ll get better mileage if we all work together on micrometer (and get that working with OTel metrics) than if we keep trying to carry our own.

As a technologist, the abstractions in micrometer make sense. It supports separation of concerns between dev and ops, and allows specialization of aggregation to happen where needed (in the registry). 

I am happy to sit down with anyone and walk them through how this works. I am confident that if you give it more than a cursory glance, you will come away with the same conclusion Ken and I have. Power outages aside, I am not hard to find. Just ask.

Ken

To unsubscribe from this group and stop receiving emails from it, send an email to microp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CALbocOkbRzih4VD3vH9xLzrAZoQPEvMx2HVA0fXbG_TO1kriNQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "MicroProfile" group.

To unsubscribe from this group and stop receiving emails from it, send an email to microp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CAKeeVe6K57xyxjOquZ_wKS9js6xA%2Bm7QkxNJbx2qw9MTuWS0_A%40mail.gmail.com.

--
You received this message because you are subscribed to a topic in the Google Groups "MicroProfile" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/mMvGzLq0Zkc/unsubscribe.

To unsubscribe from this group and all its topics, send an email to microp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CALbocOnNEZ2dwiBMTbKR0BOge%2BbrQMNz%2BymdENQZwSGDiTpM0w%40mail.gmail.com.

--

Sent from Gmail Mobile

--

Sent from Gmail Mobile

--
You received this message because you are subscribed to the Google Groups "MicroProfile" group.

To unsubscribe from this group and stop receiving emails from it, send an email to microp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CADJ_gaQ72S192K0O0Q3POP_OCd-m8eSf7kqmw55miC88vSXmhQ%40mail.gmail.com.

--
You received this message because you are subscribed to a topic in the Google Groups "MicroProfile" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/mMvGzLq0Zkc/unsubscribe.

To unsubscribe from this group and all its topics, send an email to microp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/6c34db1b-ec23-ee39-fd81-3f72d3e8bdc2%40gmail.com.

[Jan Martiška]

On Thursday, August 6, 2020 at 1:38:01 PM UTC+2 Emily Jiang wrote:

Hi Erin,

We talked a lot on this subject in various calls and 121s :o. To move this conversation forward, I think if you map the MP Metrics directly towards Micrometer APIs, it will help the community and our end users to understand, migration their apps towards Micrometer direction. I think in some situations as you told me, the end users don't need to use Micrometer APIs in their app and they will get the metrics for free.

I feel I need to comment on this common argument. This "for free" here probably means that Micrometer *itself* (in its codebase) provides metric bindings for several frameworks (Hibernate, Kafka, MongoDB...) whereas with current MP Metrics, metric bindings for these frameworks are generally provided by the MP Metrics implementation, or the runtime that wraps it (Quarkus provides metrics for quite a few frameworks).

So I disagree with the statement that end users get something "for free" compared to MP Metrics. The only difference here is purely technical - in one case, the metrics are provided by Micrometer itself, in the other case, by the MP Metrics implementation or runtime. In neither case does the user have to do anything other than provide the relevant configuration values. There is no need to use any program APIs to get any of these metrics.

And I personally believe that our approach in this case is better, because the code providing metrics for a framework is closer to the framework itself and therefore can be much more flexible. Micrometer's approach also means that there is an artifact with a large amount of 3rd party dependencies - see https://github.com/micrometer-metrics/micrometer/blob/v1.5.3/micrometer-core/build.gradle#L13 - I believe this should be split to several modules, one for each framework, to be manageable, especially if this is to be imported to projects that require full dependency convergence.

[Erin Schnabel]

On Thu, Aug 6, 2020 at 8:27 AM Jan Martiška <wray...@gmail.com> wrote:

On Thursday, August 6, 2020 at 1:38:01 PM UTC+2 Emily Jiang wrote:

Hi Erin,

We talked a lot on this subject in various calls and 121s :o. To move this conversation forward, I think if you map the MP Metrics directly towards Micrometer APIs, it will help the community and our end users to understand, migration their apps towards Micrometer direction. I think in some situations as you told me, the end users don't need to use Micrometer APIs in their app and they will get the metrics for free.

I feel I need to comment on this common argument. This "for free" here probably means that Micrometer *itself* (in its codebase) provides metric bindings for several frameworks (Hibernate, Kafka, MongoDB...) whereas with current MP Metrics, metric bindings for these frameworks are generally provided by the MP Metrics implementation, or the runtime that wraps it (Quarkus provides metrics for quite a few frameworks).

There are a lot of binders and support for micrometer out in the ecosystem already... so no, it isn’t that micrometer itself implements them. 

So I disagree with the statement that end users get something "for free" compared to MP Metrics. The only difference here is purely technical - in one case, the metrics are provided by Micrometer itself, in the other case, by the MP Metrics implementation or runtime. In neither case does the user have to do anything other than provide the relevant configuration values. There is no need to use any program APIs to get any of these metrics.

Again, more hands-on will help. Micrometer does package some common binders, but other systems provide their own. The MeterBinder separation allows for a lot of flexibility, as does the late-binding application of config. 

And I personally believe that our approach in this case is better, because the code providing metrics for a framework is closer to the framework itself and therefore can be much more flexible. Micrometer's approach also means that there is an artifact with a large amount of 3rd party dependencies - see https://github.com/micrometer-metrics/micrometer/blob/v1.5.3/micrometer-core/build.gradle#L13 - I believe this should be split to several modules, one for each framework, to be manageable, especially if this is to be imported to projects that require full dependency convergence.

Again, I think direct experience will be more helpful than continuing the discussion in this way. It is easy to prefer what you know.

Only under some scenarios you will need to call the Micrometer APIs directly in your app, which might be your argument on not creating an abstract layer over and above Micrometer. I am interested in seeing this kind of mapping and when you need to use Micrometer APIs directly.

To Emily: I think we should consider that Micrometer already is that abstraction. Yes, it would be a choice to go all in with Micrometer, but the abstraction it uses is already more flexible than what MP Metrics has currently. And I say that having written the extension for quarkus + native integration for vert.x.

The transitive dependency argument.. you don’t have to use the binders in micrometer, and the dependencies are compile/optional. They don’t impose a restriction if alternates are preferred.

To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/cb5848b0-37be-497f-af31-8bc675e9c062n%40googlegroups.com.

[Werner Keil]

CNCF for what, Micrometer?

I guess it might give it a more vendor-neutral foundation than right now where VMWare is the only sponsor and it could meet the fate of several Netflix cloud libraries that are now more or less abandoned and inactive.

Reza has a point, it's quite possible other MP attempts follow a similar fate e.g. Service Mesh (all the repos have not seen any activity for more than a year) or OpenTracing (since that's pretty dead and now merged into OpenTelemetry) 

It raises a valid question, if MicroProfile could and should ever be used as a foundation to Jakarta EE, where stability and reliability are much more important? 

CNCF has a maturity level, see: https://www.cncf.io/projects/ and IMO if any of the reasonably mature MP specs (probably just Config right now, given this discussion about this one that came soon after) should ever be usable to Jakarta EE or similar enterprise environments with a requirement for consistency and compatibility, then the same kind of maturity level should also be applied to MP.

Making clear, that most of the others that are still incubating may change or be archived, similar to what Eclipse did for a long time or CNCF also picked up: https://www.cncf.io/archived-projects/

Werner 

[Erin Schnabel]

On Thu, Aug 6, 2020 at 3:45 PM Werner Keil <werne...@gmail.com> wrote:

CNCF for what, Micrometer?

Yes.

I guess it might give it a more vendor-neutral foundation than right now where VMWare is the only sponsor and it could meet the fate of several Netflix cloud libraries that are now more or less abandoned and inactive.

Exactly.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/f6d1371f-81ef-4052-b241-1bde0eefb2aan%40googlegroups.com.

[Steve Millidge]

This discussion goes to heart of the purpose of MicroProfile. From the draft charter "MicroProfile is an open source, vendor neutral, collaborative, initiative that creates technical specifications, APIs, and technology compatibility kits (TCKs) "

Dropping MP Metrics and stating that we are now adopting Micrometer (even if some feel it is superior) is in principle not creating specifications, APIs and TCKs . If people believe that MP Metrics is heading in the wrong direction then it should evolve in a series of releases to be something that is better, adopting the best ideas from Micrometer. Just dumping the api and leaving users with no alternative but to rewrite their applications using a non-MP api is imho poor governance. With question marks over the long term future of MP open tracing and previous suggestions that MP Fault Tolerance should be dropped from the core; more and more I feel MP being torn towards being just another framework or even worse a collection of random apis rather than a body that wants to create vendor neutral APIs and specifications that form a coherent standard platform.

Back to Ken's original suggestions, if people are determined to head down this route then option 2 should be chosen.

Steve

[Werner Keil]

So to avoid everyone digging into this major thread, option 2 would be: 

MP platform spec doesn’t indicate anything about metrics, but in the future, we create a new spec that focuses on monitoring output. Note, this option might better fit with OpenTelemetry metrics from a Java ecosystem perspective.  

Either option is a decision between devil and belzebub, and means a huge blow to the reliability of the MP platform. Steve already mentioned, that Fault Tolerance because it is closely tied to just one API could be next, and who knows, if enough conclude that Spring, Apache Commons Config, Typesafe Config or Helidon Config (which is <> the MP implementation by Helidon) are superior or at least much more popular than MP Config, that could also be dropped. What would remain?

This shows MP is quite volatile and despite good intentions with API, TCK, etc. I would be very afraid to see any of it as a foundation to Jakarta EE. At least not without a strong dedication of all MP stakeholders and contributors to create a "mature" or "graduated" level similar to CNCF projects like Prometheus or Kubernetes. With a guarantee that these won't suddenly be dropped as well.

Otherwise Otavio would have been perfectly right suggesting a "Jakarta Config" instead as a more stable and reliable alternative, just to give an example.

Werner

[Edwin Derks]

Hi all,

I have been reading this thread and thinking about it. A lot actually... I would like to share my thoughts, putting on my end-user developer hat. Summarizing, maybe this all might not be as bad and scary as it looks. Please hear me out ;).

Naturally, there is competition in the software community regarding a certain feature, in this case exposing Metrics,  that might be more popular than others. The fact that Micrometer has a lot of traction and popularity, doesn't in my opinion not necessarily mean that MP Metrics is beaten.

Even if Micrometer is a more popular solution for exposing metrics, let's not forget that MP Metrics is part of a platform: namely Eclipse Microprofile. It doesn't stand on its own! So even if we conclude that MP Metrics has no future anymore in Microprofile, does that mean that Microprofile as a platform doesn't need to provide a feature to expose metrics anymore? Of course not! Therefore, the description of 'option 2' is very accurate and imho very likely the evolutionary solution for this problem.

What I need to make clear is that I, as a developer that loves Jakarta EE and MicroProfile so much, loves these frameworks because of their simplicity as a one-size-fits-almost-any-project-i-want-to-do platform. When starting a project, I only need to know how Jakarta EE and MicroProfile do stuff relating to anything I need as a developer, before even deciding on the implementation. Even if I need to put my app in a cloud, it all fits. This is the strength, core and goal of these platforms, right?

I'm still not sure if there is a clear vision or roadmap available that defines the future of Jakarta EE and MP, but this should be a core value in it, no? I mean a vision where we are not only thinking about new features for Jakarta EE 10 and Microprofile 4.1. Of course, these improve the frameworks, but shouldn't change it's core.

So summarizing, for me as a developer it's important that exposing metrics is possible with MP or Jakarta EE, and I want to learn how it's done there, in the platform that is consistent. That prevents me from having to learn how it's done in frameworks that come and go in popularity. And there's the value of keeping such features, whether it's metrics, config or whatever in the platform. When I start a project and I want to use Jakarta EE, MicroProfile and additionally the current most popular framework for metrics, that would be the first step in losing the core value of Jakarta EE and MicroProfile.

Thanks,

Edwin

You received this message because you are subscribed to the Google Groups "MicroProfile" group.

To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/6109cde0-c172-469e-b5c7-9955516af08cn%40googlegroups.com.

[Jonathan Gallimore]

I agree. I don't particularly have any objections to looking at Micrometer, and understand that MicroProfile wants to evolve rapidly. However, consumers who have adopted MP metrics who now find it dropped from the platform in favor of something else could well be frustrated. Perhaps there's some way the platform can ease the transition (maybe include both, or allow Metrics to optionally continue to be included. I haven't dug in to the specifics, so this may be a daft suggestion, but maybe some sort of compatibility layer is an option?).

If Metrics is being dropped, I prefer option 2.

Jon

You received this message because you are subscribed to the Google Groups "MicroProfile" group.

To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/604674a5-380e-4a41-8b9d-b26342db6e67n%40googlegroups.com.

--

Jonathan Gallimore

http://www.tomitribe.com

[Ruslan Synytsky]

On Fri, 14 Aug 2020 at 10:52, Jonathan Gallimore <jgall...@tomitribe.com> wrote:

I agree. I don't particularly have any objections to looking at Micrometer, and understand that MicroProfile wants to evolve rapidly. However, consumers who have adopted MP metrics who now find it dropped from the platform in favor of something else could well be frustrated. Perhaps there's some way the platform can ease the transition (maybe include both, or allow Metrics to optionally continue to be included. I haven't dug in to the specifics, so this may be a daft suggestion, but maybe some sort of compatibility layer is an option?).

+1

 

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CAJaBFGYiAAzowY125-k1nqFUN%3DPDnBayAVOa7KoWqY5Y0tpwSQ%40mail.gmail.com.

--

Ruslan Synytsky
CEO @ Jelastic Multi-Cloud PaaS

[Guillermo González de Agüero]

+1. One of the big benefits I see from Jakarta/MP is the abstraction layer they provide over the underlying implementation.

Thorntail was using Hystrixs as the base for its Fault Tolerance implementation when NetFlix decided to abandon it. The implementation was rewritten but users didn't have to worry about it.

That was IMO the first real world demonstration of the MP advantage.

Now as a user, I wouldn't like to care about MicroMeter or Dropwizard or wathever.

If I need to change dependencies everytime some technolohy is overlaped by something else, then what am I getting for MP? 

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CAJaBFGYiAAzowY125-k1nqFUN%3DPDnBayAVOa7KoWqY5Y0tpwSQ%40mail.gmail.com.

[Edwin Derks]

+1 Guillermo. I hope more people see it this way and we can together as a community evolve Jakarta EE and MicroProfile with this vision in mind.

Edwin

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CAG1ZpUb_CCpqa%3DLdiV-BJ4izKJ6gXuOdpBJ5EvQ5zyaxiCX00w%40mail.gmail.com.

[Erik Mattheis]

I think the real issue here is that MP metrics basically adopted an API (from DropWizard) that is no longer favored by a large part of the community. I agree that we need to be careful about compatibility for existing usage of MP, but the larger issue is the fact that many new users aren’t best served by MP metrics as it stands.

As a developer, the big issue I see with MP overall is that the API doesn't cover all the parts of a platform that I might use. MP compatible platforms inevitably have to include non-standard features to remain competitive in the marketplace, and this dilutes the value of MP as a portability guarantee. Indeed, I’ve been watching the MP community struggle with this issue since its inception and I think it’s done a great job overall, but I also think we’re reaching an inflection point as Jakarta EE gets back on track.

I think we need to focus more on how MP can normalize the API landscape between competing implementations and less on trying to develop novel APIs that provide abstractions for common functionality. As technologies mature, de facto standards tend to emerge and can eventually be made into formal Jakarta EE pieces. In the meantime, MP should focus on how existing Jakarta EE standards (e.g. CDI, JAX-RS) can coexist with emerging trends (e.g - GraphQL, Micrometer).

The biggest benefit of the MP community that I see right now is not a set of APIs that provide perfect portability, it’s a community of implementors that are collaborating on common problems. Industry trends are leading us as application developers to build lighter solutions on shorter, more iterative timelines. I am no longer worried about being able to drop my existing catalog of WAR files (that took years to develop) into a compliant runtime (that the company just licensed). I’m much more interested in being able to easily fork my codebase and target a new runtime or multiple runtimes with a common build system. Having some assurance that current popular libraries and frameworks are well supported by more than one implementation is more important to me than breaking compatibility with last year’s favorites.

-- 

Erik

Jonathan Gallimore

http://www.tomitribe.com

[Guillermo González de Agüero]

Hi Erik,

El vie., 14 ago. 2020 18:01, Erik Mattheis <erikma...@gmail.com> escribió:

I think the real issue here is that MP metrics basically adopted an API (from DropWizard) that is no longer favored by a large part of the community. I agree that we need to be careful about compatibility for existing usage of MP, but the larger issue is the fact that many new users aren’t best served by MP metrics as it stands.

The problem lies in the fact that Metrics was based on a de facto standard that has now faded away, OpenTracing is also out of scene... There even was a time when Istio was about to supersede MP FT.

MP Metrics was a good idea at the moment and has served us well to date. If it needs to be completely redone, I just hope we don't end with a MP MicroMeter specific spec that's thrown away again in a few years.

And I hope the same for the OpenTracing replacement, that we can get something that works OOTB with wathever is fancy at the moment, without caring much about upgrades.  

As a developer, the big issue I see with MP overall is that the API doesn't cover all the parts of a platform that I might use. MP compatible platforms inevitably have to include non-standard features to remain competitive in the marketplace, and this dilutes the value of MP as a portability guarantee. Indeed, I’ve been watching the MP community struggle with this issue since its inception and I think it’s done a great job overall, but I also think we’re reaching an inflection point as Jakarta EE gets back on track.

I think we need to focus more on how MP can normalize the API landscape between competing implementations and less on trying to develop novel APIs that provide abstractions for common functionality. As technologies mature, de facto standards tend to emerge and can eventually be made into formal Jakarta EE pieces. In the meantime, MP should focus on how existing Jakarta EE standards (e.g. CDI, JAX-RS) can coexist with emerging trends (e.g - GraphQL, Micrometer).

The biggest benefit of the MP community that I see right now is not a set of APIs that provide perfect portability, it’s a community of implementors that are collaborating on common problems. Industry trends are leading us as application developers to build lighter solutions on shorter, more iterative timelines. I am no longer worried about being able to drop my existing catalog of WAR files (that took years to develop) into a compliant runtime (that the company just licensed). I’m much more interested in being able to easily fork my codebase and target a new runtime or multiple runtimes with a common build system. Having some assurance that current popular libraries and frameworks are well supported by more than one implementation is more important to me than breaking compatibility with last year’s favorites.

I'm also not concerned about changing runtimes. I'm worried that I can't upgrade to a new version of the runtime because some API is now out of fashion.

--
You received this message because you are subscribed to the Google Groups "MicroProfile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/42CADB0C-C22B-4B59-B41A-B42673208093%40gmail.com.

[Erik Mattheis]

On Aug 14, 2020, at 12:50 PM, Guillermo González de Agüero <z06.gu...@gmail.com> wrote:

El vie., 14 ago. 2020 18:01, Erik Mattheis <erikma...@gmail.com> escribió:

I think the real issue here is that MP metrics basically adopted an API (from DropWizard) that is no longer favored by a large part of the community. I agree that we need to be careful about compatibility for existing usage of MP, but the larger issue is the fact that many new users aren’t best served by MP metrics as it stands.

The problem lies in the fact that Metrics was based on a de facto standard that has now faded away, OpenTracing is also out of scene... There even was a time when Istio was about to supersede MP FT.

Precisely. We would have been better off, in my opinion, just agreeing to use DropWizard Metrics across supported runtimes. Perhaps with some CDI guarantees around its availability.

MP Metrics was a good idea at the moment and has served us well to date. If it needs to be completely redone, I just hope we don't end with a MP MicroMeter specific spec that's thrown away again in a few years.

The key is to not spend so much time reinventing things along the way. If the community converges on something, we adopt it into the platform. As things become outmoded, we let them go. If things stick around long enough, then they deserve to be formalized in Jakarta EE.

And I hope the same for the OpenTracing replacement, that we can get something that works OOTB with wathever is fancy at the moment, without caring much about upgrades.  

As a developer, the big issue I see with MP overall is that the API doesn't cover all the parts of a platform that I might use. MP compatible platforms inevitably have to include non-standard features to remain competitive in the marketplace, and this dilutes the value of MP as a portability guarantee. Indeed, I’ve been watching the MP community struggle with this issue since its inception and I think it’s done a great job overall, but I also think we’re reaching an inflection point as Jakarta EE gets back on track.

I think we need to focus more on how MP can normalize the API landscape between competing implementations and less on trying to develop novel APIs that provide abstractions for common functionality. As technologies mature, de facto standards tend to emerge and can eventually be made into formal Jakarta EE pieces. In the meantime, MP should focus on how existing Jakarta EE standards (e.g. CDI, JAX-RS) can coexist with emerging trends (e.g - GraphQL, Micrometer).

The biggest benefit of the MP community that I see right now is not a set of APIs that provide perfect portability, it’s a community of implementors that are collaborating on common problems. Industry trends are leading us as application developers to build lighter solutions on shorter, more iterative timelines. I am no longer worried about being able to drop my existing catalog of WAR files (that took years to develop) into a compliant runtime (that the company just licensed). I’m much more interested in being able to easily fork my codebase and target a new runtime or multiple runtimes with a common build system. Having some assurance that current popular libraries and frameworks are well supported by more than one implementation is more important to me than breaking compatibility with last year’s favorites.

I'm also not concerned about changing runtimes. I'm worried that I can't upgrade to a new version of the runtime because some API is now out of fashion.

This scenario is inevitable in the fullness of time, the crux of the issue is the time window we aim to provide compatibility for. Too long and we have a backwards compatibility nightmare. Too short and we provide no safety to developers. I think as Jakarta EE reestablishes itself after the big Oracle to Eclipse migration it will be in a good position to offer longer term guarantees by achieving consensus across multiple vendors. That takes a lot of work and time, though. MP should not get bogged down in the same debates. The very worst case scenario I see is is a community de-facto standard (Micrometer, for argument’s sake) that MP tries to abstract away (MP metrics 3) and Jakarta EE eventually provides a formal standard for (Jakarta metrics). If I’ve already adopted Micrometer in my app and a robust standard is on the horizon, what value does MP metrics provide in the meantime?

When MP is breaking new ground, there’s no question it’s a great way to incubate consensus and provide vendor neutral solutions rapidly, but we have to be prepared to abandon things when alternatives emerge with more momentum.

— 

Erik

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CAG1ZpUara5g6_bOvG%2BT4spPoMs8nP693Zh4QHDNpuVQ-M%2B97Jw%40mail.gmail.com.

[Werner Keil]

+1

Although we tried to improve a few things Dropwizard never offered a CDI wrapper similar to what Spring Actuators do probably could have been better, and just like Spring Framework/Boot in recent version could have allowed a more painless transition to Micrometer than the current situation.

It is a bit similar with Fault Tolerance, while Jonathan Halterman, the only significant contributor to Failsafe contributed ideas here in the mailing list as far as I recall, he did not contribute to the actual code of MP Fault Tolerance and more oddly, Failsafe never implemented its API either. 

One would have thought something like that may happen similar to e.g. Hibernate and JPA.

The way it went MP Fault Tolerance competes with at least Failsafe and Resilience4J both actively developed (Resilience4J looks more healthy because 11 developers with 4-digit lines of code committed, while Jonathan is the only one who committed more than 5-10 LOC to Failsafe) and of course a lot of systems also still use Hystrix although it is not actively developed any more.

By "reinventing" Failsafe's "wheel" for most parts without that adopting it, Fault-Tolerance may well see a similar fate and then what's left?

MP Config is mostly used by some of these other parts, and even with some CDI wrapper around Micrometer that won't make it use MP Config, so it's also obliterated by this move. 

Erik made a very good point: "If things stick around long enough, then they deserve to be formalized in Jakarta EE."

Nothing is a stronger argument for MP more or less an "incubator" (which it also is still according to Eclipse Foundation ;-) for things that may be formalized or graduate in Jakarta EE if they stick around long enough than this whole discussion.

I would not see reliable future for the idea of a downstream use of even MP Config like it's used by e.g. Metrics right now, because who knows what comes next and how long it's going to be around and that is not a reliable foundation for Jakarta EE.

This has made the ideas by Otavio or Sebastian over the past few months for the relation between Jakarta EE and MP a lot more plausible.

Werner

[Emily Jiang]

As I pointed out earlier on this thread, I think it is best for MP Metrics to spec the abstraction layer and leave the implementation details alone, as a few others already pointed out as an end user, he/she does not care which lib is used as long as the right metrics are emitted out. Therefore, MP Metrics implementators can choose which libraries they want to bring in. This is exactly what MP Fault Tolerence did. MP FT does not limit to Failsafe/Hysterix or bring your own. In this case, MP specs protect the end users from migrating headaches due to lib popularity changes.

Thanks

Emily

[Werner Keil]

That's more or less what Spring has been doing with Dropwizard Metrics and now Micrometer ;-)

Werner

[Erin Schnabel]

I don’t think there is huge value in another abstraction layer at this point... there are already two others, going between adds overhead/wrapper objects

You received this message because you are subscribed to a topic in the Google Groups "MicroProfile" group.


To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/mMvGzLq0Zkc/unsubscribe.


To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.


To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/2a286151-09b8-440a-b5fe-5042dcb7b295n%40googlegroups.com.

[Erin Schnabel]

Perhaps to some extent. Micrometer is already an abstraction, yes. But as we have discussed, it is an abstraction over what MP considers exporters (and puts the math in the right place to satisfy that). The traditional MP abstraction (across runtime vendors) makes less sense here. Why do I need Small Rye and Liberty and Payara and .... all implementing counters and prom exporters? It doesn’t make sense, IMO. Micrometer already is portable across DI frameworks, and is config-injection-ready (as I have shown with Quarkus extension). Merit for a MicroProfile API in this case just isn’t there (same for logging). CDI injection guidelines? Sure. I can totally see that.

You received this message because you are subscribed to a topic in the Google Groups "MicroProfile" group.


To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/mMvGzLq0Zkc/unsubscribe.


To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.


To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/6ef07a49-936a-483a-8e24-a508063febcbn%40googlegroups.com.

[Emily Jiang]

Erin,

The point is that how we can make MP hides the implementation details so that it can prevent end users from too many migration headaches. If the adopted lib does not surface, MP can freely change the libs without upsetting end users. MP Metrics can adopt Micrometer. Maybe in a few years time, another metrics framework e.g. OpenTelemetry is the mainstream. MP Metrics can then adopt OTel. However, the surfaced MP APIs can stay the same. I think quite a few people on this thread suggests this as well. As I mentioned a few times MP Fault Tolerance does not surface neither Hysterix nor Failsafe, therefore, it stays intact even though Hysterix goes to maintence mode.

Thanks

Emily

[Steve Millidge]

I agree with Emily here the whole purpose of MicroProfile is to define apis, specifications and TCKs. Personally I don't see any point in adopting a 3rd party api as is as part of the platform. If people feel that the concepts of Micrometer are more robust shouldn't we be evolving MP metrics towards those concepts while enabling some vendors to ease that migration for their users.

[Erin]

If you haven’t written the bridge between metrics systems, then this won’t resonate. 

Most uses of micrometer (to measure http throughput or JDBC behavior or.. pick other thing), aren’t in the application code at all. Application migration is not a concern in that case (And this is definitely the least invasive model to follow).

We left off at an “MP Annotations/API shim” for existing users, and we will need that regardless of any other choice. Every conceptual mismatch and little counter for a counter has a cost.

On Tue, Aug 18, 2020 at 2:20 PM Steve Millidge <l33t...@gmail.com> wrote:

I agree with Emily here the whole purpose of MicroProfile is to define apis, specifications and TCKs. Personally I don't see any point in adopting a 3rd party api as is as part of the platform. If people feel that the concepts of Micrometer are more robust shouldn't we be evolving MP metrics towards those concepts while enabling some vendors to ease that migration for their users.

--

You received this message because you are subscribed to a topic in the Google Groups "MicroProfile" group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/mMvGzLq0Zkc/unsubscribe.

To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/c72c4075-1bc2-470d-85d8-17ecd84013a2o%40googlegroups.com.

[Werner Keil]

That's precisely what I wanted to say. 

Since the big players in metrics like Micrometer or Dropwizard (the latter despite having influenced MP quite a bit) won't implement the API the only way to support them would be via some bridge, a bit like the Portlet-Bridge for JSF. 

Given Failsafe despite a lot of influence also doesn't implement the MP API, it is quite similar for Fault Tolerance.

Werner

[Amelia Eiras]

Hola MicroProfilers, 

I have taken the liberty to first pin-in this topic in the forum. 

Though, I have not fully catched up on this thread +50 exchanges, it is still a red flag to me the potential that the MicroProfile project might be perceived or viewed as "pushing" the Micrometer project contributors to join any foundation for the sake of our own usage. 

How would we feel if that was MicroProfile and a User wanted to push its will onto us to "comply" to fit their specific requirements?

We voted to be fully independent from our users' requests not too long ago in 2020 while working on the MPWG. 

I believe that such a vote could be applied to also keeping independence from how we acquired & choose to  use/collaborate on amazing innovations in the Community. 

Bias to the core-- it makes no sense to drop stuff and expect the MicroProfile users to acquire / migrate to the next thing. 

The timing is off as well. This year has been extra hard for the MP project and its community. We have been brought to a half on MP regular releases, with only 1 this year alone and we are going to be extra lucky if we are able to get MP. 4.0 by early Q4 this year. 

Until the MPWG is active, no releases are allowed.  

NOW, most of us are working tirelessly on that MPWG and like me,  likely missing much of the conversation on this thread, so please bear with not having the bandwidth to read each email yet.   May I ask for some consideration from everyone who is currently adding value and input on this thread, please? 

Microprofile can buy itself as much time as it needs to move forward on technical matters but must do so without rushing.  I feel rushed b/c I am not fully catched up on this topic. 

As with everything we do, the MicroProfile Committers will be required to formally vote after there is a clear understanding of what's what. 

That is after the topic has been discussed carefully via writing, brought in to the recorded Community calls, all  AFTER this thread has exhausted all its comments & concerns. 

Please keep safe, sane and awesome, 

Amelia Eiras 

twitter.com/ameliaeiras

Tribe: http://tomitribe.com      https://tribestream.io   

OSS:  http://microprofile.io     https://jakarta.ee

Ken

--

You received this message because you are subscribed to the Google Groups "MicroProfile" group.

To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CALbocOkbRzih4VD3vH9xLzrAZoQPEvMx2HVA0fXbG_TO1kriNQ%40mail.gmail.com.

--

You received this message because you are subscribed to the Google Groups "MicroProfile" group.

To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CAKeeVe6K57xyxjOquZ_wKS9js6xA%2Bm7QkxNJbx2qw9MTuWS0_A%40mail.gmail.com.

--
You received this message because you are subscribed to a topic in the Google Groups "MicroProfile" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/mMvGzLq0Zkc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CALbocOnNEZ2dwiBMTbKR0BOge%2BbrQMNz%2BymdENQZwSGDiTpM0w%40mail.gmail.com.

--

Sent from Gmail Mobile

--

You received this message because you are subscribed to the Google Groups "MicroProfile" group.

To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CADJ_gaR%3D7otYhV_DEnCS-xU0TGO4q%2B0d8gW0a7Sp74PXWMozmw%40mail.gmail.com.

[Werner Keil]

There should probably be a broader vote and more input here. A vote just by a small number of people in their "ivory tower" may not really make sense.

I recall, there were some surveys, but probably not as detailed as to find out, who is using what for metrics, e.g. Dropwizard, Micrometer or something else?

If a vast majority prefers to use Micrometer over MP metrics or existing implementations, it makes no sense to beat a dying horse just to have its own breed.

JCenter gives a good overview of downloaded packages:  https://bintray.com/beta/#/bintray/jcenter?tab=packages 

It shows only the last 30 days, but Spring Boot and Spring Framework were downloaded 25 Million times in the last 30 days !!!

If you search for "micrometer-core" you'll get 3 artifacts, among them the JAR https://bintray.com/beta/#/spring/jars/micrometer-core?tab=overview downloaded 200,000 times in the last 30 days. In total 5,404,081 times. 

"microprofile-metrics" gives 27 artifacts, among them " https://bintray.com/beta/#/bintray/jcenter/io.helidon.microprofile.metrics:helidon-microprofile-metrics?tab=overview  had the best demand with 681 downloads in the last 30 days, 4555 total. The others don't have much more demand:   wildfly-microprofile-metrics-smallrye  143 over the last 30 days, 5276 downloads total. The others are negligible, e.g.  https://bintray.com/beta/#/bintray/jcenter/io.openliberty.features:com.ibm.websphere.appserver.org.eclipse.microprofile.metrics-1.0?tab=overview just got 164 downloads in total.

Sounds like 5 1/2 Million reasons for either dropping MP Metrics or at least creating a bridge, like https://bintray.com/beta/#/bintray/jcenter/io.vertx:vertx-micrometer-metrics?tab=overview, The vert.x Micrometer bridge was downloaded 14k times in the last 30 days, 163,271 downloads total.

Werner

[Erin Schnabel]

Just to reassure Amelia, Ken and I are talking with the Micrometer team about it going to CNCF, but there are other reasons to pursue that (beyond this conversation). I mention it here only for those that care strongly about using or supporting technology that is supported or governed by a foundation. 

 

Otherwise, what Werner said.

;) 

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/598e23dd-bd21-43c3-b5a8-408197ad1849n%40googlegroups.com.

[Erik Mattheis]

> On Aug 18, 2020, at 2:20 PM, Steve Millidge <l33t...@gmail.com> wrote:
>
> I agree with Emily here the whole purpose of MicroProfile is to define apis, specifications and TCKs.

I believe the greater value of MicroProfile is to define a platform comprised of specific APIs and guidance around how they interoperate. Indeed, the backbone of MP for me (and really all MP users) is the suite of Jakarta EE APIs that are not defined by MicroProfile (e.g. CDI, JAX-RS, JSON-P, etc.) and a big challenge of MP right now is how to define and validate the usage of these APIs in compliant platforms. Where there is a void, or lack of consensus, MP can play a useful role by stepping in and defining new APIs, but I would personally like to see those initiatives serve as incubators for Jakarta EE. As such, any API defined solely by MicroProfile should be seen as having a limited lifespan and will eventually be subsumed by Jakarta EE or abandoned.

> Personally I don't see any point in adopting a 3rd party api as is as part of the platform. If people feel that the concepts of Micrometer are more robust shouldn't we be evolving MP metrics towards those concepts while enabling some vendors to ease that migration for their users.

Assuming it finds it’s way to an open consortium and finds broader vendor support, how Is Micrometer any more a 3rd party than CDI or JAX-RS at this point? I certainly support aligning with Jakarta EE, but I see MP as a great way to glue emerging de facto standards in the broader Jakarta EE ecosystem. This was a big missing piece of Java EE in the past because as developers we could not count on non-standard APIs being present across otherwise compatible platforms, so we ended up with “portable” apps that barely used any of the core Java EE and packaged hundreds of libs into each portable artifact.

It was exciting for me to see MicroProfile emerge at a time when Java EE was faltering and it seemed like vendors might all just go their separate ways. To the extent that MP encouraged us to take a look at what parts of Java EE were worth keeping around an not reinventing, while embracing new technologies that Java EE simply couldn’t move fast enough to address, it was great. We’re at a point now where there are several solid offerings around MicroProfile, but it feels like they are innovating around it rather than with it. I know a lot of that has to do with the organizational challenges that are ongoing and I have great respect for those of you who are fighting hard to bring that to fruition, but we need to make sure that we’re not just trying to create Jakarta EE lite. As a developer, I care much more about APIs being consistently available across platforms than whether or not they were specified by a specific standards body.

—
Erik

[Ken Finnigan]

Hey All,

Apologies for the lengthy post but this turned into a multi-faceted discussion, which is great but requires more space for me to cover.

Werner, thanks for the download stats. It provides an interesting insight into usage and is definitely important input to this discussion.

Firstly, I wanted to discuss what is actionable in the short term and what impacts to MP 4.0 there are.

From the healthy discussion over the last few weeks, I can see we're not going to reach a resolution in time to make significant changes to MP 4.0.

For that reason, I'd like to suggest that MP Metrics 3.0 remains in the MP 4.0 platform,

with a minor addition to the spec document outlining the intent to revisit MP Metrics as a specification, and indicate we're considering the likes of Micrometer.

I've raised https://github.com/eclipse/microprofile-metrics/issues/606, to have that included in MP Metrics 3.0.

Don/Jan, could one of you draft something to add to the spec? Thanks

I couldn't find if it was actually documented anywhere, though I'm sure I've heard it mentioned before, is that MicroProfile should be prepared to "fail fast". One of the principles in the proposed Working Group charter is "Experiment and Innovate", but that doesn't hold much weight if we're also not prepared to move on from those experiments if they've outlived their usefulness.

When it was created MP Metrics took a difficult subject and created APIs for users of all runtimes to utilize, which was great. However, I firmly believe there is a better and more widely used alternative in Micrometer that is available to us.

To Steve and Emily's point that MicroProfile's whole purpose is to create APIs, specifications, and TCKs. I wholeheartedly agree that it was the defined purpose, but that doesn't mean that purpose can't change, or even that what is being proposed goes against that purpose.

With MP OpenTracing, there is a single annotation for fine control of tracing behavior and two classes for integrating tracing with MP REST Client. All other APIs that developers are utilizing come from the OpenTracing project, and not MicroProfile. Isn't this similar to what we're discussing for Micrometer?

My point here is to show that MP specifications can have varying degrees of APIs, specifications, and TCKs, and still be a MicroProfile specification.

If we moved to utilize Micrometer for metrics in MP, the main programmatic API would certainly be in Micrometer, but maybe there are CDI specific pieces that are in an MP Metrics API for it? And a specification and TCK would still be required, to define the use of Micrometer and indicate the expected minimum metrics to be produced from JAX-RS endpoints, etc.

Another key point that I've come to realize over the last few months, is that there are two types of APIs at play here which can cause confusion about how they should be specified. What I mean is APIs that developers use for creating their microservices with CDI, JAX-RS, Fault Tolerance, Config, etc are all APIs they will need to use directly all the time. It's not possible to auto-include Fault Tolerance into the code, because you don't know where to do it.

This is where I see observability pieces like metrics and tracing as being different to the APIs for CDI, JAX-RS, Fault Tolerance, etc. Ideally, observability for 80-90% of what is needed for a microservice can, and should, be done without the developer needing to worry about any API or adding any code. It's just a dependency and some configuration. In these situations maybe no API is needed from an MP perspective because we feel the 80-90% use case coverage is sufficient initially, and adding an API only causes confusion with developers feeling that they should be using it.

In terms of what I think it means for metrics beyond Metrics 3.0, there are several things:

• Utilize Micrometer for metrics
• Define a specification that focuses on the metrics expected to be output without developer intervention, such as JAX-RS endpoint metrics. In particular how it plays into expected metrics from other MP specifications
• For Metrics 4.0 we can retain the annotations but mark them as deprecated to indicate that their use will not be promoted anymore and will be removed at some point. Vendors can provide the necessary shim to convert these annotations into metrics in Micrometer

Investigating possible API/spec pieces related to CDI could be started during this phase, but I would prefer any CDI specific pieces being added to not block the release of the spec with Micrometer.

Given the above, while we certainly have some time before a breaking change release in June 2021 that time will pass very quickly so I would like to progress these discussions within the Metrics group over the coming months to reach a proposal of what it could look like.

And again, I appreciate the tremendous amount of feedback that was provided by the community, it's been invaluable in shaping the discussion.

Thanks

Ken

--
You received this message because you are subscribed to the Google Groups "MicroProfile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/47349732-9940-4679-A364-E40D0C0F02A7%40gmail.com.

[Ladislav Thon]

Hi,

this seems sensible to me. Just one question:

čt 20. 8. 2020 v 20:19 odesílatel Ken Finnigan <k...@kenfinnigan.me> napsal:

This is where I see observability pieces like metrics and tracing as being different to the APIs for CDI, JAX-RS, Fault Tolerance, etc. Ideally, observability for 80-90% of what is needed for a microservice can, and should, be done without the developer needing to worry about any API or adding any code. It's just a dependency and some configuration. In these situations maybe no API is needed from an MP perspective because we feel the 80-90% use case coverage is sufficient initially, and adding an API only causes confusion with developers feeling that they should be using it.

This only covers infrastructure metrics. Do you guys expect that no business metrics (which are inherently application-specific and can only possibly be defined by applications) exist? Or, if they do, that they should be defined/stored/inspected in a significantly different manner?

LT

 

In terms of what I think it means for metrics beyond Metrics 3.0, there are several things:

• Utilize Micrometer for metrics
• Define a specification that focuses on the metrics expected to be output without developer intervention, such as JAX-RS endpoint metrics. In particular how it plays into expected metrics from other MP specifications
• For Metrics 4.0 we can retain the annotations but mark them as deprecated to indicate that their use will not be promoted anymore and will be removed at some point. Vendors can provide the necessary shim to convert these annotations into metrics in Micrometer

Investigating possible API/spec pieces related to CDI could be started during this phase, but I would prefer any CDI specific pieces being added to not block the release of the spec with Micrometer.

Given the above, while we certainly have some time before a breaking change release in June 2021 that time will pass very quickly so I would like to progress these discussions within the Metrics group over the coming months to reach a proposal of what it could look like.

And again, I appreciate the tremendous amount of feedback that was provided by the community, it's been invaluable in shaping the discussion.

Thanks

Ken

On Wed, Aug 19, 2020 at 6:03 PM Erik Mattheis <erikma...@gmail.com> wrote:

> On Aug 18, 2020, at 2:20 PM, Steve Millidge <l33t...@gmail.com> wrote:
>
> I agree with Emily here the whole purpose of MicroProfile is to define apis, specifications and TCKs.

I believe the greater value of MicroProfile is to define a platform comprised of specific APIs and guidance around how they interoperate. Indeed, the backbone of MP for me (and really all MP users) is the suite of Jakarta EE APIs that are not defined by MicroProfile (e.g. CDI, JAX-RS, JSON-P, etc.) and a big challenge of MP right now is how to define and validate the usage of these APIs in compliant platforms. Where there is a void, or lack of consensus, MP can play a useful role by stepping in and defining new APIs, but I would personally like to see those initiatives serve as incubators for Jakarta EE. As such, any API defined solely by MicroProfile should be seen as having a limited lifespan and will eventually be subsumed by Jakarta EE or abandoned.

> Personally I don't see any point in adopting a 3rd party api as is as part of the platform. If people feel that the concepts of Micrometer are more robust shouldn't we be evolving MP metrics towards those concepts while enabling some vendors to ease that migration for their users.

Assuming it finds it’s way to an open consortium and finds broader vendor support, how Is Micrometer any more a 3rd party than CDI or JAX-RS at this point? I certainly support aligning with Jakarta EE, but I see MP as a great way to glue emerging de facto standards in the broader Jakarta EE ecosystem. This was a big missing piece of Java EE in the past because as developers we could not count on non-standard APIs being present across otherwise compatible platforms, so we ended up with “portable” apps that barely used any of the core Java EE and packaged hundreds of libs into each portable artifact.

It was exciting for me to see MicroProfile emerge at a time when Java EE was faltering and it seemed like vendors might all just go their separate ways. To the extent that MP encouraged us to take a look at what parts of Java EE were worth keeping around an not reinventing, while embracing new technologies that Java EE simply couldn’t move fast enough to address, it was great. We’re at a point now where there are several solid offerings around MicroProfile, but it feels like they are innovating around it rather than with it. I know a lot of that has to do with the organizational challenges that are ongoing and I have great respect for those of you who are fighting hard to bring that to fruition, but we need to make sure that we’re not just trying to create Jakarta EE lite. As a developer, I care much more about APIs being consistently available across platforms than whether or not they were specified by a specific standards body.

—
Erik

--
You received this message because you are subscribed to the Google Groups "MicroProfile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/47349732-9940-4679-A364-E40D0C0F02A7%40gmail.com.

--
You received this message because you are subscribed to the Google Groups "MicroProfile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CAKeeVe4T%3DoMWX_%3DhqNu7Gb9VckNwbU_avtc0T9nJB0Wx8HWyjA%40mail.gmail.com.

[Ken Finnigan]

Ah right. I didn't intend to imply that you'd never be writing business metrics.

What I poorly tried to convey is that a large portion of microservices/applications won't need to go beyond the 80-90% use case provided by infrastructure metrics.

For those that do need business metrics, then a programmatic API is the preferred approach, which would require direct use of the Micrometer API for it.

Hope that clarifies what I was trying to convey

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CALbocOm8UwU9emkc0Ee2GrN_WbkD71Lu5qYF-gq3gBgE6tAgSQ%40mail.gmail.com.

[Erin Schnabel]

(And further, per Werner’s observations, a vast number of applications and libraries are already using the Micrometer APIs where that is necessary).

You received this message because you are subscribed to a topic in the Google Groups "MicroProfile" group.


To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/mMvGzLq0Zkc/unsubscribe.


To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.


To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CAKeeVe4svDiuGE7owqUyNBmj%3DwsSCE%2B534Q3L87ayU%3DL2ysPAQ%40mail.gmail.com.

[Werner Keil]

To be precise Micrometer similar to most of the Spring technologies does not have a separate "API" but "Micrometer Core" is probably closest to that.

Werner

[Edwin Derks]

@Werner Keil

Yes, that is an "advantage" of a component based platform like Spring Framework. They can have a Micrometer (core) component with a Spring sauce syntax that can be used by anyone including this specific component in his/her project. In the hypothetical scenario that this component goes out of favor, it is just not being used anymore without too big of an impact to the Spring (Bot) core itself.

In MicroProfile, this scenario would mean deprecating and eventually removing this specification from the platform which, as identified, has a lot of repercussions for the evolution of the platform. Nothing is a silver bullet :)

Edwin

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/4d668788-02e1-463f-a4a2-48871757628en%40googlegroups.com.

[Werner Keil]

I would not call it an advantage except for VMware because it is a classic vendor-lock-in. 

And if Dell finds a buyer for it, the customers are at the mercy of that new owner, how much they understand or care for metrics and everything else in the Spring portfolio.

Same as Hystrix or Archaius by Netflix which are both practically EOL now, they still work but even bugs don't seem to be fixed unless a user does it in their own fork.

While Jakarta EE and to a much smaller extent MP (because it also bound itself to only a few solutions like Prometheus, Dropwizard or Failsafe before) allow vendor-independence and if all goes well even swapping a defunct or otherwise inconvenient product for another one without having to rewrite your entire application.

Werner

[Ladislav Thon]

pá 21. 8. 2020 v 16:18 odesílatel Erin Schnabel <erin.s...@gmail.com> napsal:

(And further, per Werner’s observations, a vast number of applications and libraries are already using the Micrometer APIs where that is necessary).

Those observations were download statistics from one particular Maven repository (which, AFAIK, is often used by a certain subset of the Java community, and quite unknown or unused to the other subset, so there's a certain bias), so your deduction seems to be a bit of a stretch to me. If anything, they say that Micrometer is vastly more popular than MP Metrics, but then Spring is vastly more popular than MicroProfile [backed by your favorite google trends query :grin:], and yet we are still here :-)

And if Ken is right and "a large portion of microservices/applications won't need to go beyond the 80-90% use case provided by infrastructure metrics", then even if we had perfect download statistics of Micrometer, they wouldn't say much about usages of Micrometer API. Though to be fair, I'm not buying that argument. If you're "this tall to use microservices", I'm pretty certain you need to collect business metrics as well.

Which brings me to what I really wanted to say. I wanted to repeat the two points I already made in this discussion, because I don't feel like they have been sufficiently addressed. In my opinion, we should strive to preserve

1. the MicroProfile programming model;

2. user's knowledge investment.

If MP Metrics 3.0 remains in MP 4.0, what's preventing MP Metrics 4.0 (or 5.0 to highlight the difference :-) ) from becoming a thin declarative layer on top of Micrometer, preferably very close to existing annotations, if possible? Similarly to MP OpenTracing, which was used as an argument -- where we deliberately expose the OpenTracing API and add the `@Traced` annotation on top, because we like declarative APIs.

At this point, I don't know why one shouldn't use the same system for collecting infrastructure metrics and business metrics, but I'm by no means an expert. I'm explicitly open to be convinced that business metrics deserve significantly different treatment, in which case my suggestion above obviously makes zero sense.

LT

 

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CADJ_gaRqF3pWJgLWrNGx5racVg5EkJgC%3DiCBEdb2RduQ%3DMgE_A%40mail.gmail.com.

[Ken Finnigan]

From my limited experience of metrics and specifically business type metrics, the information wanting to be captured for business metrics is a value of internal happenings within a method. Sometimes it's the result of a method call, but not often.

And when dealing with metrics that aren't the result of method calls, or parameters passed in, but some intermediary state or value, it's necessary to utilize a programmatic API to capture that information.

With my limited knowledge of the subject, I don't see business metrics as being capturable, if that's a word, by defining annotations on methods. It's for that reason that I don't see any value in offering annotations, as they're not useful in capturing meaningful business metrics.

Ken

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/CALbocOmYMoVMDcXpY6reNs7a6TUcnXjqzG6QVxovScvH1SfBkQ%40mail.gmail.com.

[Werner Keil]

> Those observations were download statistics from one particular Maven repository

That's not correct, the 25 M downloads of Spring or Spring Boot per month are clearly not just Bintray or JCenter. It is a mirror of the Sonatype repositories anyway and those numbers are pretty certain to be cummulative across all mirrors. The Sonatype equivalent isn't as good with stats or only offers them to paying customers while JCenter also shares them (for 30 days but for most packages except the most busy ones you also see a lifetime download sum in the package details) with the public community.

MP isn't even uploaded to Bintray or JCenter at first, so those numbers are clearly coming via the Sonatype mirror.

As for business metrics I know of a few cases where general purpose metrics or log analytics like Elastic are used for business metrics. E.g. a large pharmacy chain (like what Boots is in the UK) that analyses how many articles or product types are sold in a particular store.

However, this doesn't work for all cases and most other clients or use cases I saw either in my own projects or shared at MeetUps do this in separate ways.

So I would also focus mainly on the "technical" metrics.

Werner

[Ladislav Thon]

po 24. 8. 2020 v 23:02 odesílatel Werner Keil <werne...@gmail.com> napsal:

> Those observations were download statistics from one particular Maven repository

That's not correct, the 25 M downloads of Spring or Spring Boot per month are clearly not just Bintray or JCenter. It is a mirror of the Sonatype repositories anyway and those numbers are pretty certain to be cummulative across all mirrors. The Sonatype equivalent isn't as good with stats or only offers them to paying customers while JCenter also shares them (for 30 days but for most packages except the most busy ones you also see a lifetime download sum in the package details) with the public community.

Interesting, I didn't know that. Thanks for correcting me.

I wonder how they do it :-)

 

As for business metrics I know of a few cases where general purpose metrics or log analytics like Elastic are used for business metrics. E.g. a large pharmacy chain (like what Boots is in the UK) that analyses how many articles or product types are sold in a particular store.

However, this doesn't work for all cases and most other clients or use cases I saw either in my own projects or shared at MeetUps do this in separate ways.

So I would also focus mainly on the "technical" metrics.

I was more thinking about metrics that you want to see when troubleshooting, while still very much tied to the application. Things like count of created shopping carts, transaction processing time, time/error rate of external calls (can be automatic if the application uses an already-instrumented client, but that's not always the case), etc.

But if most people deal with "true" business metrics differently from "technical" metrics, then that's very good to know -- seems like most of my assumptions were false :-)

LT

 

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/36a129f3-0740-4707-b694-a4a4c49910fan%40googlegroups.com.

[Emily Jiang]

A couple of more comments:

> Those observations were download statistics from one particular Maven repository

> That's not correct, the 25 M downloads of Spring or Spring Boot per month are clearly not just Bintray or JCenter. It is a mirror of the Sonatype repositories anyway and those numbers are pretty certain to be cummulative across all mirrors. The Sonatype equivalent isn't as good with stats or only offers them to paying customers while JCenter also shares them (for 30 days but for most packages except the most busy ones you also see a lifetime download sum in the package details) with the public community.

Since MP APIs are packaged by around dozen runtimes, like Open Liberty, Payara, TomEE, Quarkus etc, there are multiple hosted repo for developers to get. Therefore the download statistics from JCenter might not the true figure for MP API usages. 

> This is where I see observability pieces like metrics and tracing as being different to the APIs for CDI, JAX-RS, Fault Tolerance, etc. Ideally, observability for 80-90% of what is needed for a microservice can, and should, be done without the developer needing to worry about any API or adding any code. It's just a dependency and some configuration. In these situations maybe no API is needed from an MP perspective because we feel the 80-90% use case coverage is sufficient initially, and adding an API only causes confusion with developers feeling that they should be using it.

It seems the argument for directly using Micrometer api is based on the assumption of 80-90% of what is needed can be done without developers' involvement. How can we be so sure? Are there any proof based on Micrometer usage? What metrics Micrometer can provide out of box already where MP Metrics have to require developers to invoke APIs or use annotations?

Thanks

Emily

[Ken Finnigan]

On Tue, Aug 25, 2020 at 5:36 AM 'Emily Jiang' via MicroProfile <microp...@googlegroups.com> wrote:

A couple of more comments:

> Those observations were download statistics from one particular Maven repository

> That's not correct, the 25 M downloads of Spring or Spring Boot per month are clearly not just Bintray or JCenter. It is a mirror of the Sonatype repositories anyway and those numbers are pretty certain to be cummulative across all mirrors. The Sonatype equivalent isn't as good with stats or only offers them to paying customers while JCenter also shares them (for 30 days but for most packages except the most busy ones you also see a lifetime download sum in the package details) with the public community.

Since MP APIs are packaged by around dozen runtimes, like Open Liberty, Payara, TomEE, Quarkus etc, there are multiple hosted repo for developers to get. Therefore the download statistics from JCenter might not the true figure for MP API usages. 

> This is where I see observability pieces like metrics and tracing as being different to the APIs for CDI, JAX-RS, Fault Tolerance, etc. Ideally, observability for 80-90% of what is needed for a microservice can, and should, be done without the developer needing to worry about any API or adding any code. It's just a dependency and some configuration. In these situations maybe no API is needed from an MP perspective because we feel the 80-90% use case coverage is sufficient initially, and adding an API only causes confusion with developers feeling that they should be using it.

It seems the argument for directly using Micrometer api is based on the assumption of 80-90% of what is needed can be done without developers' involvement. How can we be so sure? Are there any proof based on Micrometer usage? What metrics Micrometer can provide out of box already where MP Metrics have to require developers to invoke APIs or use annotations?

Pretty much any situation. A recent MP Metrics release added optional metrics for REST endpoints, that's currently the only possible overlap. Micrometer as MeterBinder's from many frameworks directly, such as Tomcat, Hibernate, etc, and other frameworks implement MeterBinder themselves. What this means is that when one of these frameworks is included, a large number of metrics are automatically available without any effort on the part of the developer.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/792005f9-32a3-46a1-927e-21cffddc98acn%40googlegroups.com.

[Jan Martiška]

On Tuesday, August 25, 2020 at 11:36:55 AM UTC+2 Emily Jiang wrote:

A couple of more comments:

> Those observations were download statistics from one particular Maven repository

> That's not correct, the 25 M downloads of Spring or Spring Boot per month are clearly not just Bintray or JCenter. It is a mirror of the Sonatype repositories anyway and those numbers are pretty certain to be cummulative across all mirrors. The Sonatype equivalent isn't as good with stats or only offers them to paying customers while JCenter also shares them (for 30 days but for most packages except the most busy ones you also see a lifetime download sum in the package details) with the public community.

Since MP APIs are packaged by around dozen runtimes, like Open Liberty, Payara, TomEE, Quarkus etc, there are multiple hosted repo for developers to get. Therefore the download statistics from JCenter might not the true figure for MP API usages. 

> This is where I see observability pieces like metrics and tracing as being different to the APIs for CDI, JAX-RS, Fault Tolerance, etc. Ideally, observability for 80-90% of what is needed for a microservice can, and should, be done without the developer needing to worry about any API or adding any code. It's just a dependency and some configuration. In these situations maybe no API is needed from an MP perspective because we feel the 80-90% use case coverage is sufficient initially, and adding an API only causes confusion with developers feeling that they should be using it.

It seems the argument for directly using Micrometer api is based on the assumption of 80-90% of what is needed can be done without developers' involvement. How can we be so sure? Are there any proof based on Micrometer usage? What metrics Micrometer can provide out of box already where MP Metrics have to require developers to invoke APIs or use annotations?

 

If we're talking just framework-generated (not application-specific) metrics, micrometer supports metrics from  multiple frameworks by itself, MP Metrics itself only specifies support for JAX-RS metrics, but the implementation can provide metrics for any other framework. And this does not force the user to use any APIs or annotations, it's just a matter of configuration (unless the implementation decides to introduce some sort of API for that, but for metrics specific to frameworks, this does not usually make sense). So the answer to your question depends on what the MP Metrics implementation supports, but usually it is zero, unless the implementation decides to do something special in order to support framework-specific metrics.

Application specific metrics will always need the user to use annotations or APIs, there's no way around that.

[Werner Keil]

These numbers are not the APIs, but the individual implementations like Open Liberty (which at least in MavenCentral/JCenter had less than 200 downloads per month, othres like Helidon are doing a bit better) especially since Micrometer does not have a separate API, the "core" module is closest to that.

However, given Micrometer is now used by hundreds of other projects and products, if you added these indirect downloads in binary form then you probably come to 400k or more per month while all open source or commercial implementations of MP APIs at most got to say 4 or 5k, the comparison is more important here and it's quite undeniable with 100 times larger figures of Micrometer.

Wonder if Micrometer has a built-in catalog of metrics like  https://pcp.io/ does? 

The documentation https://micrometer.io/docs is not so clear, but maybe digging a little deeper tells you something.

It does tell there are annotations like @Timed or (something MP does not offer but it's also an extra overhead if you use Micrometer like that) AOP aspects and these do not magically work on their own, but you'll have to use them in your code, even if a Spring Boot Starter may generate a bit of that boiler plate, so providing that is more thanks to Spring than Micrometer itself.

It also doesn't come with any REST endpoint or "actuator" unless it is combined with a runtime like Spring, Jersey, Jetty or vert.x, but that's not different from MP.

Werner

[Erin Schnabel]

I heard LT’s question, and I think we still have too few people who have tried to use micrometer. 

It *is* an API. It wraps exporter-specific counter/gauge/histogram implementations. It has a TCK for exporters. We are coming at this thing sideways.

First, we assume the MP annotations as-is are useful. I am not sure enough people have (again) used Micrometer to understand the metrics being produced without user involvement. Once you know what you are getting out-of-the-box, your domain-specific metrics can be more focused.

I am not sure this conversation is going anywhere. I think more experimentation would help. I know everyone is busy, but I think you have to have tried all the things to really be able to see what is up.

To that end, I created a completely ridiculous monster combat application that makes D&D monsters fight each other. I have spring & quarkus apps with micrometer, ken added a quarkus app with the mp metrics extension.  I welcome others. Let’s play!

https://github.com/ebullient/monster-combat

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/f2cb322f-1087-4404-a345-12f4c578f2b1n%40googlegroups.com.

[Jan Martiška]

On Wednesday, August 26, 2020 at 1:12:50 AM UTC+2 erinsc...@gmail.com wrote:

I heard LT’s question, and I think we still have too few people who have tried to use micrometer. 

It *is* an API. It wraps exporter-specific counter/gauge/histogram implementations. It has a TCK for exporters. We are coming at this thing sideways.

First, we assume the MP annotations as-is are useful. I am not sure enough people have (again) used Micrometer to understand the metrics being produced without user involvement.

I get the feeling you're selling this (meter binders in Micrometer) as something that will save the application developer from writing code, but I disagree with that, as I've unsuccessfully tried to explain several times. If a MP Metrics implementation provides auto-generated metrics for a specific framework, the user gets the same thing already (without involvement of the app developer).
With Micrometer, the metrics wiring responsibility would just generally be moved from the MP implementation to Micrometer, so, potentially it may only save some coding on part of the IMPLEMENTATION developer (by implementation here I mean the MP runtime that uses Micrometer under the hood), and it comes at potentially severe costs:

1) The MP implementation (that uses Micrometer under the hood) loses some flexibility because it does not have the fine grained control over what metrics Micrometer will produce and how, as opposed to the case where the implementation provides the wiring code by itself, where the code would be closer to the underlying framework

2) Potential version compatibility problem. The runtime will be forced to use metric binders that are designed for a particular version of a framework. A hypothetical example:

Suppose a MP implementation is using a Micrometer version that provides metric binders compatible with Hibernate 8.0 and Vert.x 6.0. Now the runtime wants to upgrade to Hibernate 9.0, which is incompatible with 8.0 and needs a whole new metric binder. There is no Micrometer version available that supports Vert.x 6.0 and Hibernate 9.0. What will the MP implementation do? 

I think this could lead to serious stymies where implementations are unable to upgrade their components to new major versions, because Micrometer hasn't caught up, therefore they will have to either align their component versions to those supported by a particular Micrometer version, or they could provide their own metric binders, which means this whole advantage (of binders being provided by Micrometer) is lost.

So to sum up, it feels like we would potentially save a few lines of code in the MP implementation codebase (and zero lines in the applications' codebases), with serious potential issues arising from it. That doesn't sound like a good deal to me.

 

Once you know what you are getting out-of-the-box, your domain-specific metrics can be more focused.

That's another facet to the problem - I say that with Micrometer, we would actually *lose* some control over what users will be getting out-of-the-box, because (most of) it would be provided by Micrometer itself (a 3rd party library that MP vendors don't directly control) as opposed to being provided by the MP vendor.

[Erin Schnabel]

Inline

On Wed, Aug 26, 2020 at 3:43 AM Jan Martiška <wray...@gmail.com> wrote:

On Wednesday, August 26, 2020 at 1:12:50 AM UTC+2 erinsc...@gmail.com wrote:

I heard LT’s question, and I think we still have too few people who have tried to use micrometer. 

It *is* an API. It wraps exporter-specific counter/gauge/histogram implementations. It has a TCK for exporters. We are coming at this thing sideways.

First, we assume the MP annotations as-is are useful. I am not sure enough people have (again) used Micrometer to understand the metrics being produced without user involvement.

I get the feeling you're selling this (meter binders in Micrometer) as something that will save the application developer from writing code, but I disagree with that, as I've unsuccessfully tried to explain several times. If a MP Metrics implementation provides auto-generated metrics for a specific framework, the user gets the same thing already (without involvement of the app developer).

No. I’m not. I am trying to get everyone to understand that using the Micrometer classes programmatically using an API with a TCK that works across multiple vendors (all of the things MP purports to do). The definition of a vendor is different, and I’m suggesting that difference is significant, and that we’re focusing on the wrong part of the problem. Vendor independence (in this case) should focus on the exporter.

With Micrometer, the metrics wiring responsibility would just generally be moved from the MP implementation to Micrometer, so, potentially it may only save some coding on part of the IMPLEMENTATION developer (by implementation here I mean the MP runtime that uses Micrometer under the hood), and it comes at potentially severe costs:

1) The MP implementation (that uses Micrometer under the hood) loses some flexibility because it does not have the fine grained control over what metrics Micrometer will produce and how, as opposed to the case where the implementation provides the wiring code by itself, where the code would be closer to the underlying framework

You don’t need to use a binder because it exists.  There are a lot of pre-existing binders, and using them can help for consistency for how metrics are emitted across a system (where that system includes a variety of Spring and not-Spring Java applications), but none of them are required, nor are they discovered/enabled automatically unless you (the runtime vendor) write it that way.

 

2) Potential version compatibility problem. The runtime will be forced to use metric binders that are designed for a particular version of a framework. A hypothetical example:

Suppose a MP implementation is using a Micrometer version that provides metric binders compatible with Hibernate 8.0 and Vert.x 6.0. Now the runtime wants to upgrade to Hibernate 9.0, which is incompatible with 8.0 and needs a whole new metric binder. There is no Micrometer version available that supports Vert.x 6.0 and Hibernate 9.0. What will the MP implementation do? 

The Hibernate Binder (if you choose to use that) is written to a Hibernate API. 

If you are the first MP vendor to get there, sure, you might need to write a revised binder that works with v9  (which in theory you would contribute back, of course, to save the next vendor some work). Micrometer uses compile optional dependencies, there is no dependency drag. If a binder doesn’t work, you don’t use it. If you write a new one, you write a new one. Binders are self-contained (an advantage for the programmatic approach), replacing them is easy.

I think this could lead to serious stymies where implementations are unable to upgrade their components to new major versions, because Micrometer hasn't caught up, therefore they will have to either align their component versions to those supported by a particular Micrometer version, or they could provide their own metric binders, which means this whole advantage (of binders being provided by Micrometer) is lost.

I think this is making mountains that don’t exist. Common Micrometer binder implementations are nice to have, but they are not a trap.

So to sum up, it feels like we would potentially save a few lines of code in the MP implementation codebase (and zero lines in the applications' codebases), with serious potential issues arising from it. That doesn't sound like a good deal to me.

I have been trying to challenge our notion of who the “vendor” needs to be for metrics. In the case of micrometer, micrometer-core is a library, the vendor is the exporter in MP terms (datadog, newrelic, stackdriver, prometheus, we can make a registry for the MP JSON format, too).  Using micrometer gets all of the MP vendors out of the business of writing exporters and dealing with their configuration requirements, etc. (while still allowing the runtime vendor to read/manage configuration according to what it needs). 

Once you know what you are getting out-of-the-box, your domain-specific metrics can be more focused.

That's another facet to the problem - I say that with Micrometer, we would actually *lose* some control over what users will be getting out-of-the-box, because (most of) it would be provided by Micrometer itself (a 3rd party library that MP vendors don't directly control) as opposed to being provided by the MP vendor.

Ok, this is the real crux of the problem, and it isn’t technical at all. It is political. 

I’ve asked people to play with the library and the implementation so you can get a hands-on feeling for what is different about the approach (most of the roadblocks fall down as soon as you start doing), but I think all of the technical arguments are moot if the real issue is that this community doesn’t trust (or is not interested in engaging with) the Micrometer community.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/a9e1780c-73a8-426f-a74d-928864ad65f1n%40googlegroups.com.

[Mark Little]

I don’t understand this “lose control” comment. If this group wants to create a “silo of excellence” whereby they implement everything and don’t rely on anything else outside, then I think we’re losing sight of the benefits of growing a wide and vibrant ecosystem and community. I don’t recall hearing those concerns in over 20 years of J2EE efforts when the platform grew to be immensely successful because it didn’t try to implement everything and leverages the communities/user bases from elsewhere, e.g., CORBA for transactions.

If Micrometer is the de facto standard for doing this right then why wouldn’t we use it? If the MP community and vendors get to a stage in the future where we feel Micrometer is heading in a different direction from what we want to do then let’s revisit then. But it’s crazy to not adopt something which is apparently so successful and beneficial for end users because some are worried that influence today may not be above a certain threshold.

Mark.

[Erik Mattheis]

On Aug 26, 2020, at 7:41 AM, Erin Schnabel <erin.s...@gmail.com> wrote:

I have been trying to challenge our notion of who the “vendor” needs to be for metrics. In the case of micrometer, micrometer-core is a library, the vendor is the exporter in MP terms (datadog, newrelic, stackdriver, prometheus, we can make a registry for the MP JSON format, too).  Using micrometer gets all of the MP vendors out of the business of writing exporters and dealing with their configuration requirements, etc. (while still allowing the runtime vendor to read/manage configuration according to what it needs). 

And this is a big deal! Being able to write portable application code is certainly nice, but having runtimes that integrate with cloud ecosystems in a consistent manner is also very important. Not having to port my application code is not a huge win when I have to rewire everything else around it anyway. I would very much like to see a MicroProfile ecosystem where runtimes vendors are providing alternatives that bridge application code to cloud environments consistently. Vendors are going to include frameworks outside of the MP spec to stay relevant anyway. If we embrace that and provide consistency about how the frameworks are integrated, we can make MicroProfile more valuable.

That's another facet to the problem - I say that with Micrometer, we would actually *lose* some control over what users will be getting out-of-the-box, because (most of) it would be provided by Micrometer itself (a 3rd party library that MP vendors don't directly control) as opposed to being provided by the MP vendor.

Ok, this is the real crux of the problem, and it isn’t technical at all. It is political. 

MicroProfile simply does not have the capacity to define EVERYTHING at a pace that keeps it relevant. We lose much more by not paying close attention to what our users want and use. We aren’t controlling anything by spending time inventing a new metrics framework while users are happily adopting Micrometer.

— 

Erik

[Erik Mattheis]

+1

[Jan Martiška]

On Wednesday, August 26, 2020 at 1:42:05 PM UTC+2 erinsc...@gmail.com wrote:

Inline

On Wed, Aug 26, 2020 at 3:43 AM Jan Martiška <wray...@gmail.com> wrote:

On Wednesday, August 26, 2020 at 1:12:50 AM UTC+2 erinsc...@gmail.com wrote:

I heard LT’s question, and I think we still have too few people who have tried to use micrometer. 

It *is* an API. It wraps exporter-specific counter/gauge/histogram implementations. It has a TCK for exporters. We are coming at this thing sideways.

First, we assume the MP annotations as-is are useful. I am not sure enough people have (again) used Micrometer to understand the metrics being produced without user involvement.

I get the feeling you're selling this (meter binders in Micrometer) as something that will save the application developer from writing code, but I disagree with that, as I've unsuccessfully tried to explain several times. If a MP Metrics implementation provides auto-generated metrics for a specific framework, the user gets the same thing already (without involvement of the app developer).

No. I’m not. I am trying to get everyone to understand that using the Micrometer classes programmatically using an API with a TCK that works across multiple vendors (all of the things MP purports to do). The definition of a vendor is different, and I’m suggesting that difference is significant, and that we’re focusing on the wrong part of the problem. Vendor independence (in this case) should focus on the exporter.

Then I'm not sure what you exactly mean by the benefits of "metrics being produced without user involvement" that you've mentioned several times in this discussion. Is there anything in it for the application developer? Or who exactly benefits from this thing in particular? This has nothing to do with TCKs and the definition of a vendor.
Sorry I might just be missing your point. This discussion is somewhat lengthy and exhausting, so it's getting easy to misunderstand arguments :)

 

With Micrometer, the metrics wiring responsibility would just generally be moved from the MP implementation to Micrometer, so, potentially it may only save some coding on part of the IMPLEMENTATION developer (by implementation here I mean the MP runtime that uses Micrometer under the hood), and it comes at potentially severe costs:

1) The MP implementation (that uses Micrometer under the hood) loses some flexibility because it does not have the fine grained control over what metrics Micrometer will produce and how, as opposed to the case where the implementation provides the wiring code by itself, where the code would be closer to the underlying framework

You don’t need to use a binder because it exists.  There are a lot of pre-existing binders, and using them can help for consistency for how metrics are emitted across a system (where that system includes a variety of Spring and not-Spring Java applications), but none of them are required, nor are they discovered/enabled automatically unless you (the runtime vendor) write it that way.

Sure, you don't have to use it if it does not fit the needs of your runtime. I just felt like it's worth pointing out that this will probably happen, and MP vendors will have to maintain their own binders even when a similar binder happens to exist in Micrometer, so the consistency advantage might be lost.

 

 

2) Potential version compatibility problem. The runtime will be forced to use metric binders that are designed for a particular version of a framework. A hypothetical example:

Suppose a MP implementation is using a Micrometer version that provides metric binders compatible with Hibernate 8.0 and Vert.x 6.0. Now the runtime wants to upgrade to Hibernate 9.0, which is incompatible with 8.0 and needs a whole new metric binder. There is no Micrometer version available that supports Vert.x 6.0 and Hibernate 9.0. What will the MP implementation do? 

The Hibernate Binder (if you choose to use that) is written to a Hibernate API. 

If you are the first MP vendor to get there, sure, you might need to write a revised binder that works with v9  (which in theory you would contribute back, of course, to save the next vendor some work). Micrometer uses compile optional dependencies, there is no dependency drag. If a binder doesn’t work, you don’t use it. If you write a new one, you write a new one. Binders are self-contained (an advantage for the programmatic approach), replacing them is easy.

If you decide to contribute it to Micrometer and update its v8 binder to v9, then you break all other Micrometer users who are still on  Hibernate 8. That's the dependency alignment problem. With potentially tens of different frameworks for which you need to keep their versions aligned, this problem grows exponentially.
If you don't contribute it yet, you have to maintain your own implementation of v9 binder until the time is "right" for Micrometer to include it, so that's quite a bit of extra overhead and mess. So, not an insurmountable problem, but a potential major nuisance.

I think this could lead to serious stymies where implementations are unable to upgrade their components to new major versions, because Micrometer hasn't caught up, therefore they will have to either align their component versions to those supported by a particular Micrometer version, or they could provide their own metric binders, which means this whole advantage (of binders being provided by Micrometer) is lost.

I think this is making mountains that don’t exist. Common Micrometer binder implementations are nice to have, but they are not a trap.

So to sum up, it feels like we would potentially save a few lines of code in the MP implementation codebase (and zero lines in the applications' codebases), with serious potential issues arising from it. That doesn't sound like a good deal to me.

I have been trying to challenge our notion of who the “vendor” needs to be for metrics. In the case of micrometer, micrometer-core is a library, the vendor is the exporter in MP terms (datadog, newrelic, stackdriver, prometheus, we can make a registry for the MP JSON format, too).  Using micrometer gets all of the MP vendors out of the business of writing exporters and dealing with their configuration requirements, etc. (while still allowing the runtime vendor to read/manage configuration according to what it needs). 

Once you know what you are getting out-of-the-box, your domain-specific metrics can be more focused.

That's another facet to the problem - I say that with Micrometer, we would actually *lose* some control over what users will be getting out-of-the-box, because (most of) it would be provided by Micrometer itself (a 3rd party library that MP vendors don't directly control) as opposed to being provided by the MP vendor.

Ok, this is the real crux of the problem, and it isn’t technical at all. It is political. 

I’ve asked people to play with the library and the implementation so you can get a hands-on feeling for what is different about the approach (most of the roadblocks fall down as soon as you start doing), but I think all of the technical arguments are moot if the real issue is that this community doesn’t trust (or is not interested in engaging with) the Micrometer community.

Sure, this probably can be remedied by proper engagement with the Micrometer community. There is just the extra overhead and a possible risk of not always getting it the way we need, because the MP community might have an incompatible view to the one at Micrometer. But you're right, it's probably not that much of a problem.

[Erin Schnabel]

Inline + trim

On Wed, Aug 26, 2020 at 9:23 AM Jan Martiška <wray...@gmail.com> wrote:

On Wednesday, August 26, 2020 at 1:42:05 PM UTC+2 erinsc...@gmail.com wrote:

Inline

On Wed, Aug 26, 2020 at 3:43 AM Jan Martiška <wray...@gmail.com> wrote:

On Wednesday, August 26, 2020 at 1:12:50 AM UTC+2 erinsc...@gmail.com wrote:

No. I’m not. I am trying to get everyone to understand that using the Micrometer classes programmatically using an API with a TCK that works across multiple vendors (all of the things MP purports to do). The definition of a vendor is different, and I’m suggesting that difference is significant, and that we’re focusing on the wrong part of the problem. Vendor independence (in this case) should focus on the exporter.

Then I'm not sure what you exactly mean by the benefits of "metrics being produced without user involvement" that you've mentioned several times in this discussion. Is there anything in it for the application developer? Or who exactly benefits from this thing in particular? This has nothing to do with TCKs and the definition of a vendor.
Sorry I might just be missing your point. This discussion is somewhat lengthy and exhausting, so it's getting easy to misunderstand arguments :)

There are more metrics available for a user to enable right now w/ micrometer w/o any direct user involvement (http metrics, JDBC metrics, kafka metrics, pick-your-poison-metrics). That is a benefit to users and operators that doesn’t require direct use of the API in applications (they can be isolated in optionally enabled binders). This is a very capable extension model.

The specific point is really trying to get people to understand what is important for “vendor neutrality”. Micrometer is a stable API with long term service releases that is “vendor neutral” with respect to metrics vendors.

With Micrometer, the metrics wiring responsibility would just generally be moved from the MP implementation to Micrometer, so, potentially it may only save some coding on part of the IMPLEMENTATION developer (by implementation here I mean the MP runtime that uses Micrometer under the hood), and it comes at potentially severe costs:

1) The MP implementation (that uses Micrometer under the hood) loses some flexibility because it does not have the fine grained control over what metrics Micrometer will produce and how, as opposed to the case where the implementation provides the wiring code by itself, where the code would be closer to the underlying framework

You don’t need to use a binder because it exists.  There are a lot of pre-existing binders, and using them can help for consistency for how metrics are emitted across a system (where that system includes a variety of Spring and not-Spring Java applications), but none of them are required, nor are they discovered/enabled automatically unless you (the runtime vendor) write it that way.

Sure, you don't have to use it if it does not fit the needs of your runtime. I just felt like it's worth pointing out that this will probably happen, and MP vendors will have to maintain their own binders even when a similar binder happens to exist in Micrometer, so the consistency advantage might be lost. 

The Quarkus Micrometer maintains its own Binder for Vert.x. This is absolutely true. 

But because my binder works with the Micrometer API, there is nothing I have to do to support any cloud/operational metrics vendor (datadog, stackdriver, prometheus, ... ).

2) Potential version compatibility problem. The runtime will be forced to use metric binders that are designed for a particular version of a framework. A hypothetical example:

Suppose a MP implementation is using a Micrometer version that provides metric binders compatible with Hibernate 8.0 and Vert.x 6.0. Now the runtime wants to upgrade to Hibernate 9.0, which is incompatible with 8.0 and needs a whole new metric binder. There is no Micrometer version available that supports Vert.x 6.0 and Hibernate 9.0. What will the MP implementation do? 

The Hibernate Binder (if you choose to use that) is written to a Hibernate API. 

If you are the first MP vendor to get there, sure, you might need to write a revised binder that works with v9  (which in theory you would contribute back, of course, to save the next vendor some work). Micrometer uses compile optional dependencies, there is no dependency drag. If a binder doesn’t work, you don’t use it. If you write a new one, you write a new one. Binders are self-contained (an advantage for the programmatic approach), replacing them is easy.

If you decide to contribute it to Micrometer and update its v8 binder to v9, then you break all other Micrometer users who are still on  Hibernate 8. That's the dependency alignment problem. With potentially tens of different frameworks for which you need to keep their versions aligned, this problem grows exponentially.
If you don't contribute it yet, you have to maintain your own implementation of v9 binder until the time is "right" for Micrometer to include it, so that's quite a bit of extra overhead and mess. So, not an insurmountable problem, but a potential major nuisance.

We could also create a hibernate-micrometer project, that does the matching between hibernate versions and long-term-service releases of micrometer.

I will repeat again, Micrometer is stable with long term service releases. Any binder in micrometer-core is there for convenience as a compile/optional dependency. Consumers of the core library are not pinned to a specific version (e.g. of hibernate). 

Let’s not borrow trouble here, this is a solvable problem with examples elsewhere (camel-micrometer, vertx-micrometer, there are others but I’m lazy with searching).

That's another facet to the problem - I say that with Micrometer, we would actually *lose* some control over what users will be getting out-of-the-box, because (most of) it would be provided by Micrometer itself (a 3rd party library that MP vendors don't directly control) as opposed to being provided by the MP vendor.

Ok, this is the real crux of the problem, and it isn’t technical at all. It is political. 

I’ve asked people to play with the library and the implementation so you can get a hands-on feeling for what is different about the approach (most of the roadblocks fall down as soon as you start doing), but I think all of the technical arguments are moot if the real issue is that this community doesn’t trust (or is not interested in engaging with) the Micrometer community.

Sure, this probably can be remedied by proper engagement with the Micrometer community. There is just the extra overhead and a possible risk of not always getting it the way we need, because the MP community might have an incompatible view to the one at Micrometer. But you're right, it's probably not that much of a problem.

I suggest good faith engagement as a start. :-P 

[Ken Finnigan]

This thread has produced very interesting discussions and raised important questions.

Moving forward, I'd like to focus on working with those involved in the Metrics spec to push Micrometer forward as we investigate options and solutions. When what would be proposed for MP 5.0, or whenever, is clearer we can circle back to have a wider discussion about the details.

There are, however, a few things I want to note that are key topics that go beyond this particular situation of using Micrometer for Metrics:

• MicroProfile needs to decide whether it wants to operate in an isolated bubble or integrate with the wider Java ecosystem. As Mark mentioned, and Erik and others agreed with, MicroProfile is not going to succeed if we "do our own thing" for every use case we find compelling. We simply do not have sufficient resources to keep adding more and more specifications for every little piece that might fit with microservices. We need to acknowledge that another part of the Java ecosystem "got it right", and endorse it rather than trying to create our own abstraction over the top. Abstractions enable MicroProfile to disregard an upstream because it's "safe" from that project changing in the future.
• From day one MicroProfile had the goal of "portability" across all runtimes. While I think that is a fine goal, part of me thinks it's a fallacy that's chased on the presumption that users want it, but I've had many users say portability isn't meaningful to them because they don't change runtimes on a frequent basis. We also need to consider whether the portability of an application is as important as portability across cloud environments. Maybe we're focusing portability at the wrong level? If a user migrates from Weblogic to Websphere, for instance, they'd expect some level of one-time cost to make that switch, and not just from an application code perspective, but integration into environments and everything that fits around the application to make something usable. I'm not saying we should make portability difficult or impossible, but it's not a panacea either.
• MicroProfile choosing to endorse, or whatever else is done, any 3rd party framework or library doesn't mean it's a "hands-off" situation, completely at the whim of what the project decides. In such a situation I would expect some members of the MicroProfile community, including vendors, that were involved in whatever aspect that the 3rd party framework was included in to participate in that upstream community. Just like all open-source projects, contributing to their upstream dependencies to help with issues or features.

I appreciate that some of the above may not align with how others have seen MicroProfile from its inception, or even where it is today, but they're topics that I firmly believe MicroProfile needs to seriously consider to continue its success and grow it further. MicroProfile can not and will not continue to succeed if it disregards what the wider Java ecosystem and community are doing.

Ken

[Werner Keil]

erinsc...@gmail.com schrieb am Mittwoch, 26. August 2020 um 16:02:14 UTC+2:

Inline + trim

On Wed, Aug 26, 2020 at 9:23 AM Jan Martiška <wray...@gmail.com> wrote:

On Wednesday, August 26, 2020 at 1:42:05 PM UTC+2 erinsc...@gmail.com wrote:

Inline

On Wed, Aug 26, 2020 at 3:43 AM Jan Martiška <wray...@gmail.com> wrote:

On Wednesday, August 26, 2020 at 1:12:50 AM UTC+2 erinsc...@gmail.com wrote:

No. I’m not. I am trying to get everyone to understand that using the Micrometer classes programmatically using an API with a TCK that works across multiple vendors (all of the things MP purports to do). The definition of a vendor is different, and I’m suggesting that difference is significant, and that we’re focusing on the wrong part of the problem. Vendor independence (in this case) should focus on the exporter.

Then I'm not sure what you exactly mean by the benefits of "metrics being produced without user involvement" that you've mentioned several times in this discussion. Is there anything in it for the application developer? Or who exactly benefits from this thing in particular? This has nothing to do with TCKs and the definition of a vendor.
Sorry I might just be missing your point. This discussion is somewhat lengthy and exhausting, so it's getting easy to misunderstand arguments :)

There are more metrics available for a user to enable right now w/ micrometer w/o any direct user involvement (http metrics, JDBC metrics, kafka metrics, pick-your-poison-metrics). That is a benefit to users and operators that doesn’t require direct use of the API in applications (they can be isolated in optionally enabled binders). This is a very capable extension model.

The specific point is really trying to get people to understand what is important for “vendor neutrality”. Micrometer is a stable API with long term service releases that is “vendor neutral” with respect to metrics vendors.

With Micrometer, the metrics wiring responsibility would just generally be moved from the MP implementation to Micrometer, so, potentially it may only save some coding on part of the IMPLEMENTATION developer (by implementation here I mean the MP runtime that uses Micrometer under the hood), and it comes at potentially severe costs:

1) The MP implementation (that uses Micrometer under the hood) loses some flexibility because it does not have the fine grained control over what metrics Micrometer will produce and how, as opposed to the case where the implementation provides the wiring code by itself, where the code would be closer to the underlying framework

You don’t need to use a binder because it exists.  There are a lot of pre-existing binders, and using them can help for consistency for how metrics are emitted across a system (where that system includes a variety of Spring and not-Spring Java applications), but none of them are required, nor are they discovered/enabled automatically unless you (the runtime vendor) write it that way.

Sure, you don't have to use it if it does not fit the needs of your runtime. I just felt like it's worth pointing out that this will probably happen, and MP vendors will have to maintain their own binders even when a similar binder happens to exist in Micrometer, so the consistency advantage might be lost. 

The Quarkus Micrometer maintains its own Binder for Vert.x. This is absolutely true. 

Does Quarkus really have its own Vert.x module, or is it using https://github.com/vert-x3/vertx-micrometer-metrics?

Odd, that keeps failing the build for about a week now btw, but it also provides interfaces in:

https://github.com/vert-x3/vertx-micrometer-metrics/tree/master/src/main/java/io/vertx/micrometer  

which are implemented directly in the same component:

https://github.com/vert-x3/vertx-micrometer-metrics/tree/master/src/main/java/io/vertx/micrometer/impl 

This is about the way Spring also defines most things without a strong separation of API and implementation, but there is nothing that would keep e.g. MP Metrics from keeping them separate and allow more than one implementation module.

Werner 

[Erin Schnabel]

On Wed, Aug 26, 2020 at 3:38 PM Werner Keil <werne...@gmail.com> wrote:

erinsc...@gmail.com schrieb am Mittwoch, 26. August 2020 um 16:02:14 UTC+2:

The Quarkus Micrometer maintains its own Binder for Vert.x. This is absolutely true. 

Does Quarkus really have its own Vert.x module, or is it using https://github.com/vert-x3/vertx-micrometer-metrics?

The quarkus micrometer extension does a lot of things, including providing a consumer of VertxOptions to register MetricsOptions and monitor the flow of traffic through vert.x. 

Odd, that keeps failing the build for about a week now btw, but it also provides interfaces in:

https://github.com/vert-x3/vertx-micrometer-metrics/tree/master/src/main/java/io/vertx/micrometer  

which are implemented directly in the same component:

https://github.com/vert-x3/vertx-micrometer-metrics/tree/master/src/main/java/io/vertx/micrometer/impl 

This is off-topic at this point. I’m happy to walk you through the decisions I made, but suffice to say that the vert.x micrometer library is .. the entirety of setting up micrometer to work with a standalone vert.x. The quarkus extension integrates micrometer with quarkus, including integrating configuration for registries with the quarkus configuration model, etc.  

Vert.x is managed as one of several binders the micrometer extension provides. 

The vert.x binder in the micrometer is heavily inspired by the vert-x3 micrometer metrics library, but using it directly was a bad fit for Quarkus (IMO).

[Emily Jiang]

> From day one MicroProfile had the goal of "portability" across all runtimes. While I think that is a fine goal, part of me thinks it's a fallacy that's chased on the presumption that users want it, but I've had many users say portability isn't meaningful to them because they don't change runtimes on a frequent basis. We also need to consider whether the portability of an application is as important as portability across cloud environments. Maybe we're focusing portability at the wrong level? If a user migrates from Weblogic to Websphere, for instance, they'd expect some level of one-time cost to make that switch, and not just from an application code perspective, but integration into environments and everything that fits around the application to make something usable. I'm not saying we should make portability difficult or impossible, but it's not a panacea either.

Even though migration is not refrequent. However, knowing it is feasible, it will be a great relief for end users to know that they are not being locked in by a particular vendor. I consider this as a great fallback for future investiment and growth. If one day they do have to migrate from Vendor a to b, they know they can because both vendors adopt MicroProfile and their apps use MicroProfile. I think this is still one of big plus points for MicroProfile. As for the configuration update for the runtime on migration, it is understandable because it is directly operating on the runtime itself instead of their Java classes.

Back to this thread topic on MP Metrics, here is my 2c.

From my understanding, there is not a big push back regarding adopting Micrometer. MicroProfile has been adopting popular libraries and specifications to date. It is more to do with how to adopt Micrometer. OpenTel is also emerging. We just need to plan ahead. Is there a way to work out a plan to cope with both?

I think the discussion surounding and questioning about the consequences of directly exposing its APIs. At least it is my understanding.

Maybe we should do step by step?

Step 1: Making decision on adopting Micrometer; Doing a quick poll?

Step 2: Work out how. I think this step will take time as we need to work out a migration path and meet the needs of the current MP Metrics customers. For this, MP Metrics group can work out a plan based on the handson experiment and then bring here for more feedback.

Thanks

Emily

[donbo...@gmail.com]

> Step 1: Making decision on adopting Micrometer; Doing a quick poll?

> Step 2: Work out how. I think this step will take time as we need to work out a migration path and meet the needs of the current MP Metrics customers. For this, MP Metrics group can work out a plan based on the handson experiment and then bring here for more feedback.

Emily, I think that's a fair suggestion.  The first part may be a bit harder to ask than it first appears though.  While I'm not seeing a lot of objection to Micrometer on this thread, it's not clear what adopting Micrometer means until there's some concept for step 2.  Perhaps step 1 could ask something like "Do you think it is a good idea for MP Metrics to be redesigned to somehow exploit the Micrometer API?"

Don

[Emily Jiang]

Fair comment, Don!

[Werner Keil]

I would keep Micrometer out of the abstract MP API wherever possible. 

In this very small interface package of the Vert.x library 
https://github.com/vert-x3/vertx-micrometer-metrics/tree/master/src/main/java/io/vertx/micrometer   

there is not a single trace of Micrometer, that comes only in "impl" or "backend" subpackages which are more on the impl side for MicroProfile as well.

Of course given the current API elements that reassemble Dropwizard "API" to a very large extent, I think they are on the wrong level.

Vert.x also has the same metric service for Dropwizard:  https://vertx.io/docs/vertx-dropwizard-metrics/java/ 

Offering a glimpse of how this might work, with a more abstract set of services that e.g. provide the name of metrics regardless of the underlying framework.

I don't think it would be backward-compatible but once that abstraction or bridge/SPI (which is what Vert.x calls it, so the "API" in that sense is indeed that of Dropwizard or Micrometer) would exist, it should not be a problem to switch between Dropwizard, Micrometer or OpenTelemetry or whatever the next big thing might be.

Werner 

[Erin Schnabel]

Right. That is a complete inversion of the current model, and would be a much better fit from a runtime vendor perspective. 

It doesn’t solve the problem of what applications choose to do, but IMO, my app is still portable if I opt to use micrometer, and the MP-compliant runtime supports micrometer, too. Or if my application opts to use dropwizard, and any MP-compliant runtime could support dropwizard, too. 

Maybe that means standardized interceptors that MP-compliant runtimes enable that can be backed by different metrics systems. This is the closest equivalent to the set of SPIs that Vert.x uses, or the way people gather metrics for Hibernate using its well-known MXBean...

--


You received this message because you are subscribed to a topic in the Google Groups "MicroProfile" group.


To unsubscribe from this topic, visit https://groups.google.com/d/topic/microprofile/mMvGzLq0Zkc/unsubscribe.


To unsubscribe from this group and all its topics, send an email to microprofile...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/8dce964f-2515-40b9-ae54-9ac217ef2865n%40googlegroups.com.

[donbo...@gmail.com]

Werner,

> These numbers are not the APIs, but the individual implementations like Open Liberty (which at least in MavenCentral/JCenter had less than 200 downloads per month...

As you've mentioned Open Liberty a couple of times, I wanted to point out that you're only counting the download numbers from Open Liberty's metrics 1.0 release (with the MP Metrics APIs from 2017).  Open Liberty has a separate API for download for each of the MP metrics releases:

com.ibm.websphere.appserver.org.eclipse.microprofile.metrics-1.0

com.ibm.websphere.appserver.org.eclipse.microprofile.metrics-1.1

com.ibm.websphere.appserver.org.eclipse.microprofile.metrics-2.0

com.ibm.websphere.appserver.org.eclipse.microprofile.metrics-2.2

com.ibm.websphere.appserver.org.eclipse.microprofile.metrics-2.3

Also, I'm not sure JCenter really is aggregating the metrics from Sonatype.  The number of downloads I see for the last 30 days at JCenter seems much much smaller than what we can see at Sonatype.  Given Open Liberty doesn't publish to, nor document using the JCenter repo, that might explain the discrepancy.

Don

[Werner Keil]

Don,

I am very sure that MP features like Metrics or Config do not deploy their packages to JCenter directly nor does Jakarta EE, yet it shows up with e.g. 614211 downloads of "jakarta.servlet-api" (only the new jakarta namespace!!!) so forget about the excuse that only those deployed via JCenter are counted that way, they are not, it is a bi-directional mirror of MavenCentral, if you do upload them (like we do with JSR 385) then both must be counted AFAIK but not just the JCenter side.

If the OpenLiberty bindings change their artifactId that dilutes the overall numbers but you can check each of them in the new JCenter accumulator and only com.ibm.websphere.appserver.org.eclipse.microprofile.metrics-1.1 has notable downloads (186 in the last 30 days, 3276 total) all the others have just a handful in the last month, at most 3-digit figures under 200 total, sorry but that's it, commercial mirrors by companies (I know, we also have an Artifactory in-house) are not counted, maybe some implementations get better results there behind the firewalls, but the Open Source community prefers other implementations or frameworks including Spring.

The API may not always counted when it is pre-bundled (otherwise the Servlet-API could have Millions of downloads because Spring Framework and many other popular libs use it) and Micrometer has no separate API only a "core" library, that's why looking at implementations makes more sense here.

While not 25 Mio within 30 days like Spring 'javax.servlet:javax.servlet-api' had nearly 1 Mio. downloads in just 30 days, showing that not all significant frameworks and libraries migrated to Jakarta Servlet yet, but nearly 100k downloads of that in 30 days isn't so shabby either.

HTH,

Werner

[donbo...@gmail.com]

Hi Werner, let me get with you in gitter to chat.  I can see our packages are mirrored to JCenter, but our Open Liberty numbers from Sonatype don't match with the numbers in JCenter.  Perhaps you and I can figure that out offline and report back if there's anything interesting to be said there.

[Werner Keil]

Where can we see those numbers in Sonatype?

I don't have time for a gitter chat or to chase individual vendors who feel their download figures between Sonatype JCEnter or another repository might be wrong.

If that happens and it's not something you (OpenLiberty) did wrong there, it may affect others, but then potentially we would see another million or more for Micrometer.

It is pretty hard to deny, that all MP implementations combined barely scratch the numbers of Vert.x or Micrometer, but if you have other links and sources please share here, it's better than half of it on Gitter and the other half here.

Werner

[Werner Keil]

It still looks extremely tedious for the Sonatype Nexus repositories including MavenCentral:  https://community.sonatype.com/t/download-statistics-for-repositorys/1435 

According to https://stackoverflow.com/questions/8874670/detailed-download-statistics-for-artifacts-of-public-repositories-like-maven-o it looks like you must log into your Sonatype account to see the stats for your artifacts, but you could share the current Sonatype numbers for those mentioned ones

com.ibm.websphere.appserver.org.eclipse.microprofile.metrics-1.0

com.ibm.websphere.appserver.org.eclipse.microprofile.metrics-1.1

com.ibm.websphere.appserver.org.eclipse.microprofile.metrics-2.0

com.ibm.websphere.appserver.org.eclipse.microprofile.metrics-2.2

com.ibm.websphere.appserver.org.eclipse.microprofile.metrics-2.3

There is no doubt, if more than those had a problem with mirrors, then servlet-api would have potentially several hundred thousands or million more downloads and so does Micrometer or Spring. 

Happy to hear those numbers, but I would be surprised if it was tens or hundreds of thousands, but please let us know.

Werner

[donbo...@gmail.com]

I'm not challenging that we need to look at how to use Micrometer for the next version of MP Metrics -- using Micrometer for the next version makes sense to me.  I agree that Micrometer's API download numbers are quite likely orders of magnitude higher than MP Metrics API download numbers.  My point was just that I have seen data in sonatype that shows higher download numbers than what JCenter is showing for OL and that we have multiple more current versions than the one you were quoting numbers for.  

[Erin Schnabel]

ok. This conversation has jumped the shark, or .. crossed the streams, or.. 

Don, I think you have some thoughts about how to move forward. How about we start that in a new thread? :-P

To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/a27c6cee-0bd1-4ce4-a56a-24b1e68e99a9n%40googlegroups.com.

--

An eye for eye only ends up making the whole world blind.  ~Mahatma Gandhi

[Werner Keil]

I think that's a good idea, why don't you start one, something like "MP Metrics architecture" or whatever you think works best.

The possible discrepancy of some implementations between MavenCentral or JCenter does not really make much of a difference, maybe Don and his colleagues can sort that out with Sonatype or JFrog, but I don't think it is really beneficial or crucial for MP metrics or this discussion ;-)

Werner

[donbo...@gmail.com]

We were just discussing on the MP Metrics community call this morning of posting our initial architecture plans in a new thread.  Coming in a few min!

Don