Lessons from Metrics breakage

[Ken Finnigan]

I know that there's a plugin to prevent API breakages in minor releases, which Emily is adding, but I wonder if we need something from a more cross spec perspective?

We already require there to be at least one implementation of an MP specification passing the TCK before it can be officially released.

I wonder whether we need a similar kind of validation gate at the platform/umbrella level as well. Namely that an MP platform release isn't released until there is at least one runtime that contains implementations of all the specifications and passes all the TCKs.

Though the API breakage situation should be fixed with the plugin, it's currently still a manual process to ensure that if a spec introduces a change, any spec that depends on it is aware of the change and releases any fixes/changes that might be necessary to support that change. In this situation, there can be implementations that pass each spec TCK separately but if they're from different implementors then it's possible the joining of the two implementations results in a previously unseen error.

Thoughts?

Ken

[Emily Jiang]

I think we should have umbrella level TCKs (aka Java/Jakarta EE platform TCKs) to verify the inter spec ecosystem.

Thanks

Emily

[Ken Finnigan]

I agree that an umbrella TCK is needed, but that's not what I'm asking for right now. From my perspective, a TCK is only of value if we require a runtime to pass it before a release, hence this request as to whether everyone thinks a requirement like that is worthwhile.

If everyone agrees that having at least one runtime implements an MP umbrella release before it's official is required, afterward we can then discuss an umbrella wide TCK as part of that as well.

--
You received this message because you are subscribed to the Google Groups "Eclipse MicroProfile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/21358887-c3a6-4758-893e-81a2050005cf%40googlegroups.com.

[Jan Martiska]

I think one fundamental problem that might need to be addressed somehow, if we want to catch these binary incompatibilities like we had with Metrics, is that if we increase a version of one spec (Metrics in this case) and leave another spec (Fault Tolerance) unchanged, and that FT spec is compiled against the previous version of Metrics, we could hit these binary incompatibilies.

And an umbrella-wide TCK might, or might NOT, reveal this problem. That depends on whether the affected code (code that will stop working) is located in the TCK itself, or in the 'old' API that was compiled against something that is not backward compatible. In the FT case I think the problem would NOT be revealed, because the offending code is in the TCK itself: https://github.com/eclipse/microprofile-fault-tolerance/blob/2.0.2/tck/src/main/java/org/eclipse/microprofile/fault/tolerance/tck/metrics/util/MetricGetter.java#L343 so if this line had been in an umbrella TCK instead, it would then have been compiled against Metrics 2.1 API and the tests would have passed!!

I think the only way to ensure compatibility would be to require that all specs and TCKs in an umbrella depend only on specs that are in the same umbrella too. But that is a bit problematic if we sometimes want to increase just some specs, and leave the rest without update.

And even if we did this, we still wouldn't be sure that applications compiled against and older umbrella will continue working with the new umbrella without recompilation. But this is more of a problem in the world of Jakarta EE than for microservices, because with traditional app servers it is normal to have thin JARs that you just deploy on a newer version of the server without recompiling it (in which case it could break due to binary incompatibilities). In the microservices world you usually have to recompile to application, which mitigates the problem.

Jan

On Monday, October 28, 2019 at 9:01:20 PM UTC+1, Ken Finnigan wrote:

[Kevin Sutter]

Requiring a compatible implementation at the platform level is sounding more and more like the Jakarta EE process...  :-)  We need the Spec, API, TCK, and Compatible Impl to ship a new version of a Jakarta EE spec.  Are we sure we want to go that far?

I had very similar discussions with Ivar Grimstad this past week at EclipseCon.  He actually likes that our MP Platform is just a collection of the component Specs.  He would actually like to see something similar for the Jakarta EE platform, but he acknowledges that there are many interdependencies that need to be spelled out in the Jakarta EE Platform spec.  Anyway, I don't want to dive into the Jakarta EE black hole...

Reading through Jan's response...  Maybe we need a requirement that all components need to run their TCKs against any updates Components in a given release?  It sounds like the FT TCK would have detected this problem if it was run against the updated Metrics 2.1.  So, instead of requiring a Compatible Impl and corresponding platform TCK, maybe we need to focus on the component TCK executions?

-- Kevin