fastutil Java 9 modularity compatibility

[Udo Kohlmeyer]

Hi there,

I work on Apache Geode, which uses the fastutil library. We are currently working on a feature where we use different classloaders to load the project and associated libraries. 

What we found is that the fastutil projects structure of fastutil-core, fastutil-extra and fastutil is causing problems because of the package splitting. I imagine (and I have not tested this out) that Java 9 modularity will have a problem with package splitting as well.

Is there any thought to have a Java 9+ compliant library for fastutil, or maybe have a maven artifact of "fastutil-all" that will combine the 3 jars into a single artifact?

--Udo

[Sebastiano Vigna]

Sorry for the late answer.

I'm still trying to understand what people would prefer. Maybe the double solution is the best. We have fastutil.jar, as always, with everything in, and then fastutil-core and fastutil-extra which are inclusive (i.e., fastutil-extra contains fastutil-core). This should solve all the compatibility problems, but in turn it will create problems if people include different packages. I really don't know what to do—suggestions are welcome.

[Dan Smith]

I also work on Apache Geode and Udo pointed me at this thread.

I think publishing a jar that is inclusive and contains fastutil-extras, fastutil-core, sounds the best to me. The alternative would be to move some classes to different packages, but that would cause more compatibility headaches.

I would lean slightly towards creating a new jar called fastutil-all or something like that to avoid confusion for users upgrading to the newer fastutil version. But adding fastutil-core to fastutil extras would be fine too.

-Dan

---

From: fast...@googlegroups.com <fast...@googlegroups.com> on behalf of Sebastiano Vigna <sebastia...@gmail.com>
Sent: Monday, June 21, 2021 12:53 AM
To: fastutil <fast...@googlegroups.com>
Subject: Re: fastutil Java 9 modularity compatibility

 

--
You received this message because you are subscribed to a topic in the Google Groups "fastutil" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fastutil/pTZ2LgcvqCQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to fastutil+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fastutil/fc4f7089-aca5-45d3-b3e3-a34ac5b239edn%40googlegroups.com.

[Sebastiano Vigna]

Il giorno martedì 29 giugno 2021 alle 19:18:41 UTC+2 Dan Smith ha scritto:

I think publishing a jar that is inclusive and contains fastutil-extras, fastutil-core, sounds the best to me. The alternative would be to move some classes to different packages, but that would cause more compatibility headaches.

I realize there are  issues, but if you guys could tell me _specifically_ what is the problem, we might try to adjust it. Note that Java 9 permits explicitly cross-package modularity, so that won't be an issue.

My hope was that publishing the OSGi-split packages someone would detect problem and suggests solutions (e.g., something wrong I've done in the splitting), but up to know unfortunately nobody did the last part. If you know anyone familiar with OSGi, please let me know.

Unless of course the problem is not OSGi  but something else. 

[Udo Kohlmeyer]

Hi there Sebastiano,

The specific problem is that package splitting is causing problems when using different ClassLoaders to load the jars. I don't want to get into specifics, but in short, we are working on a feature that gives us ClassLoader isolation between different library and code versions. Think, Application containers. We have written a plugin that takes a gradle file and converts it into a JBoss Modules module.xml file. In this situation each dependency is loaded in it's own JBoss Module, which allows for the loading of different versions of the same dependency without those versions affecting each other. This is achieved by loading each dependency into its own ClassLoader. (as to why can be a different discussion, but not relevant to the requirement)

With this approach fastutil, fastutil-extra, fastutil-core are all loaded into their own ClassLoader, which causes problems when trying to access anything in the "protected" classifier across different ClassLoaders.

After reading your comment  "Java 9 permits explicitly cross-package modularity"  I went back to read up on this subject, maybe I missed something. What I read is, Java 9 allows work-arounds for this exact situation there is still a recommendation NOT to split packages across Java 9 modules. Most all links that I found and read spoke about moving to a version of a library that supports Java 9 modularity or to manually merge the jars into 1, which is not an option for the Project.

Either way, we have found an interim work around, but I would still request if it were possible to provide a "fastutil-all" artifact that rolls all jars into a single jar.

I unfortunately do not work in OSGI and cannot offer any advice when it comes to that. But JBoss Modules allows for the runtime loading/unloading of libraries and different versions, similar to what OSGI would, just much simpler. 

Thank you

--Udo

[Sebastiano Vigna]

Il giorno mercoledì 7 luglio 2021 alle 00:43:38 UTC+2 ukohl...@gmail.com ha scritto:

Hi there Sebastiano,

The specific problem is that package splitting is causing problems when using different ClassLoaders to load the jars. I don't want to get into specifics, but in short, we are working on a feature that gives us ClassLoader isolation between different library and code versions. Think, Application containers. We have written a plugin that takes a gradle file and converts it into a JBoss Modules module.xml file. In this situation each dependency is loaded in it's own JBoss Module, which allows for the loading of different versions of the same dependency without those versions affecting each other. This is achieved by loading each dependency into its own ClassLoader. (as to why can be a different discussion, but not relevant to the requirement)

I see.

At this point, I think the only solution keeping at least partially people happy is to get back to the fastutil.jar we all know and love, but add a fastutil-core.jar artifact for people who just need the basic types and have a problem with the jar size. The downside is that if by different dependency paths you end up using both fastutil-core.jar and fastutil.jar you'll get duplicate classes. This can be fixed by excluding manually fastutil-core.jar in the POM, so I think it is a minor nuisance.

How does that seem to you? 

[Sebastiano Vigna]

BTW: rereading the comments above from you Geode people, maybe it would be actually better to do the following:

fastutil-core.jar: as of now

fastutil-extra.jar: as of now PLUS fastutil-core.jar

fastutil.jar: the known and loved fastutil.

Then people can eliminate redundant dependencies at the POM level. And more or less everybody should be happy. And all the infrastructure I painfully and painstakingly put together to split correctly across packages would still be useful.

[Udo Kohlmeyer]

Hi there,

Thank you for the response. I don't want to make your life more complicated. If you separated the code into different jars for a specific functional reason then I don't really want to negate that.

Would it be possible to generate a fastutil-all.jar? Leave all the others as they currently are, and just add another jar?

--Udo

--
You received this message because you are subscribed to the Google Groups "fastutil" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fastutil+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fastutil/fb3badab-c32a-475f-878a-e2d28731f0ean%40googlegroups.com.

[Sebastiano Vigna]

Il giorno mercoledì 7 luglio 2021 alle 08:10:45 UTC+2 ukohl...@gmail.com ha scritto:

Hi there,

Thank you for the response. I don't want to make your life more complicated. If you separated the code into different jars for a specific functional reason then I don't really want to negate that.

Would it be possible to generate a fastutil-all.jar? Leave all the others as they currently are, and just add another jar?

fastutil 8.5.5 will be distributed as a traditional fastutil.jar, and as a fastutil-core.jar with basic classes. They overlap, so if you get both of them in your dependencies you should exclude fastutil-core.jar.

Would that work for you? You can test the distribution on github.

seba 

[Udo Kohlmeyer]

Hi there Seba,

Thank you for the heads up on this. I'll test this out and get back to you.

--Udo

--
You received this message because you are subscribed to a topic in the Google Groups "fastutil" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fastutil/pTZ2LgcvqCQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to fastutil+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fastutil/0614c773-e5f1-43a5-a2fb-0c22e3f7cfcfn%40googlegroups.com.