Match classes by Classloader

[Ernesto José Pérez García]

Hi all !

I just wanted to know if there is a way of filtering the classes from a given ClassLoader or set of ClassLoaders.

Specifically, in my case I have two ClassLoaders, a Parent ClassLoader and a Child ClassLoader, and I would like to search only classes loaded with the Child one.

Any idea?

Thanks!

[Luke Hutchison]

Hi Ernesto,

Yes, try FastClasspathScanner#ignoreParentClassloaders() before calling #scan().

It's supposed to work, but I have never needed it yet -- let me know if it works for you!

You can also call FastClasspathScanner#overrideClassloaders() , if that is more convenient.

Note though that any time you override the classpath or classloader order (including ignoring parent classloaders), you can get a lot of weird effects if a class is defined multiple times on the classpath (in different classloaders), e.g. not being able to cast a class to its superclass. The reason for this is that the order in which FCS returns classes (and loads them, if you are using a MatchProcessor) may differ from the order in which the system resolves classes -- so you might end up loading the same class twice, or you might get a cached class of a given name that was already loaded, rather than the one you were expecting.

Luke

--
You received this message because you are subscribed to the Google Groups "FastClasspathScanner-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fastclasspathscanner-users+unsub...@googlegroups.com.
To post to this group, send email to fastclasspathscanner-users@googlegroups.com.
Visit this group at https://groups.google.com/group/fastclasspathscanner-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/fastclasspathscanner-users/0a294739-90b7-4cca-bd5a-c88ce3f97904%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ernesto José Pérez García]

Hi Luke!

First of all, thank you very much for your quick response!

I have tried to ignore the Parent Classloader as you suggested but it didn't work.

Here you can see a code snippet:

new FastClasspathScanner()
      .ignoreParentClassLoaders()
      .matchClassesImplementing(ProtocolObject.class, clazz -> {
         LOGGER.debug("Detected ProtocolObject: {}, CL: {} -> {}", clazz.getName(), clazz.getClassLoader(), clazz.getClassLoader().getParent());
      })
      .scan();

And the result of the execution is:

Detected ProtocolObject: com.ernestojpg.test.specific.protocol.MyProtocolObject1, CL: io.vertx.core.impl.IsolatingClassLoader@181181af -> sun.misc.Launcher$AppClassLoader@18b4aac2

Detected ProtocolObject: com.ernestojpg.test.specific.protocol.MyProtocolObject2, CL: io.vertx.core.impl.IsolatingClassLoader@181181af -> sun.misc.Launcher$AppClassLoader@18b4aac2

Detected ProtocolObject: com.ernestojpg.test.specific.protocol.MyProtocolObject3, CL: io.vertx.core.impl.IsolatingClassLoader@181181af -> sun.misc.Launcher$AppClassLoader@18b4aac2

Detected ProtocolObject: com.ernestojpg.test.specific.protocol.MyProtocolObject4, CL: io.vertx.core.impl.IsolatingClassLoader@181181af -> sun.misc.Launcher$AppClassLoader@18b4aac2

Detected ProtocolObject: com.ernestojpg.test.specific.protocol.MyProtocolObject5, CL: io.vertx.core.impl.IsolatingClassLoader@181181af -> sun.misc.Launcher$AppClassLoader@18b4aac2

Detected ProtocolObject: com.ernestojpg.test.specific.protocol.MyProtocolObject6, CL: io.vertx.core.impl.IsolatingClassLoader@181181af -> sun.misc.Launcher$AppClassLoader@18b4aac2

Detected ProtocolObject: com.ernestojpg.test.global.ProtocolObjectBase, CL: sun.misc.Launcher$AppClassLoader@18b4aac2 -> sun.misc.Launcher$ExtClassLoader@7d0b1d25

Detected ProtocolObject: com.ernestojpg.test.global.GlobalProtocolOBject1, CL: sun.misc.Launcher$AppClassLoader@18b4aac2 -> sun.misc.Launcher$ExtClassLoader@7d0b1d25

Detected ProtocolObject: com.ernestojpg.test.global.GlobalProtocolOBject2, CL: sun.misc.Launcher$AppClassLoader@18b4aac2 -> sun.misc.Launcher$ExtClassLoader@7d0b1d25

Detected ProtocolObject: com.ernestojpg.test.global.GlobalProtocolOBject3, CL: sun.misc.Launcher$AppClassLoader@18b4aac2 -> sun.misc.Launcher$ExtClassLoader@7d0b1d25

Detected ProtocolObject: com.ernestojpg.test.global.GlobalProtocolOBject4, CL: sun.misc.Launcher$AppClassLoader@18b4aac2 -> sun.misc.Launcher$ExtClassLoader@7d0b1d25

As you can see, the result contain classes from both, the current Classloader (io.vertx.core.impl.IsolatingClassLoader@181181af), and from its parent (sun.misc.Launcher$AppClassLoader@18b4aac2).

Note that "new FastClasspathScanner().findBestClassLoader()" returns the current Classloader (io.vertx.core.impl.IsolatingClassLoader@181181af) correctly.

Maybe the problem is with the Vert.x Classloader I'm using? It extends from a normal URLClassLoader: https://github.com/eclipse/vert.x/blob/master/src/main/java/io/vertx/core/impl/IsolatingClassLoader.java

Regards.

  Ernesto.

On Tuesday, 16 January 2018 20:19:04 UTC+1, Luke Hutchison wrote:

Hi Ernesto,

Yes, try FastClasspathScanner#ignoreParentClassloaders() before calling #scan().

It's supposed to work, but I have never needed it yet -- let me know if it works for you!

You can also call FastClasspathScanner#overrideClassloaders() , if that is more convenient.

Note though that any time you override the classpath or classloader order (including ignoring parent classloaders), you can get a lot of weird effects if a class is defined multiple times on the classpath (in different classloaders), e.g. not being able to cast a class to its superclass. The reason for this is that the order in which FCS returns classes (and loads them, if you are using a MatchProcessor) may differ from the order in which the system resolves classes -- so you might end up loading the same class twice, or you might get a cached class of a given name that was already loaded, rather than the one you were expecting.

Luke

On Tue, Jan 16, 2018 at 11:03 AM, Ernesto José Pérez García <ernes...@gmail.com> wrote:

Hi all !

I just wanted to know if there is a way of filtering the classes from a given ClassLoader or set of ClassLoaders.

Specifically, in my case I have two ClassLoaders, a Parent ClassLoader and a Child ClassLoader, and I would like to search only classes loaded with the Child one.

Any idea?

Thanks!

--
You received this message because you are subscribed to the Google Groups "FastClasspathScanner-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fastclasspathscanner-users+unsub...@googlegroups.com.

To post to this group, send email to fastclasspath...@googlegroups.com.

[Luke Hutchison]

Hi Ernesto,

Can you please call .verbose() before .scan(), and attach the result? This will show which classpath elements are being found by which classloaders.

Thanks,

Luke

[Ernesto José Pérez García]

Hi Luke,

Thank you very much for the support.

I attach the output of the program, with the verbose() option.

I also attach the source code of the test. For running it you just need to enter the following commands:

  1) mvn clean install

  2) java -jar global-module/target/global-module-1.0-SNAPSHOT.jar

Regards.

  Ernesto.

[Luke Hutchison]

Hi Ernesto,

It looks like your io.vertx.core.impl.IsolatingClassLoader actually has both the specific module and the global module added to it as URLs:

2018-01-17T12:33:20.234+0100 FastClasspathScanner ------ Finding classpath elements in ClassLoader io.vertx.core.impl.IsolatingClassLoader@5ae058df

2018-01-17T12:33:20.235+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/com/ernestojpg/specific-module/1.0-SNAPSHOT/specific-module-1.0-SNAPSHOT.jar

2018-01-17T12:33:20.235+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/io/vertx/vertx-core/3.5.0/vertx-core-3.5.0.jar

2018-01-17T12:33:20.235+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/io/netty/netty-common/4.1.15.Final/netty-common-4.1.15.Final.jar

2018-01-17T12:33:20.235+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/io/netty/netty-buffer/4.1.15.Final/netty-buffer-4.1.15.Final.jar

2018-01-17T12:33:20.235+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/io/netty/netty-transport/4.1.15.Final/netty-transport-4.1.15.Final.jar

2018-01-17T12:33:20.236+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/io/netty/netty-handler/4.1.15.Final/netty-handler-4.1.15.Final.jar

2018-01-17T12:33:20.236+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/io/netty/netty-codec/4.1.15.Final/netty-codec-4.1.15.Final.jar

2018-01-17T12:33:20.236+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/io/netty/netty-handler-proxy/4.1.15.Final/netty-handler-proxy-4.1.15.Final.jar

2018-01-17T12:33:20.236+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/io/netty/netty-codec-socks/4.1.15.Final/netty-codec-socks-4.1.15.Final.jar

2018-01-17T12:33:20.236+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/io/netty/netty-codec-http/4.1.15.Final/netty-codec-http-4.1.15.Final.jar

2018-01-17T12:33:20.236+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/io/netty/netty-codec-http2/4.1.15.Final/netty-codec-http2-4.1.15.Final.jar

2018-01-17T12:33:20.236+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/io/netty/netty-resolver/4.1.15.Final/netty-resolver-4.1.15.Final.jar

2018-01-17T12:33:20.236+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/io/netty/netty-resolver-dns/4.1.15.Final/netty-resolver-dns-4.1.15.Final.jar

2018-01-17T12:33:20.237+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/io/netty/netty-codec-dns/4.1.15.Final/netty-codec-dns-4.1.15.Final.jar

2018-01-17T12:33:20.237+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.9.0/jackson-core-2.9.0.jar

2018-01-17T12:33:20.237+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.9.0/jackson-databind-2.9.0.jar

2018-01-17T12:33:20.237+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.9.0/jackson-annotations-2.9.0.jar

2018-01-17T12:33:20.237+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/.m2/repository/io/github/lukehutch/fast-classpath-scanner/2.9.4/fast-classpath-scanner-2.9.4.jar

2018-01-17T12:33:20.237+0100 FastClasspathScanner -------- Found classpath element: /Users/eperez/Development/NetBeansProjects/fast-classpath-scanner-test/global-module/target/global-module-1.0-SNAPSHOT.jar

In the log, the scanner correctly does not ask the parent classloader sun.misc.Launcher$AppClassLoader for URLs. So in your case, the fix would be to make sure that the global module is not in the URL list for io.vertx.core.impl.IsolatingClassLoader.

However, your log brings up another interesting issue -- ignoring parent classloaders won't work if the parent classloaders' entries are also in java.class.path, since any paths found there will always be scanned. From the log, you can see that the global module was found in both IsolatingClassLoader and java.class.path:

2018-01-17T12:33:20.237+0100 FastClasspathScanner ------ Getting classpath entries from java.class.path

2018-01-17T12:33:20.238+0100 FastClasspathScanner -------- Ignoring duplicate classpath element: /Users/eperez/Development/NetBeansProjects/fast-classpath-scanner-test/global-module/target/global-module-1.0-SNAPSHOT.jar

I probably need to always fetch all URLs from all classloaders, then if parent classloaders are disabled, remove any URLs from java.class.path that are in parents but not in the child. This still wouldn't fix your situation though, since both module URLs are in the child classloader.

--
You received this message because you are subscribed to the Google Groups "FastClasspathScanner-Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fastclasspathscanner-users+unsubsc...@googlegroups.com.
To post to this group, send email to fastclasspathscanner-users@googlegroups.com.

Visit this group at https://groups.google.com/group/fastclasspathscanner-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/fastclasspathscanner-users/27252b2f-f010-4546-81cf-e4fb8152b19f%40googlegroups.com.

[Luke Hutchison]

I created https://github.com/lukehutch/fast-classpath-scanner/issues/169 to track this issue.

[Luke Hutchison]

Ernesto -- I just released version 2.10.0, which should stop classpath elements from being used if they are in java.class.path but are also listed in ignored parent classloaders.

You will still need to figure out why global-module and specific-module are being added to the child Vertx classloader, when you are only expecting specific-module -- it seems that you have a classpath specification issue somewhere.

[Ernesto José Pérez García]

That's awesome Luke,

I will take a look to the Vertx classloaders.

Thank you very much for your support!

  Ernesto.