reflecting/scanning the classes in. Jar file

[David Lee]

I need to scan a large jar file with thousands of classes in order to determine what types of new classes to generate.
I have a lot of Java code for reflection but since the end purpose is to use byte buddy on the results would it make sense to use byte buddy' Description and Type apis for this ? They seem to largely parallel the java reflection APIs ...
If so what entry points should I look at to either scan all the classes in a jar or in a package ( I know the later is not entirely possible)
It doesn't matter if I need to preload the jar - I can start with explicit class names as the 'root' of the search -- I only need a few levels of indirect dependencies,
the jars are largely service APIs with a similar structure of request and response objects, my goal is to analyze these in bulk in order to create a set of data mapping views, annotations etc,

Suggestions welcome

[Rafael Winterhalter]

Hi David,

the "Byte Buddy reflection API" how I like to call it is definetly supposed to be used for this. The advantage of using this API is that it is agnostic of loading code. When you load a class to inspect it, you need to go through a class loader which is not always a good option. Byte Buddy limits itself to describing classes without giving a possibility to, for example, invoke a method. Thus, initializers are never run on accident as there are no class loaders involved.

Other than that, Byte Buddy mimics the reflection API, up to the extend to return "null" from some methods whereas Byte Buddy normally never returns null. However, I streamline the API to be more concices. Especially generic types which I currently integrate should be much easier to read using Byte Buddy once the API is extended. Note however that the API still undergos a few changes until finalization as I have not yet covered Java's full feature set. I hope to be done at the end of the year and freeze the API afterwards.

As an entry point, I would use a TypePool which allows you to read from any location as long as you are able to map a name to a byte array. I currently plan to add native support for reading from jar files but it is quite straight forward to implement a ClassFileLocator.

You would then only need to walk the JarFile or the folder you are interested in.

Ping me back if you have any further questions.

[David Lee]

Thanks for your reply, this leads me in the right direction.

If my use case for reflection is primarily to determine what Byte Buddy operations are needed -  then using the bb reflection API's make the most sense.

As for Jar files - support for Jar *format* (not necessarily 'files) is critical to a few of my use cases and I suspect for others.

Reading files from a Jar is easy - given the Jar library in Java its a snap.

Pulling the .class files from that is not much extra work - although it does take a bit thought to get the paths to classname translations right.

Similarly writing .jar format output from individual classes   (and similar doing the reverse mapping).

I may write such a library and publish it - but its of little value on its own.

However that inst the end of the story..

Some use cases that are more difficult - or just more work - that would benefit from direct integration into BB.

Writing:

1) Manifest file - what should go in there ? Should there be references to dependencies ? licenses etc ?

2) Class Loaders -- and other extra bits --  If the generated classes are intended to be used with different classload options,

   perhaps different per class ! that information would be very useful in the jar, perhaps as an actual instance of a classloader

   and associated reference in the manifest

3) package level annotations - creating and writitng out the package-info.java 

Reading:

1-3 from above 

4) Creating a ClassLoader from a jar or set of jars - that come from a one-time source -

I have a project where the 'jar' files are stored in a database and I need to load them into the running JVM.

I currently have a custom class loader to do this - but I found its much more difficult then I thought to make a Jar Class loader

then it is to make a single class loader -- Studying the OpenJDK and Oracle source for URLClassLoader (the one one I know of that supports JAR format )

is enlightening - its very complicated, it assumes that the 'jar' files have URL's and that they exist and are accessible indefinitely - unchanged. 

This works great for file:// and http://  sources,  but no provision at all for a single in-memory blob --- 

I ended up having to register my own URL scheme and cache the binary jar and serve it up as if it were a URL resolvable resource.

Having a classloader that works directly on jar format 'blobs' would be a very useful thing.

Consider #2 above especially - if the classloader *also* is responsible for instantiating classes using specific bytebuddy classloader options -

that would solve multiple problems.    

Imagine being able to create BB classes and store them in a 'jar' along with the correct classloader options. 

Then the users of that (even if its you)   could just use the .jar and either put it into the CLASSPATH or load it with a provided classloader ...

( ideally one that is embedded into the jar itself) - and not have to manage keeping track of the classloader options or link with 

ByteBuddy at runtime (the jar could contain BB if needed)

[Rafael Winterhalter]

Hi David,
you can already safe Jar files with adding a manifest by using the

File toJar(File file, Manifest manifest) throws IOException;

method which is defined in the DynamicType interface. You can skip the manifest file though, it does not define many properties as of today but it might be important for OSGi and the like. Normally, it only contains a couple of optional properties like a main method which are easy to look up.

Package-level annotations and classes can already be read. In the upcoming version of Byte Buddy, you are also able to define or rebase package definitions. This is already implemented but not yet released. Same goes for class file locators from Jar files or from the jar system.

I think anything more is out of scope but can easily be implemented for any particular use case, especially writing user-customized ClassFileLocators and manifest files.