JavaCompilerTool
spamB...@agile-it.com
I'm trying to get JavaCompilerTool to work. This is in Mustang Beta 2
(Java 1.6.0)
Does anyone have any experience about how stable and reliable this is
at the present time?
I have the impression that they recently released a new version with
numerous changes
in method signatures and that sort of thing. Is this correct?
If so, are there any up-to-date examples anywhere? All the ones I find
on the web seem
to be for earlier, incompatible versions of the API.
Specifically I want to compile and load a class, entirely
memory-to-memory with no files.
I can my example code to compile, and run JavaCompilerTool, which
accepts my
generated code and reports success. But I don't know how to get the
class out of
it. I think that it is actually just throwing the byte code away,
because I didn't tell it
where to put it (I defaulted nearly everything). I'd like an example
showing how to
capture and load the output without writing files.
Thanks
jim
Piotr Kobzda
> If so, are there any up-to-date examples anywhere? All the ones I find
> on the web seem
> to be for earlier, incompatible versions of the API.
http://groups.google.com/group/pl.comp.lang.java/msg/d37010d1cce043d0
piotr
spamB...@agile-it.com
I had seen your example before, and was trying to use it.
It's obviously just what I need, but I have some kind of
issue with configuration or versions.
I downloaded Mustang, i.e. JDK 6 Beta 2, from
http://java.sun.com/javase/downloads/ea.jsp
I took the "Windows online installation" which gave me file
jdk-6-beta2-windows-i586-iftw.exe
I followed the link in the Readme to the API docs, at
http://java.sun.com/javase/6/docs/api/index.html
The code I downloaded and these docs seem to agree with each
other, but not with your example.
The problems are all in javax.tools.
DiagnosticMessage and WrapperJavaFileManager do not exist.
SimpleJavaFileObject has only one constructor, which takes
(URI uri, Kind kind) but you call it with (String, Kind)
SimpleJavaFileObject has no lengthInBytes() method, either
in the code or the doc; you have an override of that.
My ToolProvider has getSystemJavaCompilerTool() but you call
ToolProvider.defaultJavaCompiler()
etc etc - many more.
Is it obvious to you where the disconnect is?
THanks for any help you can give
Jim
Piotr Kobzda
> The problems are all in javax.tools.
Agreed. Java 6 is still in beta 2 stage and is under rapid development.
In particular javax.tools API changed a lot (mostly refactoring)
during last months.
> Is it obvious to you where the disconnect is?
Yes, it is.
My example was developed using:
java full version "1.6.0-beta2-b73"
The recent version is:
java full version "1.6.0-beta2-b86"
I have refactored (mostly) my example for recent beta version of Java 6,
the result is attached bellow.
HTH,
piotr
--
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import javax.tools.Diagnostic;
import javax.tools.DiagnosticCollector;
import javax.tools.FileObject;
import javax.tools.ForwardingJavaFileManager;
import javax.tools.JavaCompilerTool;
import javax.tools.StandardJavaFileManager;
import javax.tools.JavaCompilerTool.CompilationTask;
import javax.tools.JavaFileManager;
import javax.tools.JavaFileObject;
import javax.tools.JavaFileObject.Kind;
import javax.tools.SimpleJavaFileObject;
import javax.tools.ToolProvider;
public class OnTheFlyInRAM {
static JavaFileObject generateJavaSourceCode() {
final String source =
"package just.generated;\n" +
"public class Hello {\n" +
"public static void main(String... args) {\n" +
"System.out.println(new Object() {\n" +
"public String toString() {\n" +
"return \"just hello!\";\n" +
"}\n" +
"});\n" +
"}\n" +
"}";
return new SimpleJavaFileObject(toURI("Hello.java"), Kind.SOURCE) {
@Override
public CharSequence getCharContent(boolean
ignoreEncodingErrors)
throws IOException, IllegalStateException,
UnsupportedOperationException {
return source;
}
};
}
static class RAMJavaFileObject extends SimpleJavaFileObject {
RAMJavaFileObject(String name, Kind kind) {
super(toURI(name), kind);
}
ByteArrayOutputStream baos;
@Override
public CharSequence getCharContent(boolean ignoreEncodingErrors)
throws IOException, IllegalStateException,
UnsupportedOperationException {
throw new UnsupportedOperationException();
}
@Override
public InputStream openInputStream() throws IOException,
IllegalStateException, UnsupportedOperationException {
return new ByteArrayInputStream(baos.toByteArray());
}
@Override
public OutputStream openOutputStream() throws IOException,
IllegalStateException, UnsupportedOperationException {
return baos = new ByteArrayOutputStream();
}
}
public static void main(String[] args) throws Exception {
JavaCompilerTool compiler =
ToolProvider.getSystemJavaCompilerTool();
final Map<String, JavaFileObject> output =
new HashMap<String, JavaFileObject>();
DiagnosticCollector<JavaFileObject> diagnostics =
new DiagnosticCollector<JavaFileObject>();
JavaFileManager jfm = new
ForwardingJavaFileManager<StandardJavaFileManager>(
compiler.getStandardFileManager(diagnostics)) {
@Override
public JavaFileObject getJavaFileForOutput(Location location,
String name,
Kind kind,
FileObject sibling) throws IOException {
JavaFileObject jfo = new RAMJavaFileObject(name, kind);
output.put(name, jfo);
return jfo;
}
};
CompilationTask task = compiler.getTask(
null, jfm, diagnostics, null, null,
Arrays.asList(generateJavaSourceCode()));
if (! task.getResult()) {
for(Diagnostic dm : diagnostics.getDiagnostics())
System.err.println(dm);
throw new RuntimeException("Compilation failed");
}
System.out.println("generated classes: "+ output.keySet());
ClassLoader cl = new ClassLoader() {
@Override
protected Class<?> findClass(String name) throws
ClassNotFoundException {
JavaFileObject jfo = output.get(name);
if (jfo != null) {
byte[] bytes = ((RAMJavaFileObject)
jfo).baos.toByteArray();
return defineClass(name, bytes, 0, bytes.length);
}
return super.findClass(name);
}
};
Class<?> c = Class.forName("just.generated.Hello", false, cl);
c.getMethod("main", String[].class)
.invoke(null, new Object[] {args});
}
private static URI toURI(String name) {
try {
return new URI(name);
} catch (URISyntaxException e) {
throw new RuntimeException(e);
}
}
}
spamB...@agile-it.com
That solved it! Thank you so much.
For others who may try it: you still need to include tools.jar in your
classpath
(or for eclipse, as an external library). Other than that, Piotr's
latest example
code worked straight off in the current Beta 2.
Thanks again
jim
spamB...@agile-it.com
Earlier in this thread, Piotr Kobzda reposted his code for compiling
on-the-fly and in-RAM (no use of disk files), updated for
Java 1.6.0 Beta 2 Build 86
I needed to take it one step further: to work incrementally, so a
class compiled in one compiler-task would be callable from a class
generated later and compiled in a separate, later task. In Piotr's
original version this only worked if the classes were compiled in
the same task.
I extended Piotr's JavaFileManager to support additional things the
compiler needed, mainly the list() method when getting the 'classPath'.
The code seems to work okay. It is posted below.
I'd be very interested in comments and criticisms about the approach
taken, & especially how likely it is to be robust as time goes on.
Thanks all (and 'specially Piotr),
Jim Goodwin
package com.mak.jcttest;
import java.io.*;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.*;
import java.util.Map.Entry;
import javax.tools.*;
import javax.tools.JavaCompilerTool.CompilationTask;
import javax.tools.JavaFileObject.Kind;
/*
* Demo of on-the-fly, all-in-RAM compilation (no disk files used).
* Based on an example by Piotr Kobzda at
*
http://groups.google.com/group/pl.comp.lang.java/msg/d37010d1cce043d0
*
* This demo modifies Piotr's code to work incrementally. Each class is
* compiled on-the-fly all-in-RAM and in its own compilation unit.
Newer
* classes can call or reference older classes.
*
* The intended application is a custom scripting language, where bits
* of script arrive one at a time and cannot be batched up. We want to
* compile each one as it arrives, and be able to use it at once. Also,
* each new bit must also be able to call any of the
previously-compiled
* bits.
*
* The demo compiles two classes, Hello1 and Hello2. Hello1 calls
* Hello2. Hello2 is compiled first, in one compiler task. Then Hello1
* is compiled, in another compiler task. Finally Hello1 is loaded and
* run.
*
* Written and debugged against Java 1.6.0 Beta 2 build 86, in Eclipse
* 3.2 Jim Goodwin July 25 2006
*/
public class OnTheFlyInRAMIncrementally {
// Source for both test classes. They go in package
// "just.generated"
public static final String SRC_Hello1 = "package just.generated;\n"
+ "public class Hello1 {\n"
+ " public static void main(String... args) {\n"
+ " System.out.println(new Hello2()); \n" + "}}\n";
public static final String SRC_Hello2 = "package just.generated;\n"
+ "public class Hello2 {\n"
+ " public String toString() {\n"
+ " return \"just hello!\";\n}}\n";
public static void main(String[] args) throws Exception {
JavaCompilerTool compiler = ToolProvider
.getSystemJavaCompilerTool();
// A map of from class names to the RAMJavaFileObject that holds
// the compiled-code for that class. This is the cache of
// compiled classes.
Map<String, JavaFileObject> output = new HashMap<String,
JavaFileObject>();
// A loader that searches our cache first.
ClassLoader loader = new RAMClassLoader(output);
DiagnosticCollector<JavaFileObject> diagnostics = new
DiagnosticCollector<JavaFileObject>();
// Create a JavaFileManager which uses our DiagnosticCollector,
// and creates a new RAMJavaFileObject for the class, and
// registers it in our cache
StandardJavaFileManager sjfm = compiler
.getStandardFileManager(diagnostics);
JavaFileManager jfm = new RAMFileManager(sjfm, output, loader);
// Create source file objects
SourceJavaFileObject src1 = new SourceJavaFileObject("Hello1",
SRC_Hello1);
SourceJavaFileObject src2 = new SourceJavaFileObject("Hello2",
SRC_Hello2);
// Compile Hello2 first. getResult() causes it to run.
CompilationTask task2 = compiler.getTask(null, jfm,
diagnostics, null, null, Arrays.asList(src2));
if (!task2.getResult()) {
for (Diagnostic dm : diagnostics.getDiagnostics())
System.err.println(dm);
throw new RuntimeException("Compilation of task 2 failed");
}
// Now compile Hello1, in its own task
CompilationTask task1 = compiler.getTask(null, jfm,
diagnostics, null, null, Arrays.asList(src1));
if (!task1.getResult()) {
for (Diagnostic dm : diagnostics.getDiagnostics())
System.err.println(dm);
throw new RuntimeException("Compilation of task 1 failed");
}
// Traces the classes now found in the cache
System.out.println("\ngenerated classes: " + output.keySet());
// Load Hello1.class out of cache, and capture the class object
Class<?> c = Class.forName("just.generated.Hello1", false,
loader);
// Run the 'main' method of the Hello class.
c.getMethod("main", String[].class).invoke(null,
new Object[] { args });
}
/*
* Help routine to convert a string to a URI.
*/
static URI toURI(String name) {
try {
return new URI(name);
} catch (URISyntaxException e) {
throw new RuntimeException(e);
}
}
}
/*
* A JavaFileObject class for source code, that just uses a String for
* the source code.
*/
class SourceJavaFileObject extends SimpleJavaFileObject {
private final String classText;
SourceJavaFileObject(String className, final String classText) {
super(OnTheFlyInRAMIncrementally.toURI(className + ".java"),
Kind.SOURCE);
this.classText = classText;
}
@Override
public CharSequence getCharContent(boolean ignoreEncodingErrors)
throws IOException, IllegalStateException,
UnsupportedOperationException {
return classText;
}
}
/*
* A JavaFileManager that presents the contents of the cache as a file
* system to the compiler. To do this, it must do four things:
*
* It remembers our special loader and returns it from getClassLoader()
*
* It maintains our cache, adding class "files" to it when the compiler
* calls getJavaFileForOutput
*
* It implements list() to add the classes in our cache to the result
* when the compiler is asking for the classPath. This is the key
trick:
* it is what makes it possible for the second compilation task to
* compile a call to a class from the first task.
*
* It implements inferBinaryName to give the right answer for cached
* classes.
*/
class RAMFileManager extends
ForwardingJavaFileManager<StandardJavaFileManager> {
private final Map<String, JavaFileObject> output;
private final ClassLoader ldr;
public RAMFileManager(StandardJavaFileManager sjfm,
Map<String, JavaFileObject> output, ClassLoader ldr) {
super(sjfm);
this.output = output;
this.ldr = ldr;
}
public JavaFileObject getJavaFileForOutput(Location location,
String name, Kind kind, FileObject sibling)
throws IOException {
JavaFileObject jfo = new RAMJavaFileObject(name, kind);
output.put(name, jfo);
return jfo;
}
public ClassLoader getClassLoader(JavaFileManager.Location
location) {
return ldr;
}
@Override
public String inferBinaryName(Location loc, JavaFileObject jfo) {
String result;
if (loc == StandardLocation.CLASS_PATH
&& jfo instanceof RAMJavaFileObject)
result = jfo.getName();
else
result = super.inferBinaryName(loc, jfo);
return result;
}
@Override
public Iterable<JavaFileObject> list(Location loc, String pkg,
Set<Kind> kind, boolean recurse) throws IOException {
Iterable<JavaFileObject> result = super.list(loc, pkg, kind,
recurse);
if (loc == StandardLocation.CLASS_PATH
&& pkg.equals("just.generated")
&& kind.contains(JavaFileObject.Kind.CLASS)) {
ArrayList<JavaFileObject> temp = new ArrayList<JavaFileObject>(
3);
for (JavaFileObject jfo : result)
temp.add(jfo);
for (Entry<String, JavaFileObject> entry : output
.entrySet()) {
temp.add(entry.getValue());
}
result = temp;
}
return result;
}
}
/**
* A JavaFileObject that uses RAM instead of disk to store the file. It
* gets written to by the compiler, and read from by the loader.
*/
class RAMJavaFileObject extends SimpleJavaFileObject {
ByteArrayOutputStream baos;
RAMJavaFileObject(String name, Kind kind) {
super(OnTheFlyInRAMIncrementally.toURI(name), kind);
}
@Override
public CharSequence getCharContent(boolean ignoreEncodingErrors)
throws IOException, IllegalStateException,
UnsupportedOperationException {
throw new UnsupportedOperationException();
}
@Override
public InputStream openInputStream() throws IOException,
IllegalStateException, UnsupportedOperationException {
return new ByteArrayInputStream(baos.toByteArray());
}
@Override
public OutputStream openOutputStream() throws IOException,
IllegalStateException, UnsupportedOperationException {
return baos = new ByteArrayOutputStream();
}
}
/**
* A class loader that loads what's in the cache by preference, and if
* it can't find the class there, loads from the standard parent.
*
* It is important that everything in the demo use the same loader, so
* we pass this to the JavaFileManager as well as calling it directly.
*/
final class RAMClassLoader extends ClassLoader {
private final Map<String, JavaFileObject> output;
RAMClassLoader(Map<String, JavaFileObject> output) {
this.output = output;