Experiment with Groovy.jar
newgeekorder
compiled groovy code.
I suspect I'm stretching avian a little too far however I thought the
results would be interesting and any feedback would be useful.
Using a compiled standard compiled hello world groovy code. The java
version
./java -cp test/tools.jar:test/groovy-all-1.8.6.jar:test HelloWorld
prints the expected "hello world"
unfortunately the avian version results in:
./avian -cp test/tools.jar:test/groovy-all-1.8.6.jar:test HelloWorld
java/lang/ExceptionInInitializerError
caused by: java/lang/ExceptionInInitializerError
at HelloWorld.__$swapInit (unknown line)
at HelloWorld.<clinit> (unknown line)
caused by: java/lang/NoClassDefFoundError: java/math/BigInteger
at org/codehaus/groovy/runtime/typehandling/
DefaultTypeTransformation.<clinit> (line 39)
at HelloWorld.__$swapInit (unknown line)
at HelloWorld.<clinit> (unknown line)
I've tried to explicitly put the java tools.jar on the path with no
joy.
Any hints if this could be resolved would be greatly appreciated
Thanks, Richard
Gabe Johnson
Scala+Avian would be wonderful.
> --
> You received this message because you are subscribed to the Google Groups "Avian" group.
> To post to this group, send email to av...@googlegroups.com.
> To unsubscribe from this group, send email to avian+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/avian?hl=en.
>
--
;; Gabe Johnson
;; PhD Candidate in Computational Design - Carnegie Mellon University
;; CoDe Lab: http://code.arc.cmu.edu/
;; Personal: http://six11.org/
Joel Dice
> Guys, a little experiment to see whether avian could be used to run
> compiled groovy code.
> I suspect I'm stretching avian a little too far however I thought the
> results would be interesting and any feedback would be useful.
>
>
> Using a compiled standard compiled hello world groovy code. The java
> version
>
> ./java -cp test/tools.jar:test/groovy-all-1.8.6.jar:test HelloWorld
>
> prints the expected "hello world"
>
> unfortunately the avian version results in:
>
> ./avian -cp test/tools.jar:test/groovy-all-1.8.6.jar:test HelloWorld
> java/lang/ExceptionInInitializerError
> caused by: java/lang/ExceptionInInitializerError
> at HelloWorld.__$swapInit (unknown line)
> at HelloWorld.<clinit> (unknown line)
> caused by: java/lang/NoClassDefFoundError: java/math/BigInteger
> at org/codehaus/groovy/runtime/typehandling/
> DefaultTypeTransformation.<clinit> (line 39)
> at HelloWorld.__$swapInit (unknown line)
> at HelloWorld.<clinit> (unknown line)
That's just telling you that Avian doesn't have a java.math.BigInteger
implementation. In fact it doesn't have any java.math classes at all,
since no one has contributed them. If you were to add a compatible
BigInteger implementation and try running groovy again, you'd probably hit
another NoClassDefFoundError for another class Avian doesn't have.
Avian's built-in class library has only a very small subset of the classes
in the standard JRE, and those it has are generally incomplete relative to
the standard. That means it's unlikely that a non-trivial Java app such
as Groovy will work with it as-is.
If you want to get such an app working on Avian, you've got two choices:
1. Use the OpenJDK class library instead of the built-in one. This will
almost certainly be the easiest way. See readme.txt for details.
2. Build the app from source using the Avian class library as the boot
classpath (e.g. javac -Xmaxerrs 10000
-Xbootclasspath:../avian/build/linux-x86_64/classpath $(find . -name
'*.java')), let the compiler tell you what's missing, and implement
those things. Some apps might only need a few extra classes and
methods. Others will need an enormous number of them. Dynamic
languages like Groovy might be especially challenging if they rely
heavily on reflection, in which case the compiler might not be able
to tell you about all the classes needed at runtime. In that case,
you'll need to just keep testing and implementing until everything
works.
> I've tried to explicitly put the java tools.jar on the path with no
> joy.
> Any hints if this could be resolved would be greatly appreciated
>
> Thanks, Richard
>
Joel Dice
> I've wondered the same thing for other JVM-targeting languages.
> Scala+Avian would be wonderful.
I haven't tried Scala yet, but last time I checked, Clojure, JRuby,
JScheme and Jython all worked when using Avian with the OpenJDK class
library. In Clojure, you even get tail call optimization and first-class
continuations for free:
Joel Dice
> If you want to get such an app working on Avian, you've got two choices:
>
> 1. Use the OpenJDK class library instead of the built-in one. This will
> almost certainly be the easiest way. See readme.txt for details.
>
> 2. Build the app from source using the Avian class library as the boot
> classpath (e.g. javac -Xmaxerrs 10000
> -Xbootclasspath:../avian/build/linux-x86_64/classpath $(find . -name
> '*.java')), let the compiler tell you what's missing, and implement
Correction: use the -bootclasspath option, not -Xbootclasspath:
Joel Dice
> Hi,
>
> I have seen the same error with Scala when not using the OpenJDK.
> That's just telling you that Avian doesn't have a
> java.math.BigInteger
> at all,
> since no one has contributed them. ï¿œIf you were to add a
> BigInteger implementation and try running groovy again, you'd
> probably hit
> another NoClassDefFoundError for another class Avian doesn't
> have.
>
> never actually used, would it make sense if the Avian created stubs for them
> when they are looked up?
I doubt that would get us very far. The JIT compiler compiles methods
lazily only as they are used, so if it starts complaining that classes
aren't found, they're probably classes that are used somehow by the method
being compiled (e.g. argument type, method call, or new instance
creation).
What problem are we trying to solve here? Is OpenJDK not a good option
because it's big and monolithic? Or because of licensing issues? Or
some other reason?
If we want something that's smaller and/or more liberally licensed, but
still more complete than the Avian library, maybe it makes sense to do
something like RoboVM and make Avian compatible with the Android/Harmony
class library -- or maybe a hybrid where the Avian library is supplemented
with the Android one.
Joel Dice
> Ok.
It's absolutely ok :) I've merged the request. Thanks so much for
working on this.
> No big plans to re-implement the whole JDK, just the bare minimum to get the
> REPL working.
>
> By the way, I think I have hit a bug in the implementation of one of the
> existing classes in Avian:
>
> ./avian -cp/home/soc/Entwicklung/scala/build/pack/lib/scala-library.jar:/home/soc/Entw
> icklung/scala/build/pack/lib/scala-reflect.jar:/home/soc/Entwicklung/scala/
> build/pack/lib/scala-compiler.jar 'scala.tools.nsc.MainGenericRunner'
> -usejavacp
> java/lang/ExceptionInInitializerError
> � at scala/tools/nsc/settings/StandardScalaSettings$class.$init$ (line 33)
> � at scala/tools/nsc/settings/MutableSettings.<init> (line 19)
> � at scala/tools/nsc/GenericRunnerSettings.<init> (line 10)
> � at scala/tools/nsc/GenericRunnerCommand.<init> (line 17)
> � at scala/tools/nsc/MainGenericRunner.process (line 45)
> � at scala/tools/nsc/MainGenericRunner$.main (line 103)
> caused by: java/io/EOFException
> � at java/util/zip/InflaterInputStream.read (line 54)
> � at java/util/zip/ZipFile$1.read (line 120)
> � at java/util/zip/InflaterInputStream.read (line 39)
> � at java/util/zip/ZipFile$1.read (line 106)
> � at java/util/Properties$Parser.parse (line 106)
> � at scala/util/PropertiesTrait$class.quietlyDispose (line 43)
> � at scala/util/PropertiesTrait$class.scalaProps (line 37)
> � at scala/tools/nsc/Properties$.scalaProps$lzycompute (line 10)
> � at scala/util/PropertiesTrait$class.scalaPropOrNone (line 65)
> � at scala/util/PropertiesTrait$class.$init$ (line 77)
> � at scala/tools/nsc/Properties$.<init> (line 10)
> � at scala/tools/nsc/settings/StandardScalaSettings$class.$init$ (line 33)
> � at scala/tools/nsc/settings/MutableSettings.<init> (line 19)
> � at scala/tools/nsc/GenericRunnerSettings.<init> (line 10)
> � at scala/tools/nsc/GenericRunnerCommand.<init> (line 17)
> � at scala/tools/nsc/MainGenericRunner.process (line 45)
> � at scala/tools/nsc/MainGenericRunner$.main (line 103)
>
> I don't really see what's wrong with it, though ...
>
> Any ideas?
No ideas at the moment, but I'll take a closer look when I have a chance.
> PS: Does your VM have the capability to replace certain method calls with
> intrinsics (e. g. Integer.bitCount with POPCNT)?
Yes. See the intrinsic function in compile.cpp. To make POPCNT available
at that level, you'll need to add a new method to the Compiler interface
in compiler.h, and an implementation in compiler.cpp (which should defer
to a "thunk" on platforms which don't have that instruction). See e.g.
Compiler::fabs for an example of how this works.
Joel Dice
reproduce it yet. When I used the same command as you did, I first ran
into this:
java/lang/NoSuchFieldError: separatorChar C not found in java/io/File
at scala/reflect/io/Path.<init> (line 76)
...
which I addressed by adding a separatorChar field to java.io.File, but
then I hit
java/lang/NoClassDefFoundError: java/nio/charset/Charset
at scala/io/Codec$.<init> (line 80)
...
which will take considerably more work to address. How did you get past
these issues? Are you using the master branch of github.com:scala/scala?
Joel Dice
> Yes, I'm on master. I guess it is possible that I will also hit these issues
> sooner or later, e. g. that just the order in which they appear is
> different.
Hmm, I wonder why I'm not hitting the InflaterInputStream issue. Does it
happen every time for you? If so, would you mind uploading your Scala
jars somewhere so I could test with them myself?
Joel Dice
> Yes. �See the intrinsic function in compile.cpp. �To make POPCNT
> available
> at that level, you'll need to add a new method to the Compiler
> interface
> in compiler.h, and an implementation in compiler.cpp (which
> should defer
> to a "thunk" on platforms which don't have that instruction).
> �See e.g.
> Compiler::fabs for an example of how this works.
>
>
Not really, but I guess you could call
Architecture::planSource(BinaryOperation, ...) to find out if a thunk is
needed and then just not emit an intrinsic if so. For example, in the
intrinsic function:
...
if (MATCH(methodName(t, target), "bitCount")
and MATCH(methodSpec(t, target), "(I)I"))
{
bool thunk;
uint8_t firstTypeMask;
uint64_t firstRegisterMask;
t->arch->planSource
(BitCount, 4, &firstTypeMask, &firstRegisterMask, 4, &thunk);
if (thunk == 0) {
frame->pushInt(c->bitCount(4, frame->popInt()));
return true;
}
}
...
If we could also move that pattern into a function if it becomes useful
elsewhere. Come to think of it, that's probably how we should have
handled many of the intrinsics rather than requiring thunks written in
C++.
> How does your code detect whether the CPU is capable to execute the
> instruction in the first place (POPCNT was introduced at the same time as
> SSE 4.2 for example)?
See useSSE in x86.cpp. Currently we just check for SSE2, but we could
check for other extensions using the detectFeature function (implemented
in x86.S).
Joel Dice
> implementation into Avian's classlib? E. g. keeping it as small as possible,
> but having a lot higher chance to run applications without requiring a full
> JDK? I'm thinking of either Apache's or Google's stuff, depending on what's
> newer. Of course it is impressively easy to bundle it with OpenJDK, but I
> think having a bit more coverage of the core classes would tremendously
> improve the out-of-the-box experience.
> One good thing would be that the tests could also be imported from
> Apache/Google.
>
> What do you think?
I'd prefer not to import any external code into the Avian repository for
two reasons:
1. I don't want to maintain a fork of third-party code, especially when
it's already being maintained perfectly well by the upstream developers.
2. I'd prefer to stick with one, easy-to-understand license.
However, I think it would be a great idea to make Avian compatible with
the Android library (or at least the parts we need), as long as we do it
without actually importing and maintaining the code. I think the way we
did it with OpenJDK has been very successful in terms of keeping the
maintenance and license boundaries clear, so I'd like to stick with that
model.
Note that the Apache Harmony project is officially defunct, so it looks
like the Android library is the only viable option. Alternatively,
someone could volunteer to maintain the abandoned Harmony code (or the
subset we care about, e.g. http://wiki.apache.org/harmony/LUNI), but I
would still prefer to consider that a separate project from Avian.
Another advantage to that approach is that it could be used by other VMs
besides Avian.