View this page "Jvm Bytecode Libraries"
Kresten Krab Thorup
understanding why the JVM won't accept my buggy byte code :-)
Click on http://groups.google.com/group/jvm-languages/web/jvm-bytecode-libraries
- or copy & paste it into your browser's address bar if that doesn't
work.
Rémi Forax
> Added link to my off-line verifier, I find it really useful for
> understanding why the JVM won't accept my buggy byte code :-)
>
> Click on http://groups.google.com/group/jvm-languages/web/jvm-bytecode-libraries
> work.
>
ASM has also this kind of stuff:
http://asm.ow2.org/asm32/javadoc/user/org/objectweb/asm/util/CheckClassAdapter.html
It dumps the stack and the local variable types for each bytecode
instruction.
R�mi
Peter Veentjer
jprofiler). It contains a swing based viewer. Great help to view the
bytecode.
Which bytecode library are you using? Although the asm library has its
rough edged, the mailinglist is quite good.
> --
>
> You received this message because you are subscribed to the Google
> Groups "JVM Languages" group.
> To post to this group, send email to jvm-la...@googlegroups.com.
> To unsubscribe from this group, send email to jvm-language...@googlegroups.com
> .
> For more options, visit this group at http://groups.google.com/group/jvm-languages?hl=en
> .
>
>
Rémi Forax
> One of my big helps is jclasslib from ejtechnologies (guys from
> jprofiler). It contains a swing based viewer. Great help to view the
> bytecode.
>
> Which bytecode library are you using? Although the asm library has its
> rough edged, the mailinglist is quite good.
>
When I want to produce bytecode, I use javac (the backend, I create a
tree and call the gen pass) or ASM.
ASM is really great but I'm biased, I am one of the developer
I use javap (John Gibbons did a great job to revamp it for 1.7) and the
bytecode outline eclipse plugin
(http://asm.ow2.org/eclipse/index.html) when I want to browse bytecodes.
R�mi
Matt Fowles
When I want to produce bytecode, I use javac (the backend, I create a
tree and call the gen pass) or ASM.
ASM is really great but I'm biased, I am one of the developer
Charles Oliver Nutter
> I am interested in this technique, can you provide a pointer to some
> starting point where I can see how you do this? I thought javac didn't
> expose these classes.
Yeah, I'd like to hear more as well. I would use it in at least Duby's
JVM backend, and potentially for other things. I've hacked javac
internals, but never tried to feed it valid trees and make it spit out
bytecode.
- Charlie
Matt Fowles
Charles Oliver Nutter
> Charlie~
> We use Janino for that exact purpose. I am currently toying with the idea
> of making it into a mildly optimizing compiler (simply to reduce the size of
> the emitted bytecode and as an interesting exercise). Would you find any
> value in that?
You bet your bippy I would. I've been wishing for something like "LLVM
for JVM bytecode" for a long time. I've been meaning to look at Janino
to replace our current compiler backend, but have been spending more
time on integration aspects of JRuby lately.
FWIW, we're currently putting together (with a lot of help from Subbu
Sastry) an optimizing compiler for JRuby that can do things like
inline Ruby code, propagate constants and types, eliminate dead
code...all the bits and pieces an optimizing compiler would do, if we
had an optimizing compiler backend we could reuse. The ideal situation
would be that we'd use our optimizing compiler to produce optimized
JRuby IR, and then feed that into the JVM-targeting optimizing
compiler in Janino to produce the best-possible-bytecode for our
best-possible-IR.
- Charlie
Patrick Wright
> Sastry) an optimizing compiler for JRuby that can do things like
> inline Ruby code, propagate constants and types, eliminate dead
> code...all the bits and pieces an optimizing compiler would do, if we
> had an optimizing compiler backend we could reuse. The ideal situation
> would be that we'd use our optimizing compiler to produce optimized
> JRuby IR, and then feed that into the JVM-targeting optimizing
> compiler in Janino to produce the best-possible-bytecode for our
> best-possible-IR.
Have you all looked at Soot and the related libraries from McGill
(http://www.sable.mcgill.ca/soot/)?
Charles Oliver Nutter
It's on the list...I'll have to look at it more when I get back around
to compiler work, but that could happen Any Day Now...
- Charlie
John Cowan
<hea...@headius.com> wrote:
> You bet your bippy I would. I've been wishing for something like "LLVM
> for JVM bytecode" for a long time. I've been meaning to look at Janino
> to replace our current compiler backend, but have been spending more
> time on integration aspects of JRuby lately.
Janino has been dead for about a year, but there is now a new official
maintainer (Codehaus just flipped the necessary bits a week ago), so
we can expect to see some movement. Note that Janino is deliberately
Java 1.4, so if you want generics you have to erase them yourself. I
don't consider that a serious problem for generated code.
I proposed a long time ago to add "goto" to Janino, which would remove
a huge amount of the incentive to generate bytecode directly. The
proposal was turned down, but the new moderator might be more
receptive, especially if a patch was provided. (You can do a lot of
what goto does with "do ... while (false)" and judicious use of break,
but it's messy.)
I'm not sure that making Janino do its own optimizations is really a
win: too-clever bytecode generators, as we know, can cause JITs to
pessimize the code. But I'm willing to be convinced otherwise.
--
GMail doesn't have rotating .sigs, but you can see mine at
http://www.ccil.org/~cowan/signatures
Charles Oliver Nutter
> I proposed a long time ago to add "goto" to Janino, which would remove
> a huge amount of the incentive to generate bytecode directly. The
> proposal was turned down, but the new moderator might be more
> receptive, especially if a patch was provided. (You can do a lot of
> what goto does with "do ... while (false)" and judicious use of break,
> but it's messy.)
Ideally I should never have to worry about goto; I should be able to
feed something a Java/JVM-aware CFG and know it will produce the best
possible code for me.
> I'm not sure that making Janino do its own optimizations is really a
> win: too-clever bytecode generators, as we know, can cause JITs to
> pessimize the code. But I'm willing to be convinced otherwise.
I think this is true for small cases, but certainly not for large
ones. For example, sometimes it's just damned useful to be able to say
"inline this code everywhere...no really, just do it". Or in JRuby's
case, to be able to say "treat finals as really finals, finally!" and
not compile multiple accesses as multiple accesses. There's lots of
reasons why doing some up-front optimization can help, since even the
best JVM jits don't do *everything* for us.
In JRuby's compiler, we're going to do a lot of this on a Ruby level
before feeding it to whatever bytecode generation, but ideally our
bytecode-generating backend would just translate our Ruby CFG into a
rough Java-like CFG and the backend will do additional optimization
passes to produce the best-possible bytecode.
Another optimization a good compiler backend could do for us would be
generating smaller synthetic methods if basic blocks of code appeared
frequently in a very large body. Right now, javac and friends are
pretty dumb...you feed it a big chunk of code, it puts it into a big
method. But if it were smart about splitting up blocks into synthetic
submethods, we'd have mo' betta inlining possibilities, smaller units
of code, and generally better performance.
There's a lot of opportunity here.
- Charlie
Matt Fowles
Rémi Forax
Yes, I can :)
Let me introduce Pseudo, a language that is a simpler C/Python.
It's aim is to be used in first years of computer science
to teach algorithmics.
The cool feature of this language is gradual typing.
Some samples are available here:
https://code.google.com/p/pseudo-language/source/browse/trunk/pseudo/#pseudo/samples
The grammar of the language is defined here:
https://code.google.com/p/pseudo-language/source/browse/trunk/pseudo/compiler/pseudo.ebnf
I use a parser generator that I've written with two friends,
to generate the AST from the grammar.
Here is an example of backend that takes an AST of Pseudo,
convert it to a javac AST and run javac backend on the resulting tree:
https://code.google.com/p/pseudo-language/source/browse/trunk/pseudo/compiler/src/com/googlecode/pseudo/compiler/gen/Gen.java
PS: Gen is more complex than it could be because it generates
invokedynamic call when it goes dynamic and tail call just for fun.
cheers,
R�mi
Matt Fowles
Rémi
John Cowan
<hea...@headius.com> wrote:
> On Thu, Dec 17, 2009 at 12:17 PM, John Cowan <johnw...@gmail.com> wrote:
>> I proposed a long time ago to add "goto" to Janino, which would remove
>> a huge amount of the incentive to generate bytecode directly. The
>> proposal was turned down, but the new moderator might be more
>> receptive, especially if a patch was provided. (You can do a lot of
>> what goto does with "do ... while (false)" and judicious use of break,
>> but it's messy.)
>
> Ideally I should never have to worry about goto; I should be able to
> feed something a Java/JVM-aware CFG and know it will produce the best
> possible code for me.
I don't understand this. Janino's input is Java 1.4 source code, not
a CFG; it's an embeddable compiler.
>
>> I'm not sure that making Janino do its own optimizations is really a
>> win: too-clever bytecode generators, as we know, can cause JITs to
>> pessimize the code. But I'm willing to be convinced otherwise.
>
> I think this is true for small cases, but certainly not for large
> ones. For example, sometimes it's just damned useful to be able to say
> "inline this code everywhere...no really, just do it". Or in JRuby's
> case, to be able to say "treat finals as really finals, finally!" and
> not compile multiple accesses as multiple accesses. There's lots of
> reasons why doing some up-front optimization can help, since even the
> best JVM jits don't do *everything* for us.
>
> In JRuby's compiler, we're going to do a lot of this on a Ruby level
> before feeding it to whatever bytecode generation, but ideally our
> bytecode-generating backend would just translate our Ruby CFG into a
> rough Java-like CFG and the backend will do additional optimization
> passes to produce the best-possible bytecode.
>
> Another optimization a good compiler backend could do for us would be
> generating smaller synthetic methods if basic blocks of code appeared
> frequently in a very large body. Right now, javac and friends are
> pretty dumb...you feed it a big chunk of code, it puts it into a big
> method. But if it were smart about splitting up blocks into synthetic
> submethods, we'd have mo' betta inlining possibilities, smaller units
> of code, and generally better performance.
>
> There's a lot of opportunity here.
>
> - Charlie
>
> --
>
> You received this message because you are subscribed to the Google Groups "JVM Languages" group.
> To post to this group, send email to jvm-la...@googlegroups.com.
> To unsubscribe from this group, send email to jvm-language...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/jvm-languages?hl=en.
>
>
>
--
Matt Fowles
SimpleCompiler.cook(Java.CompilationUnit compilationUnit)
to the API expressly for this purpose.
Matt
John Cowan
> John~
> I can pretty much guarantee that the new maintainer will be amenable to
> patches that help Janino as a utility to use in compilers. As it turns out,
> I am the new maintainer ;-)
Oh, cool. I missed that point, sorry.
Unfortunately I don't have a patch (and probably won't be able to
provide one), but I still think supporting "goto" would be an
Excellent Thing. I'd much rather generate Java than byte code, even
with ASM or a similar library; but allowing "goto" makes life much
simpler for TRO (though not for proper tail calling, alas) and for
compiling languages that have gotos already.
> My current idea is not to do heavy duty optimizations like register coloring
> and variable minimization, but to focus on optimizations that will be a
> clear win (dead code elimination, constant propagation, conditional
> result propagation (is there a better term for that?)).
Fair enough: that sort of optimization is well worth doing, and the
nice thing is that it's modular: you can improve things incrementally
without tearing up the framework and starting over. Charlie's
examples are good ones too.
John Cowan
> John~
> That depends on how you squint. I never bother with Janino's parser; I walk
> my AST and generater a Janino AST and then cook that directly. I added the
> method
>
> SimpleCompiler.cook(Java.CompilationUnit compilationUnit)
>
> to the API expressly for this purpose.
Ah. I can't run my compiler on the JVM until it's capable of
compiling itself, so I need to generate Java code.
Matt Fowles
Eugene Kuleshov
It would be neat if Janino used ASM's backend for bytecode
generation. Then you could feed its output into the class
transformations or use Janino's Java syntax tree object model to
construct code and convert it into the bytecode after that.
regards,
Eugene
On Dec 17, 11:52 am, Matt Fowles <matt.fow...@gmail.com> wrote:
> Charlie~
>
> We use Janino for that exact purpose. I am currently toying with the idea
> of making it into a mildly optimizing compiler (simply to reduce the size of
> the emitted bytecode and as an interesting exercise). Would you find any
> value in that?
>
> Matt
>
> On Thu, Dec 17, 2009 at 11:42 AM, Charles Oliver Nutter <head...@headius.com
>
> > wrote:
> > On Thu, Dec 17, 2009 at 8:58 AM, Matt Fowles <matt.fow...@gmail.com>
> > wrote:
> > > I am interested in this technique, can you provide a pointer to some
> > > starting point where I can see how you do this? I thought javac didn't
> > > expose these classes.
>
> > Yeah, I'd like to hear more as well. I would use it in at least Duby's
> > JVM backend, and potentially for other things. I've hacked javac
> > internals, but never tried to feed it valid trees and make it spit out
> > bytecode.
>
> > - Charlie
>
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "JVM Languages" group.
> > To post to this group, send email to jvm-la...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > jvm-language...@googlegroups.com<jvm-languages%2Bunsu...@googlegroups.com>
Matt Fowles
To unsubscribe from this group, send email to jvm-language...@googlegroups.com.
Eugene Kuleshov
I already tried it few years back but haven't finished it due to
other work and my lack of compiler design knowledge. I wonder if it
would be easier with some changes in ASM, so an non obscured look at
the conversion should really help if you'll be able to find time for
this. Please let me know if you'll need some improvements in ASM that
would make it easier. :-)
regards,
Eugene
On Dec 17, 4:23 pm, Matt Fowles <matt.fow...@gmail.com> wrote:
> Eugene~
>
> That is also on my list of things to do. As it stands, Janino rolls its own
> bytecode classes (which I have found and fixed a few bugs in), but not
> having the code at all would be even better.
>
> Of course, if you wanted to beat me to it, I wouldn't stop you ;-)
>
> Matt
>
> > <jvm-languages%2Bunsu...@googlegroups.com<jvm-languages%252Buns...@googlegroups.com>
Charles Oliver Nutter
> theoretically stop me from bootstrapping the compiler. In fact, Janino's
> parser is only useful if the language I am compiling is actually Java. If
> my language goes
> Foo -> Foo AST -> Janino AST -> bytecode
> I can still get a bootstrapped system in the usual way (implement a Foo
> compiler in Java, use that to implement a Foo compiler in Foo, throw away
> the Java one and compile using earlier versions of the compiler).
> Matt
This is what I hope to do, in terms of Ruby -> Ruby IR (CFG) -> Janino
AST -> bytecode. To be honest, I would also appreciate things like
register (local var) allocation, but for debugging sanity we'll
probably have to hand-select local variables anway.
- Charlie
Charles Oliver Nutter
> Eugene~
> That is also on my list of things to do. As it stands, Janino rolls its own
> bytecode classes (which I have found and fixed a few bugs in), but not
> having the code at all would be even better.
> Of course, if you wanted to beat me to it, I wouldn't stop you ;-)
> Matt
>
> On Thu, Dec 17, 2009 at 4:17 PM, Eugene Kuleshov <ekul...@gmail.com>
> wrote:
>>
>> It would be neat if Janino used ASM's backend for bytecode
>> generation. Then you could feed its output into the class
>> transformations or use Janino's Java syntax tree object model to
>> construct code and convert it into the bytecode after that.
Yeah, it seems like there's a coalescing of tools needed here. Janino
to use ASM, and maybe something from Soot too provide an optimization
phase in front or in back.
Tobias (from Jython) also had started working on an optimizing SSA
compiler for JVM, but I don't think he's had a chance to return to
that work. So once again, we're all rolling our own pieces when we
could be collaborating. Stop the madness!
- Charlie
Matt Fowles
Charles Oliver Nutter
> I agree that variable minimalization could definitely be useful long term,
> but my understanding is that it is an expensive optimization to compute.
> One other nice benefit of this is that (for debugging), you can go
> Ruby -> Ruby IR (CFG) -> Janino AST -> Java src
> We make heavy use of struct types, so a lot of primitive operations expand
> to several statements and it makes the debug cycle faster for initial
> development if you can see the code you are genning.
Now that could be very useful :) And for Duby, that would eliminate
the need for a separate .java backend like we have right now.
- Charlie
John Cowan
> I am confused... The fact that I don't use Janino's parser doesn't
> theoretically stop me from bootstrapping the compiler. In fact, Janino's
> parser is only useful if the language I am compiling is actually Java. If
> my language goes
> Foo -> Foo AST -> Janino AST -> bytecode
> I can still get a bootstrapped system in the usual way (implement a Foo
> compiler in Java,
Well, yes, if I were willing to do that, but I'm not, for several
reasons. So the Foo compiler is written in Foo and runs on a non-JVM
Foo implementation, spitting out Java. When finished, it'll be able
to compile itself, and then I can add JVM-specific calls to it, but
I'll continue to generate Java and just call Janino internally to
compile that into bytecode. Changing the compiler to construct a
Janino AST instead of Java is a project for another day (and maybe
another person, given the glacial speed at which I'm able to work on
it).
Given that, and given that Foo has gotos, I can use a much less
complicated code generator if I can generate Java+goto instead of pure
Java.
So, gotos, pretty please?
Kresten Krab Thorup
is only making basic verification; i.e. what can be done without
loading dependent classes. If you enable computing frames in ASM
(computing frames do require some more information), then it tries to
load the files using a classloader, which is not what I need: I need
to have a way to represent classes without them being loaded as normal
classes, with the hazards of running class initializers etc. Consider
this usecase: running a class weaver inside eclipse, in response to
builder events. There you do not want the real classes loaded at all.
Also, CheckClassAdapter does not check the applicability of receivers
and method arguments. If ASM could do those things, it would be
awesome.
Working with Java byte code, I often find myself having something like
Javassist's CtClass, except I want to use ASM and not get another
class library mixed in. ASM is really nice and clean cut, imho.
Being at run-time, I need something that can wrap *either* a real java
class, *or* something loaded from bytecode path; or something that
describes a class that does not exist yet (maybe it is just about to
be bytecode generated). I have done several half-hearted attempts
(ClassMirror in my kilim branch for instance), but it would be a great
if someone did a full API for this that was compatible with ASM.
On Dec 17, 10:47 am, Rémi Forax <fo...@univ-mlv.fr> wrote:
> Le 17/12/2009 10:43, Kresten Krab Thorup a crit :
>
> > Added link to my off-line verifier, I find it really useful for
> > understanding why the JVM won't accept my buggy byte code :-)
>
> > Click onhttp://groups.google.com/group/jvm-languages/web/jvm-bytecode-libraries
> > - or copy& paste it into your browser's address bar if that doesn't
> > work.
>
> ASM has also this kind of stuff:http://asm.ow2.org/asm32/javadoc/user/org/objectweb/asm/util/CheckCla...
>
> It dumps the stack and the local variable types for each bytecode
> instruction.
>
> R mi
Eugene Kuleshov
I though there was some optimizing transformations implemented for
Maxine JVM... Wouldn't it be cool to reuse those?
regards,
Eugene
On Dec 17, 4:36 pm, Charles Oliver Nutter <head...@headius.com> wrote:
> On Thu, Dec 17, 2009 at 3:23 PM, Matt Fowles <matt.fow...@gmail.com> wrote:
> > Eugene~
> > That is also on my list of things to do. As it stands, Janino rolls its own
> > bytecode classes (which I have found and fixed a few bugs in), but not
> > having the code at all would be even better.
> > Of course, if you wanted to beat me to it, I wouldn't stop you ;-)
> > Matt
>
> > On Thu, Dec 17, 2009 at 4:17 PM, Eugene Kuleshov <ekules...@gmail.com>
Charles Oliver Nutter
reuse something from Jikes internals. There's so much prior art
available, it seems like a damn shame to have to rewrite anything
anymore :)
Eugene Kuleshov
The CheckClassAdapter provides utility static methods for the actual
bytecode verification, that should cover types applicability to some
extent.
Also, the ClassWriter can be extended to not load classes when
calculating frames (there is an example in ASM's tests).
It seem like it is time to update ASM's FAQ with some details about
those issues. It is all in the javadoc, but these issue still keep
poping up. :-(
regards,
Eugene