Binary build of MLVM available

[Attila Szegedi]

Hi folks,

I managed to build a MLVM binary today for Mac OS X 10.5 with Intel
CPUs. As I'm not a Sun employee, I can redistribute it under GPL, so
here it is for your own experimentation:

<http://dl.getdropbox.com/u/362958/mlvm/mlvm-macosx-i586-20090419.tgz>

Use at your own risk, and if it doesn't work, well, build a better one
yourself :-)

Seriously, it'd be hard to determine for most defects if they're a bug
in the OpenJDK source code, in a MLVM patch, or in my build process.
Therefore, if you run into a defect, the best way to proceed is to
build it yourself from source code. Of course, if building it yourself
fixed your issue, then I'd love to hear back about what did I do wrong.

Attila.

--
twitter: http://twitter.com/szegedi

[Frank Wierzbicki]

On Sun, Apr 19, 2009 at 4:24 PM, Attila Szegedi <szeg...@gmail.com> wrote:
>
> Hi folks,
>
> I managed to build a MLVM binary today for Mac OS X 10.5 with Intel
> CPUs. As I'm not a Sun employee, I can redistribute it under GPL, so
> here it is for your own experimentation:
>
> <http://dl.getdropbox.com/u/362958/mlvm/mlvm-macosx-i586-20090419.tgz>
>
> Use at your own risk, and if it doesn't work, well, build a better one
> yourself :-)

Cool! Running Jython's regression tests now -- looks to be going
well, though I haven't had a chance to try out any mlvm goodness yet.
This should make it much easier, thanks so much!

-Frank

[Kashif Qayyum]

Thank you! Now my lazy ass can experiment with this stuff without
having to fight the build process.

- kashif Q.
--------------------------------------------------------------------

[Thomas E Enebo]

You rock! Thanks...and now build them forever more for all of us as
the changesets come in :)

-Tom

--
Blog: http://www.bloglines.com/blog/ThomasEEnebo
Email: en...@acm.org , tom....@gmail.com

[Attila Szegedi]

On Mon, Apr 20, 2009 at 9:18 PM, Thomas E Enebo <tom....@gmail.com> wrote:
>
> You rock!  Thanks...and now build them forever more for all of us as
> the changesets come in :)

Every morning right between putting on the coffee and waking up the
kids for school ;-)

Attila.

[anon]

Hi, thanks for building this it is very helpful since I use a Mac.

I am having trouble getting it to work though. It works fine for none
Invoke Dynamic examples, e.g. HelloWorld, but when I try John Rose's
simple example from: http://wikis.sun.com/display/mlvm/ProjectCoinProposal
I get:

wizziewol-ln:src lov080$ $a/javac -version invokedynamicvirgin/
Hello.java
javac 1.7.0-internal
invokedynamicvirgin/Hello.java:3: package java.dyn does not exist
import java.dyn.*;
^
...

Almost certainly something I am doing wrong - can you help with my
mistake? Is there a command-line argument I need?

Thanks in advance for any help.

-- Howard.

On Apr 21, 5:23 am, Attila Szegedi <szege...@gmail.com> wrote:

> On Mon, Apr 20, 2009 at 9:18 PM, Thomas E Enebo <tom.en...@gmail.com> wrote:
>
>
>
> > You rock! Thanks...and now build them forever more for all of us as
> > the changesets come in :)
>
> Every morning right between putting on the coffee and waking up the
> kids for school ;-)
>
> Attila.

On Apr 21, 5:23 am, Attila Szegedi <szege...@gmail.com> wrote:

[Attila Szegedi]

Hm... java.dyn. classes are definitely present in rt.jar. Are you sure
your javac is getting classes from the rt.jar in jre/lib of the MLVM?
Is your JAVA_HOME set to the MLVM directory (or its JRE?)

Attila.

[hlovatt]

JAVA_HOME wasn't set, but if I do set it then nothing changes. I have
tried pointing JAVA_HOME to the top level directory, to the jre, and
jre/lib. Don't know what is going wrong!

[hlovatt]

Even if I explicitly include rt.jar then I get the same:

wizziewol-ln:src lov080$ echo $a ; echo $l ; ($a/jar tf $l/rt.jar |
grep java/dyn) ; $a/javac -version -cp $l/rt.jar:. invokedynamicvirgin/
Hello.java
/System/Library/Frameworks/JavaVM.framework/Versions/1.7.777/Home/bin
/System/Library/Frameworks/JavaVM.framework/Versions/1.7.777/Home/jre/
lib
java/dyn/AnonymousClassLoader.class
java/dyn/CallSite.class
java/dyn/ConstantPoolParser.class
java/dyn/ConstantPoolPatch$1.class
java/dyn/ConstantPoolPatch$2.class
java/dyn/ConstantPoolPatch.class
java/dyn/ConstantPoolVisitor.class
java/dyn/Dynamic.class
java/dyn/InvalidConstantPoolFormatException.class
java/dyn/InvokeDynamicBootstrapError.class
java/dyn/Linkage.class
java/dyn/LinkagePermission.class
java/dyn/MethodHandle.class
java/dyn/MethodHandles.class
java/dyn/MethodType.class
java/dyn/MethodTypeForm.class
java/dyn/NoAccessException.class
java/dyn/WrongMethodTypeException.class

javac 1.7.0-internal
invokedynamicvirgin/Hello.java:3: package java.dyn does not exist
import java.dyn.*;
^

[Ben Evans]

I've got an alternative binary build available - it's the codebase as
of 2009-04-10. There's a download link available from my blog: http://boxcatjunction.blogspot.com

I've been using it as my primary JVM without any problems (including
the invdyn features) for the last 10 days or so.

Sent from my iPhone

[hlovatt]

Hi Ben,

I get exactly the same result with your build:

wizziewol-ln:src lov080$ echo $JAVA_HOME
/System/Library/Frameworks/JavaVM.framework/Versions/1.7.20090410/
Home/
wizziewol-ln:src lov080$ echo $a/System/Library/Frameworks/
JavaVM.framework/Versions/1.7.20090410/Home/bin/
wizziewol-ln:src lov080$ echo $l/System/Library/Frameworks/
JavaVM.framework/Versions/1.7.20090410/Home/jre/lib/
wizziewol-ln:src lov080$ $a/jar tf $l/rt.jar | grep java/dynjava/dyn/

AnonymousClassLoader.class
java/dyn/CallSite.class
java/dyn/ConstantPoolParser.class
java/dyn/ConstantPoolPatch$1.class
java/dyn/ConstantPoolPatch$2.class
java/dyn/ConstantPoolPatch.class
java/dyn/ConstantPoolVisitor.class
java/dyn/Dynamic.class
java/dyn/InvalidConstantPoolFormatException.class
java/dyn/InvokeDynamicBootstrapError.class
java/dyn/Linkage.class
java/dyn/LinkagePermission.class
java/dyn/MethodHandle.class
java/dyn/MethodHandles.class
java/dyn/MethodType.class
java/dyn/MethodTypeForm.class
java/dyn/NoAccessException.class
java/dyn/WrongMethodTypeException.class

wizziewol-ln:src lov080$ $a/javac -version -cp $l/rt.jar:.
invokedynamicvirgin/Hello.java

javac 1.7.0-internal
invokedynamicvirgin/Hello.java:3: package java.dyn does not exist
import java.dyn.*;
^

Thanks for the posting of your version, I have know idea what is going
on :(

-- Howard.

On Apr 22, 8:28 pm, Ben Evans <benjamin.john.ev...@googlemail.com>
wrote:

> I've got an alternative binary build available - it's the codebase as  
> of 2009-04-10. There's a download link available from my blog:http://boxcatjunction.blogspot.com
>
> I've been using it as my primary JVM without any problems (including  
> the invdyn features) for the last 10 days or so.
>
> Sent from my iPhone
>

[John Rose]

On Apr 22, 2009, at 4:43 PM, hlovatt wrote:

> Hi Ben,
>
> I get exactly the same result with your build:

It's a pain when the tools won't listen to your intended configuration.

One way to investigate (that sometimes helps me) is to look underneath
the tools to see what pathnames they are accessing. On Solaris it is
"truss", on older Macs it is "ktrace", and on Leopard it is "dtrace".
It requires a very modest "sudo" setup. Here's an article on dtrace:

http://www.macosxhints.com/article.php?story=20071031121823710

I have no idea why this should be necessary, but you could try to
force your explicit rt.jar earlier into javac's search order:

javac -Xbootclasspath/p:$myjrelib/rt.jar ...

-- John

[hlovatt]

Hi,

I have made some progress:

wizziewol-ln:src lov080$ $b/javac -source 1.7 -bootclasspath $m/rt.jar
-g -Xlint invokedynamicvirgin/Hello.java

Works - hurray! You need both -source 1.7 and -bootclasspath $m/rt.jar
(-g and -Xlint aren't needed). Presumably this is a bug in the 1.7
compiler that doesn't set the defaults properly. Same result with both
Ben and Attila builds. However I still have a problem with running the
code, e.g. with Attila's build:

wizziewol-ln:src lov080$ $b/java -showversion -ea -XX:
+EnableMethodHandles invokedynamicvirgin.Hello
openjdk version "1.7.0-internal"
OpenJDK Runtime Environment (build 1.7.0-internal-
aszegedi_2009_04_19_15_49-b00)
OpenJDK Server VM (build 15.0-b02, mixed mode)

Hello, world (from a statically linked call site)
Hello, world
Exception in thread "main" java.lang.IncompatibleClassChangeError
at invokedynamicvirgin.Hello.main(Hello.java:30)

Identical result on both builds. The code in question is:

public static void main( String... av ) {
if ( av.length == 0 ) { av = new String[] { "world" }; }
greeter( av[ 0 ] + " (from a statically linked call site)" );
for ( String whom : av ) {
greeter.<void>invoke( whom ); // strongly typed direct call

// previous line generates invokevirtual MethodHandle.invoke
(String)void
Dynamic x = whom;
x.hail(); // weakly typed invokedynamic

// previous line generates invokedynamic
// MethodHandle.invoke(Dynamic)Dynamic
}
}

And line 30 is the x.hail() line. The code is Simple Example from:
http://wikis.sun.com/display/mlvm/ProjectCoinProposal

Can you guys run this code, if so, what are your command line options?

Thanks,

-- Howard.

[hlovatt]

Thanks John I had independently discovered this by messing around with
options and our posts crossed. I am now having a problem with running
the example :(. See my previous post that crossed with yours.

-- Howard.

[John Rose]

On Apr 22, 2009, at 5:36 PM, hlovatt wrote:

> Can you guys run this code, if so, what are your command line options?

I'm in the middle of a different micro-version of the system, so I
don't get the ICCE. (I get something else, which I'm fixing.)

The ICCE is the generic error for a bytecode-level configuration
problem. Are you running a 64-bit JVM? There is no 64-bit support
yet (help?!) and you'll get an ICCE like the one you report.

I need to update the mlvm repo with (a) the results of the push that
just went into the JVM and (b) my latest hackings on the java.dyn and
sun.dyn classes. Please stay tuned.

This pain you feel is reserved for the prestigious group called the
Early Adopters!

-- John

[hlovatt]

Hi John,

Thanks for your reply:

On Apr 23, 11:05 am, John Rose <John.R...@Sun.COM> wrote:
[snip]

> The ICCE is the generic error for a bytecode-level configuration  
> problem.  Are you running a 64-bit JVM?  There is no 64-bit support  
> yet (help?!) and you'll get an ICCE like the one you report.

I assume the Mac builds are 64 bit, the other Mac VMs are. So that is
probably the problem.

> I need to update the mlvm repo with (a) the results of the push that  
> just went into the JVM and (b) my latest hackings on the java.dyn and  
> sun.dyn classes.  Please stay tuned.

It may not be a problem for me, I am more interested in the strongly
typed direct call which seems to be working fine. I am wanting to get
multiple dispatch running using method handles rather than a
dispatcher object which I currently use.

> This pain you feel is reserved for the prestigious group called the  
> Early Adopters!

No need to apologise, it is great that you make the code available
early in the development cycle.

-- Howard.

[Attila Szegedi]

I know I'm late with the answer, but a very quick and zero setup
solution is to have a separate terminal window with:

sudo fs_usage | grep javac

(fs_usage must be run as a superuser)

Attila.

[Attila Szegedi]

Nope, the OpenJDK builds are 32-bit, i.e. this command in the MLVM jre/
bin directory:

MacBook-Pro-Ati:bin aszegedi$ lipo -info java
Non-fat file: java is architecture: i386

"i386" means 32 bit. In contrast, running lipo -info on the Apple-
shipped 1.6.0 java (which is 64-bit) gives "x86_64":

MacBook-Pro-Ati:bin aszegedi$ lipo -info /System/Library/Frameworks/
JavaVM.framework/Versions/1.6.0/Commands/java
Non-fat file: /System/Library/Frameworks/JavaVM.framework/Versions/
1.6.0/Commands/java is architecture: x86_64

So, that MLVM build is definitely 32-bit.

Attila.

[hlovatt]

Attila - can you run the example?

[Attila Szegedi]

No, I can't. I also get the ICCE at the x.hail() invocation...:

MacBook-Pro-Ati:j2sdk-image aszegedi$ bin/javac -Xbootclasspath/p:jre/
lib/rt.jar -source 1.7 Hello.java
MacBook-Pro-Ati:j2sdk-image aszegedi$ bin/java -Xbootclasspath/p:jre/
lib/rt.jar Hello
MacBook-Pro-Ati:j2sdk-image aszegedi$ bin/java -Xbootclasspath/p:jre/
lib/rt.jar -XX:+EnableMethodHandles Hello

Hello, world (from a statically linked call site)
Hello, world
Exception in thread "main" java.lang.IncompatibleClassChangeError

at Hello.main(Hello.java:11)

Attila.

[Tom Davies]

Which patches are included in this?

[Tom Davies]

On Apr 23, 7:17 pm, Tom Davies <tgdav...@gmail.com> wrote:
> Which patches are included in this?

In particular I'm interested in tail call optimisation via the patches
mentioned here: http://wikis.sun.com/display/mlvm/TailCalls

[Arnold Schwaighofer]

Hi Tom,

the patch that is available in the mlvm repository is quite dated. The
patches mentioned on this site are not available online yet. If you
want i can send you a more recent patch set (the patches mentioned on
http://wikis.sun.com/display/mlvm/TailCalls) personally via email.
These patches will work against vanilla jdk7 b50.

I am currently working on rebasing the tail call patches to vanilla
jdk7 b56. There were some changes (method handles) such that i have to
merge some stuff by hand. When this is done it should be easier
(crossing fingers) to apply the tail call patches on top of the other
mlvm stuff.

regards
arnold

[Tom Davies]

On Apr 25, 2:59 am, Arnold Schwaighofer

<arnold.schwaigho...@gmail.com> wrote:
> Hi Tom,
>
> the patch that is available in the mlvm repository is quite dated. The
> patches mentioned on this site are not available online yet. If you
> want i can send you a more recent patch set (the patches mentioned onhttp://wikis.sun.com/display/mlvm/TailCalls) personally via email.

Thanks Arnold, but realistically I'm unlikely to find the time to
build the JDK myself in the near future :-(

I'll wait for them to appear in a pre-built version.

I'm interested in trying to convert CAL (http://openquark.org) from
trampolining to tail calls for mutually recursive tail calls, but this
is hobby programming and I don't have a great deal of time!

Tom

[John Rose]

Here's a bit of new about invokedynamic:  I just pushed my updated code (about 10KLOC) to an OpenJDK integration area:

  http://hg.openjdk.java.net/jdk7/tl/jdk/file/tip/src/share/classes/java/dyn/

It's much more functional than before.  See demo below for a small example.  Or look at the combinator definitions (e.g., FromGeneric) in the JDK code.

I'm 2/3 done pulling that version back into mlvm patches, on top of bsd-port.

Over time, the official JDK bits get integrated first into Sun's weekly builds, and then they (I hope) will flow to the BSD-port repo, allowing the MLVM patch set to shrink.  This may make private "bleeding edge" builds easier to cope with.

Best wishes,

-- John

import java.dyn.*;

import static java.dyn.MethodHandles.*;

import static java.dyn.MethodType.*;

public class FidgetDemo {

    public static void main(String... av) {

        if (av.length == 0)  av = new String[]{ "fred", "buster", "ricky" };

        System.out.println("Fidgety self-modifying call site...");

        for (int i = 0; i < 6; i++) {

            System.out.println("--- loop #"+i);

            for (String a : av) {

                String result = InvokeDynamic.<String>fidget(a);

                System.out.println(result);

            }

        }

    }

    private static String itch(String x) { return x+" can't get comfortable"; }

    private static String fuss(String x) { return x+" is feeling moody"; }

    private static String bore(String x) { return x+" needs a change of scenery"; }

    private static final String[] NAMES = { "itch", "fuss", "bore" };

    private static MethodHandle which(int i) {

        return lookup().findStatic(FidgetDemo.class, NAMES[i % NAMES.length],

                    make(String.class, String.class));

    }

    // It is easy to build hybrid closure-like method handles out of classes.

    private static class Guard extends JavaMethodHandle {

        MethodHandle target;  // wrapped target

        CSite        site;    // included site

        // constructor folds a sneaky site reference into a direct target

        Guard(CSite site, MethodHandle target) {

            super(INVOKE);

            this.target = target;

            this.site = site;

        }

        String invoke(String x) {

            if ((++site.count & 3) == 0)

                site.setTarget(new Guard(site, which(site.count)));

            return target.<String>invoke(x);

        }

        private static final MethodHandle INVOKE =

            lookup().findVirtual(Guard.class, "invoke",

                make(String.class, String.class));

        public String toString() { return "Guard:"+target.toString(); }

    }

    // Use a local call site subclass.  (These are optional but fun.)

    private static class CSite extends CallSite {

        int count;

        public CSite(Class caller, String name, MethodType type) {

            super(caller, name, type);

            //?? super(String.class, "gloat", make(void.class));

            System.out.println("[link] new call site: "+this);

            setTarget(new Guard(this, which(0)));

        }

        // this is just for the noise value:

        public void setTarget(MethodHandle t) {

            System.out.println("[link] set target to "+t);

            super.setTarget(t);

        }

    }

    // Set up a class-local bootstrap method.

    static { Linkage.registerBootstrapMethod("linkDynamic"); }

    private static CallSite linkDynamic(Class caller, String name, MethodType type) {

        assert((Object)name == "fidget" &&

                type == make(String.class, String.class));

        return new CSite(caller, name, type);

    }

}

/* resulting output

--------

Fidgety self-modifying call site...

--- loop #0

[link] new call site: CallSite#14491894[fidget(java.lang.String)java.lang.String => null]

[link] set target to Guard:itch(java.lang.String)java.lang.String

fred can't get comfortable

buster can't get comfortable

ricky can't get comfortable

--- loop #1

[link] set target to Guard:fuss(java.lang.String)java.lang.String

fred can't get comfortable

buster is feeling moody

ricky is feeling moody

--- loop #2

fred is feeling moody

[link] set target to Guard:bore(java.lang.String)java.lang.String

buster is feeling moody

ricky needs a change of scenery

--- loop #3

fred needs a change of scenery

buster needs a change of scenery

[link] set target to Guard:itch(java.lang.String)java.lang.String

ricky needs a change of scenery

--- loop #4

fred can't get comfortable

buster can't get comfortable

ricky can't get comfortable

--- loop #5

[link] set target to Guard:fuss(java.lang.String)java.lang.String

fred can't get comfortable

buster is feeling moody

ricky is feeling moody

--------

 */

[Rémi Forax]

John Rose a écrit :

> Here's a bit of new about invokedynamic: I just pushed my updated
> code (about 10KLOC) to an OpenJDK integration area:
> http://hg.openjdk.java.net/jdk7/tl/jdk/file/tip/src/share/classes/java/dyn/
>
> It's much more functional than before. See demo below for a small
> example. Or look at the combinator definitions (e.g., FromGeneric) in
> the JDK code.
>
> I'm 2/3 done pulling that version back into mlvm patches, on top of
> bsd-port.
>
> Over time, the official JDK bits get integrated first into Sun's
> weekly builds, and then they (I hope) will flow to the BSD-port repo,
> allowing the MLVM patch set to shrink. This may make private
> "bleeding edge" builds easier to cope with.
>
> Best wishes,
> -- John

Impressive !
the bootstrap method now returns a call site.
we just agree about that 2 days ago !

And combinators work...
It's a huge leap forward, Champagne !

Rémi

[John Rose]

On May 6, 2009, at 1:11 PM, Rémi Forax wrote:

> It's a huge leap forward, Champagne !

Cheers! <sip/> -- John