Calling non-Java languages from JavaScript

[Kenneth Russell]

All: there's a new feature in Java SE 6 Update 10 which might be of
interest to subscribers of this list.

If you host a Java applet on a web page, the JavaScript engine can
operate upon it: call methods, fetch fields, operate upon other objects
returned from previous method invocations, etc.

We wanted to enable language developers and users on the Java platform
such as yourselves to write applets in non-Java languages like JRuby,
Jython, Groovy, Scala, Fan, Clojure, JavaFX Script, etc., and still be
able to script these applets from the JavaScript engine in the web browser.

Java SE 6 Update 10 exposes some new interfaces from the Java Plug-In's
LiveConnect implementation that allows a non-Java language runtime to
hook in to the dispatch sequence for JavaScript operations coming in
from the web browser.

This lets you, the language developer, use your language to write
applets, and call into those applets from the web page. Essentially, you
define a mapping from your language to JavaScript syntax.

To give the basic idea, here's an example written in JavaFX Script. The
.fx file looks like this:

var color = Color.YELLOW;

public function setColor(red: Number,
green: Number,
blue: Number) : Void {
color = Color { red: red, green: green, blue: blue };
}

public function run(args: String[]) {
Stage {
scene: Scene {
fill: Color.DARKGRAY
content: [
Circle {
centerX : 125
centerY : 125
radius: 100
fill: bind color
}
]
}
}
}

and we can operate upon it from the web page using JavaScript functions
like these (assume it is hosted in an applet whose ID is "app"):

function makeRed() {
try {
app.script.setColor(1.0, 0.0, 0.0);
} catch (e) {
reportException(e);
}
}

function makeBlue() {
try {
app.script.setColor(0.0, 0.0, 1.0);
} catch (e) {
reportException(e);
}
}

Note the use of the synthetic "script" field attached to the applet,
which provides access to the script-level scope. This field is
synthesized by the JavaFX runtime using the new APIs in the Java Plug-In.

More complex examples are possible. Here's JavaScript which dives into a
scene graph (similar structure to the one above) and modifies the color
directly:

function makeRed() {
try {
app.stage.scene.content[0].fill =
app.Packages.javafx.scene.paint.Color.RED;
} catch (e) {
reportException(e);
}
}

function makeBlue() {
try {
app.stage.scene.content[0].fill =
app.Packages.javafx.scene.paint.Color.BLUE;
} catch (e) {
reportException(e);
}
}

This example uses the built-in per-applet Packages keyword to access the
javafx.scene.paint namespace.

Complex data type conversions are possible. The JavaFX runtime uses the
new APIs to define conversion operations between JavaScript arrays and
JavaFX sequences. Here's a JavaFX function from another example which
takes in two sequences of numbers and makes a Timeline animation out of
them:

public function animate(times: Number[],
xyCoords: Number[]) {
var t = Timeline {
keyFrames: for (i in [0..(sizeof times - 1)]) {
KeyFrame {
time: Duration.valueOf(times[i]);
values: [
x => xyCoords[2*i],
y => xyCoords[2*i+1],
]
}
}
}

if (timeline != null) {
timeline.stop();
}

timeline = t;
timeline.play();
}

The function above can be called from JavaScript as follows:

function clockwise() {
try {
app.script.animate([250, 500, 750, 1000],
[225, 225,
25, 225,
25, 25,
225, 25]);
} catch (e) {
reportException(e);
}
}

function counterclockwise() {
try {
app.script.animate([250, 500, 750, 1000],
[25, 25,
25, 225,
225, 225,
225, 25]);
} catch (e) {
reportException(e);
}
}

We think these new APIs offer a lot of interesting interoperability
possibilities for dynamic language implementors on the JVM. They tie
into the meta-object protocol work being done by Attila Szegedi and
others, and while they aren't currently related, it would be interesting
to see if unification is possible.

The documentation (including javadoc) for the new APIs is available here:

https://jdk6.dev.java.net/plugin2/liveconnect/#NON_JAVA_LIVECONNECT

but of course the most useful thing is example code. The first and
proof-of-concept implementation of these interfaces has been done for
JavaFX Script. Roughly 90% of the code is in the open-source repository
for the JavaFX compiler at https://openjfx-compiler.dev.java.net/, under
the directory src/share/classes/com/sun/javafx/runtime/liveconnect/ .
The only pieces missing are those which instantiate and register the
delegates; these necessarily went into a different (currently non-open)
repository but are pretty straightforward.

If you have any feedback on this new functionality please post it to the
list. We're hoping this opens up new interesting possibilities for web
application development on the JVM.

Thanks,

-Ken

[Charles Oliver Nutter]

Kenneth Russell wrote:
> Java SE 6 Update 10 exposes some new interfaces from the Java Plug-In's
> LiveConnect implementation that allows a non-Java language runtime to
> hook in to the dispatch sequence for JavaScript operations coming in
> from the web browser.
>
> This lets you, the language developer, use your language to write
> applets, and call into those applets from the web page. Essentially, you
> define a mapping from your language to JavaScript syntax.

When I read this a couple days ago, I had one word: WOW. This is really
awesome news. About a year ago I fiddled around with making JRuby in an
applet usable from the browser's JS engine, and it was really painful. I
eventually got it to hobble along, but it was not really usable. But
this is very exciting to see.

What's the progress of getting this u10 functionaliy out to other
platforms, including Mac OS X?

- Charlie

[Kenneth Russell]

Charles Oliver Nutter wrote:
> Kenneth Russell wrote:
>> Java SE 6 Update 10 exposes some new interfaces from the Java Plug-In's
>> LiveConnect implementation that allows a non-Java language runtime to
>> hook in to the dispatch sequence for JavaScript operations coming in
>> from the web browser.
>>
>> This lets you, the language developer, use your language to write
>> applets, and call into those applets from the web page. Essentially, you
>> define a mapping from your language to JavaScript syntax.
>
> When I read this a couple days ago, I had one word: WOW. This is really
> awesome news. About a year ago I fiddled around with making JRuby in an
> applet usable from the browser's JS engine, and it was really painful. I
> eventually got it to hobble along, but it was not really usable. But
> this is very exciting to see.

I'm glad you think so. If you or anyone else decides to try out this new
inter-language bridge, please post to the list and let us know how your
work is going, especially if you have any questions or run into any
problems.

> What's the progress of getting this u10 functionaliy out to other
> platforms, including Mac OS X?

6u10 was released for Linux, Solaris and Windows simultaneously. The new
Java Plug-In (and therefore this new functionality) works in all
supported browsers: IE 6 and later, and Firefox 3. Other non-Firefox
browsers supporting the latest versions of the NPAPI and NPRuntime are
known to work with the new Java Plug-In for Firefox (npjp2.dll /
libnpjp2.so).

We have been collaborating with Apple for a few months now and have an
advanced stage prototype of the new Java Plug-In running identically in
the WebKit and Firefox 3.1 nightly builds on the Mac. Further work is
ongoing.

-Ken

[Greg Brown]

This sounds like a very cool feature. I've been trying to figure out
how I might build an applet in Groovy. Are you aware of any examples
that demonstrate this?

[Guillaume Laforge]

You can already build applets in Groovy, even without that bridge.
Am I missing something?

--
Guillaume Laforge
Groovy Project Manager
G2One, Inc. Vice-President Technology
http://www.g2one.com

[Greg Brown]

I guess *I'm* missing something. I haven't been able to find any
examples that show how to do this.

[Guillaume Laforge]

The Grapplet module is a nice improved Groovy / Applet mix, for
instance: http://groovy.codehaus.org/Grapplet
But beyond this, there's perhaps not much documentation because Groovy
can be compiled to bytecode, and you can just bundle that bytecode in
normal JARs without much more complexity (beyond signing the Groovy
jar).

[Greg Brown]

Interesting. Just wondering - why does the JAR need to be signed?

I'll be more specific about what I'm trying to accomplish - I want to write a Pivot application using Groovy:

http://pivot-toolkit.org

All Pivot applications implement the pivot.wtk.Application interface. Pivot includes a bootstrap applet that instantiates and executes the application's lifecycle methods (startup(), shutdown(), suspend(), and resume()).

Theoretically, I should be able to implement Application as a Groovy class and launch that using the Pivot applet. However, I wasn't sure what else I might need to do (e.g. signing the JAR, including the right libraries on the applet's classpath, etc.).

Apologies if there are obvious answers to these questions - I'm relatively new to Groovy.

Thanks,
Greg

[Guillaume Laforge]

I guess we'd better move that discussion on the Groovy lists, since
it's perhaps too Groovy specific?
Hoping I'm not off-topic, the jar needs to be signed as it uses
reflection, bytecode generation, etc, some things that are prevented
by the default security manager.

[Greg Brown]

> I guess we'd better move that discussion on the Groovy lists, since
> it's perhaps too Groovy specific?

Agreed. What might be a more appropriate group?

[Charles Oliver Nutter]

Guillaume Laforge wrote:
> The Grapplet module is a nice improved Groovy / Applet mix, for
> instance: http://groovy.codehaus.org/Grapplet
> But beyond this, there's perhaps not much documentation because Groovy
> can be compiled to bytecode, and you can just bundle that bytecode in
> normal JARs without much more complexity (beyond signing the Groovy
> jar).

Why is signing necessary?

- Charlie

[Charles Oliver Nutter]

Didn't see this email before I responded.

But I think the discussion is just fine here, since various languages
will have similar challenges deploying an applet. For example JRuby
would require signing if you wanted full performance, but could run
interpreted without a signed applet just fine.

Is there a way to run Groovy so that it would not need to be signed?

- Charlie

[Guillaume Laforge]

On Tue, Nov 11, 2008 at 12:11 AM, Charles Oliver Nutter
<charles...@sun.com> wrote:
>
> Didn't see this email before I responded.
>
> But I think the discussion is just fine here, since various languages
> will have similar challenges deploying an applet. For example JRuby
> would require signing if you wanted full performance, but could run
> interpreted without a signed applet just fine.
>
> Is there a way to run Groovy so that it would not need to be signed?

Not as far as I know.
I guess it'd require a fair amount of work to allow that.

[Guillaume Laforge]

The Groovy user mailing-list may be more appropriate, I guess.
But Charlie thinks we're not too off-topic, so I guess we can stay
here for the moment.

[Greg Brown]

>Is there a way to run Groovy so that it would not need to be signed?

I think this is the key question. The idea that apps can be written for the Java Plugin using any JVM scripting language is very compelling; somewhat less so if the code needs to be signed in order for it to work.

Ken, can you offer any insight here?

Thanks,
Greg

[Charles Oliver Nutter]

I guess the key problem is being able to generate and load arbitrary
code. In JRuby, that would happen in JIT mode and if methods are bound
using generated stubs/invokers. But we also can just run interpreted and
use reflection, which I think gets around the security issues. It
seems like Groovy could do the same by precompiling, unless there's some
additional code generation at runtime even when all code is precompiled.

It seems like CLR/DLR/Silverlight would have to support code generation
to allow IronRuby and IronPython to run. Assuming that's the case, why
is their security model more permissive about this kind of thing than
the Java plugin's?

- Charlie

[Patrick Wright]

Are the problems in applets related to reflection or to creating
custom classloaders to load and cache newly-created bytecode, or ?
What calls are bouncing off the walls of the sandbox?

Patrick

[Rémi Forax]

Patrick Wright a écrit :

Interresting question, indeed.
The problem is that to load new bytecodes you need to create
a custom classloader and because Java security model relies on
classloader too,
to provide security context, you need to sign the applet.

The question is why do we have to relies on classloader to load new
bytecodes
and the answer is : we don't have to, AnonymousClassLoader (with no host
class)
is our friend. Too bad it's Sun VM specific .

>
> Patrick
>
Rémi

[Kenneth Russell]

In the context of the new JavaScript/[arbitrary language] bridge, it is
required to sign at least one of the classes in the language runtime,
namely the one which installs the adapter for that language. The reason
is that the bridge allows you to potentially intercept all incoming
JavaScript calls against objects in the scope of that applet, including
objects that don't belong to that language, so it seemed too risky to
allow this to be done by unsigned code until we better understand the
security implications.

-Ken

[John Rose]

On Nov 11, 2008, at 8:23 AM, Rémi Forax wrote:
> Interresting question, indeed.
> The problem is that to load new bytecodes you need to create
> a custom classloader and because Java security model relies on
> classloader too,
> to provide security context, you need to sign the applet.
>
> The question is why do we have to relies on classloader to load new
> bytecodes and the answer is : we don't have to,
> AnonymousClassLoader (with no host
> class) is our friend. Too bad it's Sun VM specific .

The anonymous class feature is an experimental part of the HotSpot
implementation, to be used internally only. The proposed API in
java.dyn.AnonymousClassLoader is experimental only, and is not yet
proven secure.

In general, class loading without ClassLoaders won't be ready to
adopt as any kind of standard until we understand all required
changes to the security model. (Anybody want to write a paper on this?)

Remember that the JVM assigns permissions according to the least
privileged of all methods on a thread's stack (up to a doPrivileged
mark). A class is "tainted" by its origin, and passes that taint to
its callees (again, down to a doPrivileged mark). Introducing an
anonymous class with no host class creates code which does not carry
around any privilege limitations. Therefore it had better be trustable.

A more likely use case for applets would be an anonymous class which
is hosted from an untrusted applet class, so that it shares the same
"trust taint" as the original applet code.

So the anonymous class stuff is (currently) just an implementation
detail in HotSpot, for use by trusted code only.

-- John

[Greg Brown]

> > Is there a way to run Groovy so that it would not need to be signed?
>
> Not as far as I know.
> I guess it'd require a fair amount of work to allow that.

Just to make sure I understand - does this apply to interpreted Groovy
*and* Groovy that is compiled to bytecode, or just interpreted Groovy?

On another topic:

I'm also interested in using JavaScript in an applet. I can't use the
version of Rhino that ships with Java 6 because Apple doesn't include
it (and Apple doesn't support Java 6 in applets yet anyways). Does
anyone know if it is possible to use Mozilla's Rhino implementation in
"unoptimized" (interpreted) mode to implement a single interface at a
time without requiring class generation?

This article mentions that Sun's version limits JavaScript to
implementing a single interface to work around the class generation
issue, but it doesn't say whether or not Mozilla's Rhino will always
use class generation even in simpler cases that may not require it:

http://java.sun.com/javase/6/docs/technotes/guides/scripting/programmer_guide/index.html

If Mozilla Rhino always uses class generation, is Sun's version
available for download separately anywhere, so I can include it with
an applet?

Thanks,
Greg

[Jochen Theodorou]

Greg Brown schrieb:

>>> Is there a way to run Groovy so that it would not need to be signed?
>> Not as far as I know.
>> I guess it'd require a fair amount of work to allow that.
>
> Just to make sure I understand - does this apply to interpreted Groovy
> *and* Groovy that is compiled to bytecode, or just interpreted Groovy?

there is not really interpreted Groovy. Groovy compiles always. The
scripting ability is reached by compiling the source file first and then
execute it. In that case the class files exist in memory only.

bye Jochen

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/

[Greg Brown]

>>>> Is there a way to run Groovy so that it would not need to be signed?
>>> Not as far as I know.
>>> I guess it'd require a fair amount of work to allow that.
>>
>> Just to make sure I understand - does this apply to interpreted Groovy
>> *and* Groovy that is compiled to bytecode, or just interpreted Groovy?
>
>there is not really interpreted Groovy. Groovy compiles always. The
>scripting ability is reached by compiling the source file first and then
>execute it. In that case the class files exist in memory only.

I see. I'm working under the assumption that it is possible to "pre-compile" Groovy into bytecode. If so, then the Groovy runtime theoretically shouldn't need to perform class generation, wouldn't need security privileges, and thus wouldn't require a signed applet. Is this accurate?

Also, if Groovy is pre-compiled, is the Groovy JAR still required at deployment time?

[Guillaume Laforge]

On Thu, Nov 13, 2008 at 10:35 PM, Greg Brown <gkb...@mac.com> wrote:
> [...]

> Also, if Groovy is pre-compiled, is the Groovy JAR still required at deployment time?

Yes.

--
Guillaume Laforge
Groovy Project Manager

Head of Groovy Development at SpringSource
http://www.springsource.com/g2one

[Greg Brown]

> > Also, if Groovy is pre-compiled, is the Groovy JAR still required at deployment time?
>
> Yes.

Thanks. What about this part?

[Guillaume Laforge]

Missed that part, sorry.

Even if we pre-compile Groovy, I think we're still doing some runtime
bytecode generation, and that's something that's not allowed if you're
not signing your jar, as far as I recall the details.

[Greg Brown]

>>>> I'm working under the assumption that it is possible to "pre-
>>>> compile" Groovy into bytecode.
>>>> If so, then the Groovy runtime theoretically shouldn't need to
>>>> perform class generation,
>>>> wouldn't need security privileges, and thus wouldn't require a
>>>> signed applet. Is this accurate?
>
> Missed that part, sorry.
>
> Even if we pre-compile Groovy, I think we're still doing some runtime
> bytecode generation, and that's something that's not allowed if you're
> not signing your jar, as far as I recall the details.

OK - thanks for the confirmation.
Greg

[Attila Szegedi]

On 2008.11.13., at 21:02, Greg Brown wrote:

> I'm also interested in using JavaScript in an applet. I can't use the
> version of Rhino that ships with Java 6 because Apple doesn't include
> it (and Apple doesn't support Java 6 in applets yet anyways). Does
> anyone know if it is possible to use Mozilla's Rhino implementation in
> "unoptimized" (interpreted) mode to implement a single interface at a
> time without requiring class generation?

Trouble is, Rhino's code for interface implemention actually generates
bytecode; it doesn't go i.e. through java.lang.reflect.Proxy (its
codegen capabilities predate Java 1.3). So regardless of interpreted
mode, Rhino will need to have the ability to load generated code if
you're using adapters...

> If Mozilla Rhino always uses class generation, is Sun's version
> available for download separately anywhere, so I can include it with
> an applet?

You can always get it from the OpenJDK source code.

Attila.

[Greg Brown]

>Trouble is, Rhino's code for interface implemention actually generates
>bytecode; it doesn't go i.e. through java.lang.reflect.Proxy (its
>codegen capabilities predate Java 1.3). So regardless of interpreted
>mode, Rhino will need to have the ability to load generated code if
>you're using adapters...

Huh. That's too bad.

Any idea if the reference implementation for JSR 223 would work (i.e. not require signing)? If not, OpenJDK is probably the way to go.

[mjw]

On Fri, 2008-11-14 at 15:20 +0100, Attila Szegedi wrote:
> You can always get it from the OpenJDK source code.

The Sun modified Rhino sources aren't available through OpenJDK
unfortunately. It isn't completely clear why not (some legal issue).

For IcedTea (http://icedtea.classpath.org/) we modified the sources so
it can just use upstream Rhino as packaged by any GNU/Linux distro,
when
configure detects them.
http://mail.openjdk.java.net/pipermail/build-dev/2008-June/001176.html

There is also the scripting.dev.java.net project which provides plugs
for upstream Rhino 1.6R7.

Cheers,

Mark