ECMAScript 4th edition functionality in Rhino
Cameron McCormack
At some point this year, I'd like to implement XBL 2.0[1] in Batik.
Rhino only supports ES 3rd ed, but some of features of 4th ed would be
really useful for defining bindings, specifically getter and setter
functions.
Is there an overall plan yet on how work towards 4th ed features will be
introduced? Will massive architectural changes be needed? If I manage
to write getter/setter support, can it be checked in and included in a
release?
Thanks,
Cameron
[1] http://dev.w3.org/cvsweb/~checkout~/2006/xbl2/Overview.html
--
Cameron McCormack, http://mcc.id.au/
xmpp:hey...@jabber.org ▪ ICQ 26955922 ▪ MSN c...@mcc.id.au
David P. Caldwell
> Rhino only supports ES 3rd ed, but some of features of 4th ed would be
> really useful for defining bindings, specifically getter and setter
> functions.
__defineGetter__ and __defineSetter__ are implemented already on the
trunk (just in the last few days).
__lookupGetter__, __lookupSetter__, and the get and set keywords are
not.
See bug at https://bugzilla.mozilla.org/show_bug.cgi?id=224334 (and
note that the last patch has a bug so was not checked in).
-- David Caldwell
http://www.inonit.com/
Cameron McCormack
> __defineGetter__ and __defineSetter__ are implemented already on the
> trunk (just in the last few days).
Oh, cool!
> __lookupGetter__, __lookupSetter__, and the get and set keywords are
> not.
>
> See bug at https://bugzilla.mozilla.org/show_bug.cgi?id=224334 (and
> note that the last patch has a bug so was not checked in).
I may have a poke around the parser code some time.
Attila Szegedi
ended up reading the "ECMAScript Netscape Proposal" at
http://www.mozilla.org/js/language/es4/ (assuming that's the right
document to go to).
I have stated earlier here that I hope Rhino can be extended to support
ES4. I have more doubts now. ES4 does a quite good job of preserving ES3
as the subset of ES4 on script source level, which gives an initial
illusion that the runtime system for ES4 could be naturally evolved from a
runtime system for ES3. While I don't claim that isn't true, I'm getting
the feeling that it isn't exactly feasible in the concrete case of Rhino.
Why, you may ask?
Here's why (I stress this is my personal opinion): the ammount of rework
is such that it could only be undertaken by people who know Rhino codebase
like the back of their hand, *AND* wish to actively work on it in the
future. I myself will humbly admit that there are areas of code that I
wouldn't dare touch because I don't understand its relation to other parts
of the code and don't know what would it break. Yes, I know we have tests,
and *lot* of them, but I also think we don't have *enough* of them. We'd
really need to combine it with a code coverage tool (i.e. EMMA) to gain a
bit more certainity about how good a safety net our existing test suite
is. But I'm going off on a tangent.
To look at a single example of Rhino's complexity and one that's highly
relevant if we were to evolve it to ES4: the parser is handwritten. It is
actually a port of the C code (its source code comments claim its heritage
to be "jsparse.c and jsparse.h in the jsref package"). That makes
extending it not an easy task (although naturally doable with enough
studying). A brave pioneer would go in and say, "hell, let's rewrite it
using {JavaCC|SableCC|Grammatica|ANTLR|...}". While that certainly has its
appeal, both ES3 and ES4 grammar have several peculiarities that are
awkward to express within the capabilities of the currently existing
parser generators. In particular, the syntactic case of automatic
semicolon insertion (the [no LineTerminator here] restriction), as well as
the ExpressionStatement production being restricted with a negative
lookahead ("next noken is NEITHER '{' NOR 'function'"). Not to speak about
parametrized productions in ES4 -- while they are easy to implement in a
handwritten parser (you just add parameters to the methods that match
those productions), they are difficult to catch in the grammar languages
of parser generators. BTW, ES3 also has a parametrized production, namely
the "NoIn", it is however expanded in the specification into "vanilla"
productions and "noIn" productions. As I said, ES4 has more of them, some
even have two parameters.
(Looking out of my window now I see the most beautiful January sunset I've
seen in a good while.)
Back to the point, introducing new semantical elements usually means
implementing them in four places: Parser, Interpreter, Codegen, and
finally ScriptRuntime.
Rhino also "suffers" from having lots of its internals being a rather
straight port of C code - manual implementation of a hashtable in
ScriptableObject, number formatting code in the DToA.java. Then there's
lot of relics from Java 1.x times -- using Hashtable and Vector classes,
not using JDK 1.2 weak references for certain caches, not using JDK 1.3
java.lang.reflect.Proxy in JavaAdapter, and so forth. These of course
don't preclude it from being evolved from ES3 to ES4, but would be nice to
get "evolved" in a different meaning from JDK 1.1 to at least 1.3.
Finally let me say that Rhino is a fine piece of software written by lot
of talented people. OTOH, it is somewhat natural that we who now steward
it are unsatisfied with it. Otherwise, why'd we want to advance it, right?
Attila.
--
home: http://www.szegedi.org
weblog: http://constc.blogspot.com
On Thu, 25 Jan 2007 05:50:08 +0100, Cameron McCormack <c...@mcc.id.au>
wrote:
David P. Caldwell
> Here's why (I stress this is my personal opinion): the ammount of rework
> is such that it could only be undertaken by people who know Rhino codebase
> like the back of their hand, *AND* wish to actively work on it in the
> future. I myself will humbly admit that there are areas of code that I
> wouldn't dare touch because I don't understand its relation to other parts
> of the code and don't know what would it break. Yes, I know we have tests,
> and *lot* of them, but I also think we don't have *enough* of them.
[snip]
> Finally let me say that Rhino is a fine piece of software written by lot
> of talented people. OTOH, it is somewhat natural that we who now steward
> it are unsatisfied with it. Otherwise, why'd we want to advance it, right?
I -- and my opinion is much more poorly informed than Attila's -- think
that both Optimistic Attila and Pessimistic Attila have good points
here. And I'm glad they got to watch a beautiful sunset together. ;)
I tend more toward Optimistic Attila's point of view than Pessimistic
Attila's.
That's mostly a side point, though. My main point I wanted to mention
is that it sounds like there's one thing that I think Optimistic and
Pessimistic Attila and Optimistic and Pessimistic David can *all* agree
on.
More tests. Both adding tests in the style of the SpiderMonkey
conformance tests (to which I think we owe a great debt, by the way --
at least I feel that way when I explore scary changes) and adding our
own Java-specific tests. I'm thinking about adding the ability to have
the Java JsDriver run tests outside the SpiderMonkey tests (and then
maybe we could put our own test-suite-style tests specifically in the
mozilla/js/rhino/ section of the tree somewhere?). But as Attila
mentioned when he started down the JUnit path, we can add more granular
tests, too.
Just trying to spark discussion,
-- David Caldwell.
http://www.inonit.com/
Mike Shaver
> To look at a single example of Rhino's complexity and one that's highly
> relevant if we were to evolve it to ES4: the parser is handwritten. It is
> actually a port of the C code (its source code comments claim its heritage
> to be "jsparse.c and jsparse.h in the jsref package"). That makes
> extending it not an easy task (although naturally doable with enough
> studying).
The current Tamarin source includes an AS3 compiler written in AS3,
which I understand will be maintained to track ES4 developments. It
currently produces AS3 bytecode, but making it produce Java bytecode
(or even just ASTs which can then get Codegen love from another part
of Rhino) might be a reasonable path to overcoming the parser obstacle
-- which is indeed quite a bear if taken from first principles.
The bootstrapping process would likely involve compiling
rhino-compiler.as to Tamarin bytecodes (ABC) and then running the
result on rhino-compiler.as again to get a version compiled to Java
bytecodes, but that's basically the same model as is being used on the
Tamarin side right now anyway, I do believe.
Not to say that the effort involved in implementing ES4 is trivial
even with that route available, but I thought it might help Optimistic
Attila win a round against Pessimistic Attila. :)
Mike
Cameron McCormack
Am I right in thinking you are the only active developer at the moment?
Attila Szegedi wrote:
> I've tried evaluating this -- I was hit by insomnia two nights ago and
> ended up reading the "ECMAScript Netscape Proposal" at
> http://www.mozilla.org/js/language/es4/ (assuming that's the right
> document to go to).
Fun bed time reading. :)
Maybe http://developer.mozilla.org/es4/ is more up to date, I'm not sure.
> I have stated earlier here that I hope Rhino can be extended to support
> ES4. I have more doubts now. ES4 does a quite good job of preserving ES3
> as the subset of ES4 on script source level, which gives an initial
> illusion that the runtime system for ES4 could be naturally evolved from a
> runtime system for ES3. While I don't claim that isn't true, I'm getting
> the feeling that it isn't exactly feasible in the concrete case of Rhino.
>
> Why, you may ask?
>
> Here's why (I stress this is my personal opinion): the ammount of rework
> is such that it could only be undertaken by people who know Rhino codebase
> like the back of their hand, *AND* wish to actively work on it in the
> future. I myself will humbly admit that there are areas of code that I
> wouldn't dare touch because I don't understand its relation to other parts
> of the code and don't know what would it break. Yes, I know we have tests,
> and *lot* of them, but I also think we don't have *enough* of them. We'd
> really need to combine it with a code coverage tool (i.e. EMMA) to gain a
> bit more certainity about how good a safety net our existing test suite
> is. But I'm going off on a tangent.
I tend to agree. There's a lot of code in there that isn't documented
that well.
Personally, I am always drawn to the option of a rewrite in situations
like this, regardless of whether that would eventually be the best use
of my time.
> To look at a single example of Rhino's complexity and one that's highly
> relevant if we were to evolve it to ES4: the parser is handwritten. It is
> actually a port of the C code (its source code comments claim its heritage
> to be "jsparse.c and jsparse.h in the jsref package"). That makes
> extending it not an easy task (although naturally doable with enough
> studying). A brave pioneer would go in and say, "hell, let's rewrite it
> using {JavaCC|SableCC|Grammatica|ANTLR|...}". While that certainly has its
> appeal, both ES3 and ES4 grammar have several peculiarities that are
> awkward to express within the capabilities of the currently existing
> parser generators. In particular, the syntactic case of automatic
> semicolon insertion (the [no LineTerminator here] restriction), as well as
> the ExpressionStatement production being restricted with a negative
> lookahead ("next noken is NEITHER '{' NOR 'function'"). Not to speak about
> parametrized productions in ES4 -- while they are easy to implement in a
> handwritten parser (you just add parameters to the methods that match
> those productions), they are difficult to catch in the grammar languages
> of parser generators. BTW, ES3 also has a parametrized production, namely
> the "NoIn", it is however expanded in the specification into "vanilla"
> productions and "noIn" productions. As I said, ES4 has more of them, some
> even have two parameters.
Though I haven't looked at the code yet, I'd think that the parser would
be one of the parts of Rhino that could be rewritten reasonably
independently of the rest of the code.
I agree, the production parameters shouldn't be hard to handle: for
recursive descent parsers, you just need to pass them down the call
stack. I haven't any experience with parser generators apart from
bison, but I'm sure there must be some that support synthesised
attributes. If not, SLR parser generators are not difficult to write
(assuming the language is SLR!), and such a generator could have support
for these parameters.
> Back to the point, introducing new semantical elements usually means
> implementing them in four places: Parser, Interpreter, Codegen, and
> finally ScriptRuntime.
>
> Rhino also "suffers" from having lots of its internals being a rather
> straight port of C code - manual implementation of a hashtable in
> ScriptableObject, number formatting code in the DToA.java. Then there's
> lot of relics from Java 1.x times -- using Hashtable and Vector classes,
> not using JDK 1.2 weak references for certain caches, not using JDK 1.3
> java.lang.reflect.Proxy in JavaAdapter, and so forth. These of course
> don't preclude it from being evolved from ES3 to ES4, but would be nice to
> get "evolved" in a different meaning from JDK 1.1 to at least 1.3.
>
> Finally let me say that Rhino is a fine piece of software written by lot
> of talented people. OTOH, it is somewhat natural that we who now steward
> it are unsatisfied with it. Otherwise, why'd we want to advance it, right?
Sure.
I think, as you point out, the optimizer is one of the parts of Rhino
that is the hardest for non-authors to understand, and it would be a
shame to lose this and other functionality if a rewrite happened.
Maybe the best way forward, if major changes are needed, would be for
the current Rhino design to be written down, a new design to be worked
out, and determine how the old maps to the new, to make it easier to
integrate as much of the old code into the new codebase as possible.
I haven't contributed much to Rhino so far, but I'm willing to help in
as much as my time permits.
Attila Szegedi
wrote:
> Hi Atilla (both versions of you ;)).
>
> Am I right in thinking you are the only active developer at the moment?
Well, define active.
David P. Caldwell joined the ranks recently as a committer - he's busy
writing us a shiny new E4X implementation that doesn't depend on 3rd party
stuff, but just plain DOM. He also added the
__defineGetter__/__defineSetter__ functionality, plus significant
enhancements to the unit testing package. By my standards, he's actually
the only active developer at the moment :-)
Me, I mostly review patches in Bugzilla every other sunday afternoon (or
in other odd bits of free time, i.e. 3 a.m. on nights when I can't fall
asleep) and commit them when I judge they're okay. I also try keeping an
eye on how involved certain people are with Rhino in order to find further
collaborators who I'd dare give CVS commit rights to it.
Norris Boyd is also around in a somewhat silent manner. I typically turn
to him with project administrativish stuff and he is a tremendous help
with those; he also contributes bugfixes concerning those code areas that
he originally wrote and I don't dare touch :-)
>
> Attila Szegedi wrote:
>> I've tried evaluating this -- I was hit by insomnia two nights ago and
>> ended up reading the "ECMAScript Netscape Proposal" at
>> http://www.mozilla.org/js/language/es4/ (assuming that's the right
>> document to go to).
>
> Fun bed time reading. :)
>
Well, it's not exactly bed, but rather a "give up trying to fall asleep at
3 a.m. and sit in front of the machine in the home office".
> Maybe http://developer.mozilla.org/es4/ is more up to date, I'm not sure.
Well, Google turned up the other fist...
>
> Though I haven't looked at the code yet, I'd think that the parser would
> be one of the parts of Rhino that could be rewritten reasonably
> independently of the rest of the code.
>
Well, it depends on whether the intermediate "JS bytecode" it generates
would remain the same for ES4. I don't think it would. And then you'd have
to go and rewrite the runtime system too. Or in what sense of the word
"rewrite" would you want to rewrite it? Preserving existing behaviour, but
presumably beating it into a shape where it's easier to evolve it?
> I agree, the production parameters shouldn't be hard to handle: for
> recursive descent parsers, you just need to pass them down the call
> stack. I haven't any experience with parser generators apart from
> bison, but I'm sure there must be some that support synthesised
> attributes. If not, SLR parser generators are not difficult to write
> (assuming the language is SLR!), and such a generator could have support
> for these parameters.
>
Wow. Writing a new parser generator so that we generate a parser for
Rhino. Now that's an idea :-) I'm probably getting old, but I couldn't
bring myself to write an abstraction in order to use it only once. What's
more my modus operandi would be to pick up an existing Open Source parser
generator, and submit patches to it that'd allow it to elegantly define
ES4 grammar :-). That's more or less how I got entangled into any OSS
project to date :-)
>
> Sure.
>
> I think, as you point out, the optimizer is one of the parts of Rhino
> that is the hardest for non-authors to understand, and it would be a
> shame to lose this and other functionality if a rewrite happened.
>
> Maybe the best way forward, if major changes are needed, would be for
> the current Rhino design to be written down, a new design to be worked
> out, and determine how the old maps to the new, to make it easier to
> integrate as much of the old code into the new codebase as possible.
It could be done given enough time, of course :-) But as I said, I'm a bit
skeptical. ES4 introduces classes, introduces fixed properties on classes,
all as a measure to better integrate it with strongly-typed languages (at
least that's specified in the "rationale" section of the ES4 doc I read).
That means we might be able to better integrate Rhino with Java, i.e. even
map ES4 classes to Java classes without wrappng (of course, dynamic
properties would have to be handled somehow, so maybe we'll need wrappers
after all). What I wanted to say is that I expect that a ES4 conformant
ScriptableObject would look significantly different than the current one.
Of course, talk is cheap and I can talk as much as I want, the only real
answer to these questions would be to actually roll up the sleeves and do
it :-)
>
> I haven't contributed much to Rhino so far, but I'm willing to help in
> as much as my time permits.
You're naturally welcome to do that.
Attila.
Mike Shaver
> > Maybe http://developer.mozilla.org/es4/ is more up to date, I'm not sure.
>
Yeah, http://developer.mozilla.org/es4/ is the right one. We should
mark the other more clearly as deprecated, or something.
Mike
David P. Caldwell
a couple of E4X bugs that are un-fixable without altering the parser.
I've fixed one (an easy one for which someone else supplied a patch),
and opened an issue for the other problems under bug #368520.
-- David P. Caldwell
http://www.inonit.com/
Hannes Wallnoefer
>
> The current Tamarin source includes an AS3 compiler written in AS3,
> which I understand will be maintained to track ES4 developments. It
> currently produces AS3 bytecode, but making it produce Java bytecode
> (or even just ASTs which can then get Codegen love from another part
> of Rhino) might be a reasonable path to overcoming the parser obstacle
> -- which is indeed quite a bear if taken from first principles.
>
> The bootstrapping process would likely involve compiling
> rhino-compiler.as to Tamarin bytecodes (ABC) and then running the
> result on rhino-compiler.as again to get a version compiled to Java
> bytecodes, but that's basically the same model as is being used on the
> Tamarin side right now anyway, I do believe.
I just had a look at the AS3 parser in Tamarin <http://lxr.mozilla.org/
mozilla/source/js/tamarin/esc/parser.as> and it came to me that it
probably wouldn't be too hard to port this to Java straight away. The
AS3/ES4 syntax and coding style is very close to Java. Certainly it
would be easier than it must have been to port the original rhino
parser from the C version. Wouldn't that be a viable upgrade path for
Rhino?
Hannes