Differences between Parser API and Esprima harmony

[Yusuke SUZUKI]

Hi all.

Unfortunately, there's differences between Parser API and Esprima harmony
branch.
For example, Esprima harmony defines ArrowFunctionExpression, but Parser
API defines ArrowExpression.
This doesn't sound nice for developers using Parser API.

Do you have any plan about this?
I think that a open process to propose a new AST node interface is needed.
This resolves differences among parsers and makes Parser API spec more
powerful.

Thanks,
Yusuke Suzuki

[Luke Wagner]

This brings up a related question: what is the proposed future of the builtin Reflect.parse now that we have pure-JS alternatives like Esprima?

On the 'keep' side, I've found jsreflect to be simple, well-written, and easy to maintain. Also, there seem to be a existing uses of Reflect.parse from devtools which could potentially benefit from the native performance (but I'm not sure how it is used).

On the 'remove' side, Reflect.parse isn't exposed to content and I'm not aware of any specification effort being planned (it seems like the motivation would be low, given Esprima et al). Furthermore, if we have it, we shouldn't let it languish which means some amount of ongoing maintenance effort to address issues like the one below (which might not end up being too much work).

Does anyone have any opinions either way? I'm mostly on the fence.

Cheers,
Luke

> _______________________________________________
> dev-tech-js-engine-internals mailing list
> dev-tech-js-en...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals
>

[Andy Wingo]

On Fri 06 Sep 2013 18:32, Luke Wagner <lu...@mozilla.com> writes:

> This brings up a related question: what is the proposed future of the
> builtin Reflect.parse now that we have pure-JS alternatives like
> Esprima?

On the "remove" side, Reflect.parse makes it impossible to do
"desugaring" at the ParseNode level. Removing it would enable the
parser to express more things in terms of the AST, which would be useful
in new ES6 features.

Andy

[Brandon Benvie]

On 9/6/2013 9:32 AM, Luke Wagner wrote:
> Also, there seem to be a existing uses of Reflect.parse from devtools which could potentially benefit from the native performance (but I'm not sure how it is used).

One recent feature added (which exposed the ArrowExpression discrepency)
was a "beautify" feature added to the Scratchpad
(https://bugzilla.mozilla.org/show_bug.cgi?id=906640). It uses
Reflect.parse to generate the AST and escodegen to then decompile the
AST back to source (since Reflect.decompile was never implemented). This
fails whenever there's discrepancies between the ASTs, which happens
mostly because of Mozilla specific syntax extensions, but also in this
case where different names were used for the same standardized feature
(arrow functions).

[David Bruant]

Le 06/09/2013 18:32, Luke Wagner a écrit :
> This brings up a related question: what is the proposed future of the builtin Reflect.parse now that we have pure-JS alternatives like Esprima?

> On the 'keep' side, I've found jsreflect to be simple, well-written, and easy to maintain. Also, there seem to be a existing uses of Reflect.parse from devtools which could potentially benefit from the native performance (but I'm not sure how it is used).
Esprima might not want to support Firefox-specific syntax (granted,
which there is less and less and hopefully which will fully disappear
eventually). But Mozilla may contribute a patch with a parsing option
for its specific syntax.

How much better is Reflect.parse performance?

I'm also happy that there exists 2 implementations as it forces
independent people/teams to agree on what is best (exactly what is
happening in this thread) instead of just shipping something
unilaterally (regardless of how well-intended).

David

[Brandon Benvie]

On 9/6/2013 9:46 AM, David Bruant wrote:
> How much better is Reflect.parse performance?

Well I just ran a comparison and got about 7x faster
(https://gist.github.com/espadrine/3262696). It's a year old version of
esprima but I doubt the differential could be that much.

[Yusuke SUZUKI]

On the 'keep' side, new syntax introduced into SpiderMonkey is basically
supported by Reflect.parse and it can be the reference implementation to
the other parsers.

On Sat, Sep 7, 2013 at 2:54 AM, Brandon Benvie <bbe...@mozilla.com> wrote:

> On 9/6/2013 9:46 AM, David Bruant wrote:
>
>> How much better is Reflect.parse performance?
>>
>
> Well I just ran a comparison and got about 7x faster (

> https://gist.github.com/**espadrine/3262696<https://gist.github.com/espadrine/3262696>).

> It's a year old version of esprima but I doubt the differential could be
> that much.
>

> ______________________________**_________________
> dev-tech-js-engine-internals mailing list
> dev-tech-js-engine-internals@**lists.mozilla.org<dev-tech-js-en...@lists.mozilla.org>
> https://lists.mozilla.org/**listinfo/dev-tech-js-engine-**internals<https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals>
>

[Luke Wagner]

That is a lot faster. I assume beautification happens on the whole file at once (hopefully in a WebWorker :)? For a concrete basis, I timed zimbra_combined.js (in parsemark) and it took 1x with the native Reflect.parse, so the 7x factor would definitely be noticeable. This is a pretty strong point on the 'keep' side.

(For fun, I also tried Epic Citadel (24MB of minified JS, unzipped) and it took 36 seconds!)

----- Original Message -----
> On 9/6/2013 9:46 AM, David Bruant wrote:
> > How much better is Reflect.parse performance?
>
> Well I just ran a comparison and got about 7x faster

> (https://gist.github.com/espadrine/3262696). It's a year old version of

> esprima but I doubt the differential could be that much.

[Luke Wagner]

That's a good point. I was thinking about this and it seems like we could flag the ParseNode somehow so that Reflect.parse could resugar. This sounds a bit like the old bad times in the decompiler (/me involuntarily shudders at memories of decompiling desugared destructuring let), but I suspect we'd have a much easier time with a parse tree. Having a more concrete idea of what you might want to desugar in the parser, do you think this would work?

----- Original Message -----
> On Fri 06 Sep 2013 18:32, Luke Wagner <lu...@mozilla.com> writes:
>

> > This brings up a related question: what is the proposed future of the
> > builtin Reflect.parse now that we have pure-JS alternatives like
> > Esprima?
>

[Jason Orendorff]

Yusuke, thanks for bringing this up.

Ariya, I think we should switch to ArrowFunctionExpression, so I filed a
bug against Reflect.parse to make that change.[1]

The names we use are listed in [2] in case you ever want to check. When
you add new node types, please send mail to this list, and I'll try to
do the same.

`yield * EXPR` is being implemented in SM right now[3]. The structure
proposed in the patch is different but I'll suggest switching to what
Esprima does in my code review.

-j

[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=913617
[2]: https://mxr.mozilla.org/mozilla-central/source/js/src/jsast.tbl
[3]: https://bugzilla.mozilla.org/show_bug.cgi?id=666396

[Yusuke SUZUKI]

Sounds good to me. Thanks Jason.

So I'll track this list to implement a new Node in Esprima with the same
name as the name already used by SpiderMonkey.
Ariya, do you have any comments?

And I'll start supporting ArrowFunctionExpression in Escodegen[1]. :D

[1]: https://github.com/Constellation/escodegen/issues/127

Thanks,
Yusuke Suzuki

2013年9月7日土曜日 Jason Orendorff joren...@mozilla.com:

> Yusuke, thanks for bringing this up.
>
> Ariya, I think we should switch to ArrowFunctionExpression, so I filed a
> bug against Reflect.parse to make that change.[1]
>
> The names we use are listed in [2] in case you ever want to check. When
> you add new node types, please send mail to this list, and I'll try to do
> the same.
>
> `yield * EXPR` is being implemented in SM right now[3]. The structure
> proposed in the patch is different but I'll suggest switching to what
> Esprima does in my code review.
>
> -j
>

> [1]: https://bugzilla.mozilla.org/**show_bug.cgi?id=913617<https://bugzilla.mozilla.org/show_bug.cgi?id=913617>
> [2]: https://mxr.mozilla.org/**mozilla-central/source/js/src/**jsast.tbl<https://mxr.mozilla.org/mozilla-central/source/js/src/jsast.tbl>
> [3]: https://bugzilla.mozilla.org/**show_bug.cgi?id=666396<https://bugzilla.mozilla.org/show_bug.cgi?id=666396>
>

[Andy Wingo]

Hi,

On Fri 06 Sep 2013 21:49, Luke Wagner <lu...@mozilla.com> writes:

> I was thinking about this and it seems like we could flag the
> ParseNode somehow so that Reflect.parse could resugar.

It could work, but it seems quite tricky to get right. I can imagine
lots of debugging related regressions. I don't know how Reflect.parse
is really used though.

I was thinking of using it to desugar yield to take another argument,
the generator object. That object would be bound to a "hidden" variable
-- allocated like any other, but not visible to eval or Reflect.parse.
That's not really affected by this issue, though.

I was thinking of doing something similar with yield* -- encoding the
delegating loop using AST nodes. But it turned out that's just as
possible with the bytecode emitter, though it takes longer to get right,
and even now I'm not 100% sure I have it right. Doing it in the parser
would be more convenient, but if there is still Reflect.parse, I'm not
sure how I'd make that desugared loop appear as "yield*".

Andy