An update on strong mode

[Andreas Rossberg]

Hi everybody,

here's a (long overdue!) update on our strong mode experiment. My apologies for the extended silence, but it took some time for us to settle on the new directions.

PROGRESS

First of all, let me say that everything went much slower than we had hoped for. Various random reasons for that not worth going into here, but ES6 eating up all our resources wasn't the least relevant. :)

We finished implementing most of the core of strong mode. Basically, V8 4.9 contains everything from the proposal except throwing on writes, some restrictions on arrays, and a few loose ends with scoping checks. You can find a detailed list at the end of this post.

We also did a limited amount of experiments with applying strong mode to existing code. Specifically, we converted a subset of the Octane benchmark suite to strong mode, the Traceur compiler (partially), and a couple of other examples. We also got valuable feedback from the community, including members of this list. Thanks a lot for that!

We did not make much progress on types last year, but we're hoping that we'll get to that this year. We have just started implementing type parsing in a branch.

WHAT WORKED WELL

So what did we learn? First of all, most of the work of porting code to strong mode actually consists of porting code to contemporary ES6, in particular introducing lexical scoping and classes (which make so much code so much nicer!). Many of the additional strong mode restrictions only require minor tweaks, and turned out to work surprisingly well (on the admittedly limited data set that we have). Most importantly these included:

* Lexical Scoping: We encountered no problems with porting existing code from `var` to `let` and `const`. Likewise, making unbound variables an error works smoothly most of the time, even though a few rare feature detection patterns become slightly more verbose. The one exception is with mutually recursive classes, see below.

Perhaps unsurprisingly, the scoping checks uncovered a number of actual bugs in existing code, e.g. unbound variables in the Octane benchmark.

* No implicit conversions: This perhaps was the most positive. Judging from plenty of feedback, it is by far the most popular strong mode feature, and apparently one of the biggest pain points with the language. Moreover, it was surprising how smooth it was to actually apply this restriction to existing code: none of the code we touched relied on any implicit coercion other than ToBoolean and ToString. The former is still allowed by strong mode, so no issue. For the latter there are two cases: indexed property names, for which strong mode explicitly allows numbers (which is enough), and + with strings. The latter was mostly easy to replace with (often much clearer) ES6 template literals, where string conversion is considered "explicit" and thus legal in strong mode.

* Throwing on failed property access: This was the feature we had worried about most, because returning `undefined` is so deeply engrained in JavaScript and various coding patterns. But we did not encounter too many occurrences of these patterns, and they weren't difficult to spot and rewrite. ES6 features like destructuring and defaults helped as well.

* No property deletion: Again a feature for which there were a-priori interop concerns. But we did not hit any problems yet. This may of course be because we haven't run into the critical programs.

* Disallowing fall-through in switch: Hardly used in practice, and if so, easily rewritten using a local function (something you cannot do in C!).

But more notably, some of the few cases of fall-through that we encountered actually were real bugs! In particular, this occurred in the Traceur compiler.

WHAT NOT WORKED WELL

But not everything is golden. The most serious problems we ran into:

* Locking down classes: This was our biggest hope and the biggest failure. One of the main motivations for strong mode was the desire to restrict classes to behave more as in conventional languages, such that they can be implemented more efficiently. The design consisted of a combination of making classes less mutable, restricting the uses of `super` and `this` in constructors, and most importantly, sealing instances. The constructor restrictions were an issue already, but the sealing idea fell apart entirely! The main problem here was the interoperability with weak classes: we need to allow inheriting strong from weak classes and vice versa (or even alternating inheritance chains), because ruling that out would severely limit the utility of strong mode. But we could not find a semantics that would work without breaking either weak parent classes or weak child classes. In the end we had to give up on this one.

* Mutually recursive classes: the scoping checks turned out to be too restrictive for certain patterns of mutual recursive classes (sequences of class declarations where a latter class inherits from an earlier, while the earlier refers to the latter in a method). These cases could be handled by more relaxed rules, but those would be a bit more ugly, and it's yet unclear if there aren't more problems lurking down the road.

* ES6 classes lack property declarations: This was an issue for two reasons: lacking instance property declarations was one of the main blockers preventing the sealing semantics, because the set of properties cannot be determined at creation time; the lack of static property declarations was an issue for freezing class objects, because in their absence, the ES5 pattern of adding such properties imperatively is still needed (a lot of ES5 code uses this pattern!). A proposal is currently on the way to add property declarations to ES7/8, but until then, it's a blocker for strong mode.

* ES6 performance sucks! Strong mode is a mode for ES6, you cannot use it without using various ES6 features. However, idiomatic ES6 code currently is substantially slower than ES5, across all browsers -- easily by 2x, often by 10x or more. Due to the sheer size of ES6, plus a number of unfortunate design choices, it will likely take years until implementations catch up with ES5 optimisations, and the hundreds of man years that went into those. Until then, strong mode is not going to be an attractive target.

* Implementation complexity: Strong mode tweaks many small bits of the language semantics (see the list at the end). Consequently, it requires special-casing all over the compiler(s), run time, and libraries. In V8, it amounts to literally hundreds of new code paths, and that does not even include libraries. As you can probably imagine, VM implementors are not happy about such added complexity, and it gets in the way frequently. While some of that was to be expected, it was worse than we had anticipated.

TAKE-AWAY

On the plus side, our experiments have showed that (at least some aspects of) strong mode indeed has benefits for code correctness. Also, we believe (though we have no evidence) that strong mode would steer programmers away from performance cliffs lingering in the language.

But our hope for being to able to utilise strong mode for _improving_ performance did not materialise, mainly due to the constraints imposed by the weak mode interop requirement, and the disappointing results for classes. For the foreseeable future, ES6 performance in fact will make strong mode substantially worse. In addition, it adds a lot of cross-cutting complexity to the implementation.

Considering all that, we have reluctantly decided not to pursue strong mode further. We learned some worthwhile lessons, but overall it is not clear that the benefits justify the costs. Starting with the next version of V8, we will hence remove support for strong mode.

We apologise if this is a disappointment! And we want to thank everybody on this list who has followed this experiment (and perhaps had high hopes). In particularly, many thanks to everybody who provided valuable feedback! You can still play with strong mode in Chrome 49, or by checking out V8 4.9, to try for yourself.

TYPES?

Not all is lost. We continue to investigate the addition of types to V8. And in fact, typing may bring some of the benefits of strong mode for lower cost.

However, the strong mode experience, as well as a number of other recent developments (e.g. scientific evaluations of the potential cost of sound gradual typing, more troublesome features in the ES pipeline, but also  the advent of WebAssembly changing the premise for evolving JavaScript) will probably have us readjust our design goals. In particular, we no longer believe that requiring type soundness _by default_ can ever work successfully in JavaScript. It may still be possible to provide sound types as an _opt-in_, though. But more on this at some other time.

APPENDIX: STRONG MODE FEATURES IMPLEMENTED IN V8 4.9

Opt-in

- "use strong"

- --use-strong

- implies strict mode

Scoping

- 'var' is a syntax error

- 'function' is lexically scoped everywhere

- unbound variables are an early reference error

- forward reference to anything but functions or classes is an early reference error

- only allow forward references to functions in the same group

- only allow forward references to classes in the same group

Objects & Arrays

- literals create strong objects

- rest patterns create strong arrays

- array elisions are syntax error

- reading a missing property is a type error

- setting prototype is a type error

- deleting a property is rejected

- freezing non-configurable property is type error

Functions & Generators

- 'arguments' is syntax error

- literals create strong functions/generators

- 'function' bindings are immutable

- functions are not constructors

- functions have no 'prototype'

- functions have no 'caller', 'callee', 'arguments'

- function objects are frozen

- too few arguments are type error

Classes

- "use strong" in constructor body is a syntax error

- 'this' in constructor only for assignment [*]

- no reference to 'this' before 'super' call in constructor

- no reference to 'this' after 'return' in constructor

- no 'this' assignment nested in other statement in constructor

- no use of 'super' in constructor other than call

- no 'super' call nested in other statement in constructor

- no 'return' before the super call in constructor

- no 'return' with value in constructor

- 'null' as a super class is type error

- 'class' bindings are immutable

- class objects are frozen

- class prototypes are frozen

- inheriting from a strong class outside strong mode is a type error [*]

- strong class constructors create strong instances [*]

- instances of strong classes are sealed [*]

Conversions

- numeric operators require number

- '+' requires both numbers or both strings

- '++' and '--' require numbers

- inequality requires both numbers or both strings

- indexing requires number, string, or symbol

Undefined

- binding 'undefined' is syntax error outside destructuring

- assigning to 'undefined' is syntax error outside destructuring

Eval

- direct 'eval' is syntax error

Switch

- fall-through is a syntax error

- declaration in case block is a syntax error

Obsoletion

- 'for'-'in' is syntax error

- 'delete' is a syntax error [*]

- ==' and '!=' are syntax errors

- dependent empty statement is syntax error

Things that have not been finished:

- scoping checks when using 'eval'

- making duplicate properties in object literals an error

- throwing when writing a missing property

- restrictions on arrays' 'length' property

- restrictions on arrays' non-index properties

- making out-of-bounds indices an error for arrays and strings

- invoking a class method on non-instances is a type error

- libraries

[Dave Methvin]

Thank you!  I think we all appreciate you taking the time to give the results of your experiments and it provides a great roadmap for further investigation. Would you be able to share some examples of the code you transformed to strong mode? It might be enlightening to see the diffs.

[Robert Knight]

> Due to the sheer size of ES6, plus a number of unfortunate design choices, it will likely take years until implementations catch up with ES5 optimisations,
> and the hundreds of man years that went into those.

Can you expand a little more on the design choices that were/are proving most problematic for performance?

[Sébastien Doeraene]

Thank you, Andreas, for this analysis!

It's been fun working with Strong Mode in the Scala.js code emitter. The required changes actually brought several improvements to our internal Intermediate Representation and linking pipeline. Although it is disappointing that Strong Mode is canceled, it was a valuable experiment.

Cheers,

Sébastien

--
You received this message because you are subscribed to the Google Groups "Strengthen JS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to strengthen-j...@googlegroups.com.
To post to this group, send email to streng...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/strengthen-js/c2e3fd6a-ab3c-4bfa-8a23-ff5f5c76caeb%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Andreas Rossberg]

On 12 February 2016 at 17:36, Dave Methvin <dave.m...@gmail.com> wrote:

Thank you!  I think we all appreciate you taking the time to give the results of your experiments and it provides a great roadmap for further investigation. Would you be able to share some examples of the code you transformed to strong mode? It might be enlightening to see the diffs.

Not all, but I'll look into sharing the strongfied Octane. Traceur is open anyway.

Be aware, though, that although most of the changes were fairly mechanical, the diffs will be mostly useless, because the vast majority of code consists of method implementations, and the move to ES6 classes thus touches almost every line in those examples, including moving code around.

/Andreas

[Andreas Rossberg]

On 12 February 2016 at 21:40, Robert Knight <robert...@hypothes.is> wrote:

> Due to the sheer size of ES6, plus a number of unfortunate design choices, it will likely take years until implementations catch up with ES5 optimisations,
> and the hundreds of man years that went into those.

Can you expand a little more on the design choices that were/are proving most problematic for performance?

For ES6? Many things, but generally speaking, the fact that there are far more observations and intercession points now, and yet more relevant stuff is mutable at any time. So there are even fewer invariants than there were before. In various cases, the effect probably depends on specific implementation choices of a VM, and affects some more than others.

To give just one specific example that all implementers are struggling with currently is the introduction of the @@species hook. Implementing it in a straightforward manner can easily regresses existing, array-heavy ES5 code by 20% or more. So VMs have to jump through various new hoops and global property tracking for the mere purpose of maintaining the status quo. Let alone code that actually _uses_ these features actively (or that just loads a single library using this feature): ensuring that that isn't many x slower still is man months of more work, and respective complexity.

And that's just one tiny fringe piece of ES6.

/Andreas

[Joe Pea]

Engines and optimizations have advanced since the last time this was considered. Could it be more viable to add static types to JS engines nowadays?