Intent to implement: ES6 `String.fromCodePoint` and `String.prototype.codePointAt`

[Mathias Bynens]

# TC39 acceptance

Norbert Lindenberg fleshed out a proposal that defined these methods back in May 2012: <http://norbertlindenberg.com/2012/05/ecmascript-supplementary-characters/#String>. Since then, the document received multiple rounds of feedback through the es-discuss list and during TC39 meetings, and and has been revised multiple times.

Finally, Allen-Wirfs Brock made the algorithms for these methods more consistent with `String.fromCharCode` and `String.prototype.charCodeAt` and added them to the ECMAScript 6 draft specification (July 8, 2012). Since then, no changes were made to these definitions, indicating their behavior is fairly stable.

One spec bug remains to be fixed: <https://bugs.ecmascript.org/show_bug.cgi?id=1153>. However, it’s pretty clear the intent is to return `undefined` and not `NaN` (the algorithms in both the proposal and the ES6 draft agree on it).

There’s also a discussion on whether JavaScript should allow lone surrogates in strings or not: http://esdiscuss.org/topic/code-points-vs-unicode-scalar-values But historically, this is what JavaScript has always done, and changing this behavior might break backwards compatibility or introduce new forms of hazardous silent data corruption.

# Interest from other browser vendors

Firefox/SpiderMonkey has a patch to implement these features: https://bugzilla.mozilla.org/show_bug.cgi?id=918879 Note: the patch doesn’t hide the feature behind an optional flag.

# Testing

This patch implements the feature under the `--harmony` flag: <https://codereview.appspot.com/13816046/>. It also adds conformance tests for these methods, taken from <https://github.com/mathiasbynens/String.prototype.codePointAt/blob/master/tests/tests.js> and <https://github.com/mathiasbynens/String.fromCodePoint/blob/master/tests/tests.js>. All conformance tests pass.

[Mathias Bynens]

On 24 Sep 2013, at 12:17, Mathias Bynens <mat...@qiwi.be> wrote:

> There’s also a discussion on whether JavaScript should allow lone surrogates in strings or not: http://esdiscuss.org/topic/code-points-vs-unicode-scalar-values But historically, this is what JavaScript has always done, and changing this behavior might break backwards compatibility or introduce new forms of hazardous silent data corruption.

This was apparently resolved during last week’s TC39 meeting: <https://mail.mozilla.org/pipermail/es-discuss/2013-September/033559.html> Excerpt follows:

> #### Consensus/Resolution
>
> - No Change. If you want to use lone surrogates you'll have to escape them after JSON.stringify().

@Dmitry, given that this issue was your main concern regarding this patch, what do you think?

[Daniel Clifford]

Thanks for reaching out. I agree with Dmitry's assessment in 13816046. Regardless of TC39 acceptance and remaining spec issues, I see no advantage into baking this into V8 now rather than using it as a polyfill, especially since it would be under a flag. I also don't classify this as one of the features that requires early prototyping in the core V8 code base to test feasibility and answer architectural questions. Given the limited upside, possible feature flux, and no clear urgency for adding now, I advise waiting to land this. 

Danno

--
--
v8-users mailing list
v8-u...@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups "v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Daniel Clifford]

Sorry, adding Mathias back in, whom I accidentally dropped from the recipients list.

[Dmitry Lomov]

Mathias,

we appreciate your effort in implementing this, and are thankful to you for following our launch process and coming up with a good intent-to-implement e-mail, supported by reasoned argument and conformance tests.  (Yours is our first intent-to-implement e-mail under the new process and I am really glad to see it being as good as yours - thanks much for that!)

However, the consensus on V8 team is not to implement these APIs at this time, even under experimental flag.

(besides Daniel's reply below, see my comment [1], and a comment from Erik Arvidsson [2] for additional TC39 perspective).

V8 team defers to TC39 in this case to ensure that this particular addition to ECMAScript is sound and correct. 

For these APIs, implementers' feedback is not crucial for making decisions, and since the APIs are easily polyfillable, no significant new capabilities will be denied to web developers if we do not implement this.

Based on our launch process criteria, at this time usefulness of this addition ("Impact on Web Platform") *does not* outweigh compatibility risks (no finalized spec, no shipping support in other browsers).

Thanks again and kind regards,

Dmitry 

[1] https://codereview.appspot.com/13816046/#msg5

[2] https://codereview.appspot.com/13816046/#msg7

[Mathias Bynens]

Does https://code.google.com/p/v8/issues/detail?id=2840#c3 mean that the patch is now accepted?

[Sven Panne]

On Tue, Oct 22, 2013 at 7:01 PM, Mathias Bynens <mat...@qiwi.be> wrote:

Does https://code.google.com/p/v8/issues/detail?id=2840#c3 mean that the patch is now accepted?

Nope, "Accepted" in the issue tracker just means that the issue looks valid and we intend to look into it at some point. Acceptance of the patch itself would be done via an LGTM of a project owner in the CL https://codereview.appspot.com/13816046, which is probably the wrong review site BTW (should be on https://codereview.chromium.org).

[Mathias Bynens]

Support for these methods just landed in Firefox/SpiderMonkey: http://hg.mozilla.org/integration/mozilla-inbound/rev/2411714cd058 Note: the new methods aren’t hidden behind an optional flag.

It is time to reconsider this for v8, and take another look at the patch? https://codereview.appspot.com/13816046

[Mathias Bynens]

For the record, Firefox 29 just shipped with support `String.fromCodePoint` and `String.prototype.codePointAt` by default (not behind a flag).

It is time to reconsider this for V8, and take another look at the patch? https://codereview.appspot.com/13816046