Intent to Implement: Intl.DisplayNames API Proposal

[Frank Tang]

Title: Intent to Implement: Intl.DisplayNames API Proposal

 

Contact emails

ft...@chromium.com

 

Explainer

https://github.com/tc39/proposal-intl-displaynames

https://tc39.github.io/proposal-intl-displaynames/

Design doc/Spec

Engineering Plan https://goo.gl/im9wy4

Summary

Provides a new API to get localized names of language, region, script, currency codes of international standard as well as commonly used names of date fields and symbols.

Motivation

Main motivation for Intl.DisplayNames project was to enable developers to get translation of language, region or script display names on the client. Translation of languages, regions or script display names requires large amount of data to transmit on the network, which is already available in most browsers. These display name translations also carry steep data size penalty for developers. This API will allow web developers to shrink the size of their HTML and/ or ECMAScript code without the need to include the human readable form of display names and therefore reduce the download size to decrease latency. Also, this API will reduce the localization cost for the web developers. Our goal is to expose this data through Intl API for use in e.g. language, region and script pickers, etc.

Benefit

• Reduce download size of apps and therefore improve latency.

• Easy for users to build internationalized language, region or script selection UI components (drop down menu or other kinds).

• Reduce translation cost for developers.

• Consistent translation of language, region and script display name on the web.

Risks

Interoperability and Compatibility

This is a new API. It is currently under TC39 Stage 1 and we plan to propose to advance it to Stage 2 in June 2019.

Ergonomics

Activation

Web developers could use the API immediately upon our shipment.

Debuggability

Nothing special.

 

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Is this feature fully tested by web-platform-tests?

Tests under tc39/test262 will includes many tests to test this API.  

Link to entry on the feature dashboard

https://www.chromestatus.com/feature/4965112605573120

Requesting approval to ship?

“No”. The feature is behind a runtime flag harmony_intl_display_names and I will later send an Intent to Ship email when I am ready to enable by default.

[Mathias Bynens]

Can the doc at https://goo.gl/im9wy4 be made public please?

Usually we only implement features at stage 3. What’s the motivation for implementing this particular API earlier? (Not saying I’m opposed to it.)

[Frank Tang (譚永鋒)]

From: Mathias Bynens <mt...@google.com>
Date: Mon, 13 May 2019 at 18:08
To: Frank Tang
Cc: v8-users, blink-dev, Adam Klein, Frank Yung-Fong Tang, Sathya Gunasekaran, Nebojša Ćirić, <v8-...@googlegroups.com>, Jungshik Shin, Shane Carr

Can the doc at https://goo.gl/im9wy4 be made public please?

It always is.

 

Usually we only implement features at stage 3. What’s the motivation for implementing this particular API earlier? (Not saying I’m opposed to it.)

Since I am the champion of the proposal itself, having a prototype implementation help me to figure out some tricky issues in the specification level. In specific, tricky issues about speed, memory and size that might be avoid if I specify differently. Sometime these thing won't be obvious until implementation. Of course, we can wait till stage 3 then implement it, and then if we find out tricky issues which will increase binary size or cause latency problem, we will have to go back to request to change the specification.

--

Frank Yung-Fong Tang
譚永鋒 / 🌭🍊 
Sr. Software Engineer 

[Frank Tang]

On Mon, May 13, 2019 at 6:56 PM 'Frank Tang (譚永鋒)' via v8-dev <v8-...@googlegroups.com> wrote:

From: Mathias Bynens <mt...@google.com>
Date: Mon, 13 May 2019 at 18:08
To: Frank Tang
Cc: v8-users, blink-dev, Adam Klein, Frank Yung-Fong Tang, Sathya Gunasekaran, Nebojša Ćirić, <v8-...@googlegroups.com>, Jungshik Shin, Shane Carr

Can the doc at https://goo.gl/im9wy4 be made public please?

It always is.

sorry it was not. I got confused since I shared with my other account. It is now. 

--
--
v8-dev mailing list
v8-...@googlegroups.com
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups "v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-dev+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/v8-dev/CA%2B7fzPEZnnAGHh6mijLRByRm2A1xrgNFjSLr3gWf5Qs42bShrA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[PhistucK]

I wonder whether this should be accompanied by an HTML equivalent declarative way of displaying those strings, otherwise the HTML strings might be left blank or JavaScript will be required for filling up what could sometimes be simple forms that were simply internationalized.

Pseudo-code example -

<label>

 <intl template="state-or-province">State/province</intl>:

 <input type="text" required="required" name="state-or-province"/>

</label>

☆PhistucK

You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOcELL8wEy1NrdM10dd9xPmBmrUuy8CrV95eHoOwdg_TzhmLnw%40mail.gmail.com.

[Frank Tang]

Sorry, I am not quite sure what you mean. It seems to me a suggestion of proposing additional standard in HTML spec. Do you mind to raise the issue in https://github.com/tc39/proposal-intl-displaynames/issues with more information so other members in ECMA402 subcommittee can help me to take action to talk to the related parties? 

Thanks 

[Shane Carr]

Hi PhistucK,

Thanks for the feedback!  Adding HTML for Intl features would be out of scope for this feature.  We already have JavaScript-based i18n features like Intl.NumberFormat, Intl.DateFormat, etc., and Intl.DisplayNames is being implemented in the same fashion.

You may be interested in discussions such as:

https://github.com/tc39/ecma402/issues/92

You can also open a new thread on the ECMA 402 repo.

Shane

[PhistucK]

Thank you for your thoughts, all.

I filed an issue with HTML -

https://github.com/whatwg/html/issues/4630

☆PhistucK

[Frank Tang]

Contact emails

ft...@chromium.org, sf...@chromium.org

Explainer

https://github.com/tc39/proposal-intl-segmenter

Specification

https://tc39.github.io/proposal-intl-segmenter/

Design docs

https://docs.google.com/document/d/1xugLpLmgRFnNXK8ztariTAbD2IXueDw1T3VNuuZCz8k/edit#heading=h.xgjl2srtytjt
https://docs.google.com/presentation/d/1X2zBU3bZ4ergVMWfubCsdnHFzeaDgqiTRJVgvNGjQBs/edit#slide=id.p

TAG review

review by ECMA402

Summary

Intl.Segmenter implements methods for finding the location of boundaries in text, including grapheme, line, word and sentence boundary analysis.

Motivation

Currently, chrome is shipped with Intl.v8BreakIterator - a non standard way for similar functionality. According to https://www.chromestatus.com/metrics/feature/timeline/popularity/556 on 2020 Feb there are 0.74% of the web page use it. Intl.Segmenter is the web standard to replace it.

Risks

Interoperability and Compatibility

The specification is moved to Stage 3 in TC39 2020-Jul meeting with support from ECMA402.

Gecko: In development (https://bugzilla.mozilla.org/show_bug.cgi?id=1423593)

WebKit: No signal

Web developers: No signals

Ergonomics

Engineer from Apple believe we should not add line break support to the Intl.Segmenter because the developer may abuse the API and perform text layout by themselves instead of depending on CSS. The line break feature then were removed from the specification in the current shape.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes

Is this feature fully tested by web-platform-tests?

Yes https://github.com/tc39/test262/tree/master/test/intl402/Segmenter

Link to entry on the Chrome Platform Status

https://www.chromestatus.com/feature/6099397733515264

This intent message was generated by Chrome Platform Status.

[Frank Tang]

[Note: Resent due to message header problem. Sorry]

[Joshua Bell]

Awesome! I've been following this API through the TC39 process, as it is a critical primitive for building full-text indexes using storage APIs. 

On Wed, Aug 12, 2020 at 3:17 PM Frank Tang <ft...@chromium.org> wrote:

[Note: Resent due to message header problem. Sorry]

Contact emails

ft...@chromium.org, sf...@chromium.org

Explainer

https://github.com/tc39/proposal-intl-segmenter

Specification

https://tc39.github.io/proposal-intl-segmenter/

Design docs

https://docs.google.com/document/d/1xugLpLmgRFnNXK8ztariTAbD2IXueDw1T3VNuuZCz8k/edit#heading=h.xgjl2srtytjt
https://docs.google.com/presentation/d/1X2zBU3bZ4ergVMWfubCsdnHFzeaDgqiTRJVgvNGjQBs/edit#slide=id.p

TAG review

review by ECMA402

Summary

Intl.Segmenter implements methods for finding the location of boundaries in text, including grapheme, line, word and sentence boundary analysis.

Motivation

Currently, chrome is shipped with Intl.v8BreakIterator - a non standard way for similar functionality. According to https://www.chromestatus.com/metrics/feature/timeline/popularity/556 on 2020 Feb there are 0.74% of the web page use it. Intl.Segmenter is the web standard to replace it.

Risks

Interoperability and Compatibility

The specification is moved to Stage 3 in TC39 2020-Jul meeting with support from ECMA402.

Gecko: In development (https://bugzilla.mozilla.org/show_bug.cgi?id=1423593)

The Gecko bug indicates it was discussed 3 years ago (with a patch?) but I don't see anything meaningful since then. Is there more activity than there seems to be?

 

WebKit: No signal

Web developers: No signals

Ergonomics

Engineer from Apple believe we should not add line break support to the Intl.Segmenter because the developer may abuse the API and perform text layout by themselves instead of depending on CSS. The line break feature then were removed from the specification in the current shape.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes

Is this feature fully tested by web-platform-tests?

Yes https://github.com/tc39/test262/tree/master/test/intl402/Segmenter

Link to entry on the Chrome Platform Status

https://www.chromestatus.com/feature/6099397733515264

This intent message was generated by Chrome Platform Status.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOcELL-m7DZ5OAwZj9FqX9VKZKWYd_Qf0YeaXCs3YAEbcnPsKA%40mail.gmail.com.

[Emilio Cobos Álvarez]

On 8/13/20 12:17 AM, Frank Tang wrote:

[Note: Resent due to message header problem. Sorry]

Contact emails

ft...@chromium.org, sf...@chromium.org

Explainer

https://github.com/tc39/proposal-intl-segmenter

Specification

https://tc39.github.io/proposal-intl-segmenter/

Design docs

https://docs.google.com/document/d/1xugLpLmgRFnNXK8ztariTAbD2IXueDw1T3VNuuZCz8k/edit#heading=h.xgjl2srtytjt
https://docs.google.com/presentation/d/1X2zBU3bZ4ergVMWfubCsdnHFzeaDgqiTRJVgvNGjQBs/edit#slide=id.p

TAG review

review by ECMA402

Summary

Intl.Segmenter implements methods for finding the location of boundaries in text, including grapheme, line, word and sentence boundary analysis.

Motivation

Currently, chrome is shipped with Intl.v8BreakIterator - a non standard way for similar functionality. According to https://www.chromestatus.com/metrics/feature/timeline/popularity/556 on 2020 Feb there are 0.74% of the web page use it. Intl.Segmenter is the web standard to replace it.

Risks

Interoperability and Compatibility

The specification is moved to Stage 3 in TC39 2020-Jul meeting with support from ECMA402.

Gecko: In development (https://bugzilla.mozilla.org/show_bug.cgi?id=1423593)

FWIW, in development seems a bit of a stretch since there hasn't been activity in the bug for a while.

The patch is three years old and there was a bit of a concern due to the binary size growing quite a bit.

(Not an expert on this, but Gecko's layout engine doesn't use ICU for line-breaking, IIRC, so a lot of the ICU data that would be required for this has to be imported).

There may be alternative implementation strategies or what not, but it doesn't seem to be actively worked on.

WebKit: No signal

Web developers: No signals

Ergonomics

Engineer from Apple believe we should not add line break support to the Intl.Segmenter because the developer may abuse the API and perform text layout by themselves instead of depending on CSS. The line break feature then were removed from the specification in the current shape.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes

Is this feature fully tested by web-platform-tests?

Yes https://github.com/tc39/test262/tree/master/test/intl402/Segmenter

Link to entry on the Chrome Platform Status

https://www.chromestatus.com/feature/6099397733515264

This intent message was generated by Chrome Platform Status.

[Shane Carr]

There are some Mozillans who attend our monthly ECMA-402 calls, and they've signed off on the latest version of Intl.Segmenter.  The spec is written such that it's not necessary to use ICU as the breaking engine, and IIRC, the intl team at Mozilla got approval to add in the segmentation data if they decide to go the ICU route.

[Frank Tang (譚永鋒)]

On Thu, 13 Aug 2020 at 02:19, Emilio Cobos Álvarez <emi...@mozilla.com> wrote:

On 8/13/20 12:17 AM, Frank Tang wrote:

[Note: Resent due to message header problem. Sorry]

Contact emails

ft...@chromium.org, sf...@chromium.org

Explainer

https://github.com/tc39/proposal-intl-segmenter

Specification

https://tc39.github.io/proposal-intl-segmenter/

Design docs

https://docs.google.com/document/d/1xugLpLmgRFnNXK8ztariTAbD2IXueDw1T3VNuuZCz8k/edit#heading=h.xgjl2srtytjt
https://docs.google.com/presentation/d/1X2zBU3bZ4ergVMWfubCsdnHFzeaDgqiTRJVgvNGjQBs/edit#slide=id.p

TAG review

review by ECMA402

Summary

Intl.Segmenter implements methods for finding the location of boundaries in text, including grapheme, line, word and sentence boundary analysis.

Motivation

Currently, chrome is shipped with Intl.v8BreakIterator - a non standard way for similar functionality. According to https://www.chromestatus.com/metrics/feature/timeline/popularity/556 on 2020 Feb there are 0.74% of the web page use it. Intl.Segmenter is the web standard to replace it.

Risks

Interoperability and Compatibility

The specification is moved to Stage 3 in TC39 2020-Jul meeting with support from ECMA402.

Gecko: In development (https://bugzilla.mozilla.org/show_bug.cgi?id=1423593)

FWIW, in development seems a bit of a stretch since there hasn't been activity in the bug for a while.

The main reason is there is a long discussion of the approach in the spec and the spec was moved from Stage 3 back to stage 3 last 2 years for the new champion to revise it. It finally reach a better shape and moved to Stage 3 in TC39 in July meeting after getting folks from Mozilla supporting during monthly ECMA402 meeting. 

The patch is three years old and there was a bit of a concern due to the binary size growing quite a bit.

(Not an expert on this, but Gecko's layout engine doesn't use ICU for line-breaking,

I know, I hand wrote that between 1998-2002 when I was Mozilla's i18n module owner and Netscape i18n client manager. They have not changed that part code for the last 20+ years as I know (beside the work I worked with some Thai folks in late 2003 to deal with Thai line breaking).  The current version of Intl.Segmenter spec took out the line break support two years ago so that is irrelevant anyway. 

IIRC, so a lot of the ICU data that would be required for this has to be imported).

I reduced the ICU break rule table size by ~50% in https://github.com/unicode-org/icu/pull/1100 so the data size for break iterator in ICU68 scheduled to be released in late Oct 2020 will be ~230K less than 67. 

There may be alternative implementation strategies or what not, but it doesn't seem to be actively worked on.

WebKit: No signal

Web developers: No signals

Ergonomics

Engineer from Apple believe we should not add line break support to the Intl.Segmenter because the developer may abuse the API and perform text layout by themselves instead of depending on CSS. The line break feature then were removed from the specification in the current shape.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes

Is this feature fully tested by web-platform-tests?

Yes https://github.com/tc39/test262/tree/master/test/intl402/Segmenter

Link to entry on the Chrome Platform Status

https://www.chromestatus.com/feature/6099397733515264

This intent message was generated by Chrome Platform Status.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAOcELL-m7DZ5OAwZj9FqX9VKZKWYd_Qf0YeaXCs3YAEbcnPsKA%40mail.gmail.com.

[Emilio Cobos Álvarez]

I see, thanks for the clarification. If that is so, "Positive signals" may be better than "In development"? It doesn't seem like anybody is working on it (yet).

 -- Emilio

[Emilio Cobos Álvarez]

Sure, to be clear, I wasn't trying to push back. Jbliust wanted to point out that it doesn't seem to be worked on right now, so "in development" doesn't seem quite accurate. "Positive" seems like a more accurate description per this document?

The patch is three years old and there was a bit of a concern due to the binary size growing quite a bit.

(Not an expert on this, but Gecko's layout engine doesn't use ICU for line-breaking,

I know, I hand wrote that between 1998-2002 when I was Mozilla's i18n module owner and Netscape i18n client manager. They have not changed that part code for the last 20+ years as I know (beside the work I worked with some Thai folks in late 2003 to deal with Thai line breaking).  The current version of Intl.Segmenter spec took out the line break support two years ago so that is irrelevant anyway.

That's an awesome bit of trivia, thanks :)

FWIW, it seems that for a bunch of more complex languages (Thai included) nowadays we rely on platform-native APIs instead (see bug 389520, bug 336959, bug 390048, etc).

 -- Emilio

[Frank Tang (譚永鋒)]

Thanks for this info. These are all related to Line Breaking only. As I mentioned, Intl.Segmeter specification , since two years ago in the TC39 Apple meeting, removed line break type so the specification is now without line breaking type. Therefore these issues only related to line break are out of scope and irrelevant to the implementation of Intl.Segmenter.