L10n Module in 2.5 cycle - Summary

[zbran...@mozilla.com]

Hi Gaiaers!

I'd like to start a new tradition of providing you with a summary of work that has been done during the a given cycle in my module[*].

It may be surprising for people less familiar with complexity of the l10n ecosystem to notice that our L10n team has been so visible in the number of commits - both me and :stas accruing over 120 commits in this cycle placing us both in top 20 commiters.

The reason is: L10n is complex. It's also severely under-developed compared to most layers of modern app systems. And that is both, a curse and a blessing. A curse because we have no-one to learn from. Nobody has figured it out, no GUI toolkit has a good l10n system and we're with l10n more or less where the webdev world was with HTML4 and PHP.

The blessing is that we have an opportunity at Mozilla to push the state of the whole industry. It's a niche where with limited resources that we have, we can make the Web Stack better than any other stack.
I humbly believe that we are doing it, and 2.5 is an important milestone on that route.


The biggest accomplishment of this cycle for us is the introduction and deployment of new L10n API. This API is based on years of research work by an L10n team in Mozilla led by Axel Hecht. When we took over L10n module in Firefox OS, we started moving the module and refactoring the codebase to work in the paradigms of that approach.

And at the beginning of 2.5 cycle Stas landed revision 3 of our API in Gaia source. Right now it's opt-in: the code needs to link to shared/js/intl/l20n.js to use it.

For the rest of the cycle we improved the new codebase vastly improving performance and memory consumption. At this point L20n 3.0 beats the previous API in features, security, performance and memory.

This, and a lot of refactoring of the code to be more asynchronous, allowed us to move the first four Gaia applications to it - Music, FM, SMS and FTU are fully transitioned and running on top of the new API.

Other highlights:

* Significant improvements in our test coverage, asynchronous non-racy APIs and security of HTML l10n
* Huge progress in reduction of our technical debt. We removed thousands of lines of complex JS code replacing them with simple declarative HTML attributes
* DOM Overlays API allows us to localize Document Fragments
* Bidi markers make composed localized strings work well in LTR/RTL scenarios
* We transitioned whole Gaia to use Intl API for date and time formatting which significantly improves Firefox OS user experience in non-latin based languages
* We moved to use the same Intl API in our composed localization strings for number formatting, so any numbers passed to l10n as variables will be formatted properly for the current locale
* After trying multiple solutions we settled on an amazing module bundler internally, rollup.js, which helped us save memory.
* We leveraged the new internal architecture to properly use client-service bridge architecture in multi-view apps (see Music)
* Made our build system much more strict about l10n errors allowing us to catch many potential race conditions with localization resources
* Introduced first experimental version of our future localization format - l20n
* We deployed mozIntl API that is a staging ground for future JavaScript Intl API
* Lots of work on ECMA TC39 group to get a lot of features needed for Firefox OS into Intl API rev. 3 (ES7).
* migrated our pseudo-locales to work well with Intl API (following Google pseudo-locales rather than Microsoft pseudo-locales)
* We're helping with a lot of performance related work, with both me and :stas becoming peers of the Performance Module and improving our ability to make Gaia apps start fast in any locale
* We are working with JS/Gecko and Gonk teams to fix long standing time/timezone/date problems to make our platform work well across timezones.
* deprecation of the old l10n_date.js library in favor of Intl API and mozIntl
* removal of over 400 calls to the deprecated mozL10n.get method!
* some ground work to enable our whole build/l10n workflow to handle our new l10n format


That's a lot of work. Over 120 commits in gaia 2.5 is just the visible part of the iceberg with over 300 commits to our l20n.js repository and lots of work going to various l10n/intl polyfills, ECMA specs and other repos.

With 2.5 being done, we already started working on 2.6 cycle which we're very excited about because it feels like we have less and less of cleanup work, and we start to work on real improvements and new features! We still have some debt, but we believe 2.6 may be *the* cycle in which we'll finish that part of our work.
So, what's in our plans? Here are some teasers:

* Final rush to remove all mozL10n.get calls (almost 500 calls to remove). Most of that work is in System and Communications
* Integration tests - we now have a framework for that and we can start testing our HTML/DOM API in the browser
* Move (almost) all apps to l20n.js and deprecate l10n.js
* Introduce declarative Intl API for HTML
* Try to remove build time optimizations for l10n
* More work on advancing Intl API and getting 3rd edition of ECMA 402 to have everything we learned we need to make Firefox OS look good
* Start using our new localization format and introduce new l10n features that it allows for
* Move language packs to our build system
* We are starting to look at Web Extensions and browser.html and poke them with a probing stick
* We are experimenting with exposing l20n.js 3.x to the Web community. Supporting some older browsers like IE11 is the current crux.

I don't know how many of those goals we will be able to accomplish in this cycle and how many will have to catch the next train, but with those features, I believe that we will have the best multi locale UI platform in the World. And that's a goal worth pursuing!

Thank you to all reviewers, committers, allies, RTL team, and the amazing sheriffs who had to back out a few eagerly landed patches. We're working on it! :)
zb.


[*] I believe it would be valuable for us to start providing such per-cycle summaries rather than per-quarter summaries. Quarters are artifacts of our management structure, not our project structure.

[Frederik Braun]

Dear people of L10n,
Zibi, Stas,

Your work on l10n and on the security improvements especially is highly
appreciated! It's no understatement to say that this has made Firefox OS
more secure and the work of the security team easier :-)

Thank you for this!

Freddy

[Justin D'Arcangelo]

Amazing work guys! Thanks to the shiny new API in l20n.js, the Music NGA re-write was able to take full advantage of all the goodness it brings, including RTL-compliant time formatting and the client/service split for views via bridge.js.

Also, many, many thanks for all the perf improvements! :-)

-Justin

> _______________________________________________
> dev-fxos mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-fxos

[Michael Henretty]

Thanks for this update zb, please keep sending these!

It is amazing how little you guys break Gaia considering the amount things your code affects. Reforming L10n takes surgical precision. And as Justin said, your focus on performance has made all of FxOS better. So keep it up!

[Naoki Hirata]

Zibi does a lot of manual testing for himself before he checks in the code as far as I know.

He also checks in automation tests.  He also follows up and creates bugs for all the issues he's aware of.

He's asked me a few times when he ran into flashing issues on device or compile issues.

I think he's a pretty good engineer.  :)