L20n API 3.0

[Staś Małolepszy]

Hello tools-l10n,

Below you'll find a summary of my thought process behind the next
revision of the L20n API. I split it into three emails for your
reading convenience:

Email #1: Design goals and constraints (this email)
Email #2: Language registration
Email #3: Specifying and modifying the supported languages
Email #4: Getting translations

* * *

L20n API versions 1.0 and 2.0 (see [1] for the story behind versions
and how they map to branches) were based on a concept of a single
localization context which held the state of language negotiation and
resource loading. These contexts also had a synchronous 'get' method
for retrieving translations which in case of errors internally made use
of sync XHR calls to fetch resources in fallback languages.

For the past year of working on Gaia's l10n.js (L20n API 2.0), we have
learned a lot about what a good localization framework should look
like. Here are some high-level design criteria for API 3.0:

Design goals:

G1. when translations fail to load or format the fallback should be
graceful,
G2. apps can be available in more langs than they're bundled with,
G3. it should be easy to create multiple contexts per document.

Design constraints:

C1. user language preferences are an array in which ordering matters,
C2. resources can only be fetch asynchronously

Note that G1 has already been achieved in API 2.0 to some extent.
However, to achieve other goals and keep within the constraints, we
need to make a number of breaking changes to the API. Following is the
run-down, based on the goals that we want to meet.

Consequences:

G1. There's a chain of languages negotiated between the user's
preferred languages (see C1) and the app's available ones; I'll
call them supported languages.

When a translation in one supported language errors out, the next
supported language is fetched to fall back on; this means that
the 'get' method needs to be asynchronous.

G2. An app can register a number of languages with the l10n framework
to say 'here are the languages I'm available in'; this list can
be extended by a language package service, so the registration
needs to be asynchronous.

G3. Multiple contexts per document can still share the state of
language negotiation. It should be possible then to abstract the
language negotiation into a higher-level object.

In the rest of this thread, this higher-level object will be
called a localization environment (env for short). Envs spawn
contexts.

We end up with two classes: environment and context, which interact
with each other and which primarily use async methods. This
interaction is interesting: since all contexts share a common parent
env, we can use the env as a cache to store localization resources.
This way, creating new contexts which want to re-use resources is fast
and easy. In fact, all the information a context should have about
itself is the order of resources in which to look translations up.
Take the following for instance:

var ctx1 = env.require(['res1.{locale}.l20n', 'res2.{locale}.l20n']);
var ctx2 = env.require(['res2.{locale}.l20n', 'res3.{locale}.l20n']);

Actual localization files are fetched and stored by the env. Res2 is
fetched only once. When ctx1.get(id) is called, we first look for the
identifier in res1 and then in res2. We don't even have to store flat
AST for each context: we can simply iterate over the cached resources
in the env.

The exact API of this will be the topic of my next email. To know
which files to fetch, the env has to already have negotiated its
supported languages. This is also where things become a bit tricky :)

Stay tuned,
-stas


[1] https://groups.google.com/forum/#!topic/mozilla.tools.l10n/MJ_sMgqOKT4


--
@stas

[Staś Małolepszy]

The Env API: Do we need env.register?
=====================================

The initialization process of an env can be described in three stages:

1. create an Env instance:

var env = new Env('http://example.com');

2. register the available and the default languages taken from
a manifest:

env.register(manifest).then(…);

This step is async because in the future we will want to query the
language package service about any additional languages available
for this app, or updates to the existing ones.

3. negotiate the user's preferred languages and compute the list of
supported languages:

env.request(navigator.languages);

This step can be sync if we're using our own langnego shim
(Intl.prioritizeLocales); or it could be forced to be async for a)
consistency with the rest of the API and b) future compatibility
with an asynchronous language negotiator (Worker thread, Settings
API etc.).

Apart from the above actions, it should also be possible to create new
contexts once we have an Env instance. Contexts should have access to
env's supported languages so that the correct resources can be fetched.

var ctx = env.require(['res1.{locale}.l20n', 'res2.{locale}.l20n']);
ctx.get('foo').then(function(val) {
console.log(val);
});


My first doubts are about step #2 and env.register. If we expose it as
a separete method, we open up ways to call it out of order. There are
three solutions to this:

- mantain a state of language registration and throw when env.request
and env.require are called before env.register,

- internally wait for env.register to complete when env.request or
env.require are called (e.g. listen to the
'availablelanguageschange' event or have a promise which fulfills
when the manifest is successfully registered,

- pass the manifest (or a promise resolving to the manifest) to the
Env constructor and not expose env.register as part of the API.

In my WIPs so far I went for the third solution which I think is
elegant: it simplifies the code and avoids hidden magic.

https://github.com/stasm/l20n.js/tree/newapi

The env objects are created like this:

var env = new Env('http://example.com', manifest);

'manifest' can be a JSON or a promise resolving to a JSON. A newly
created env has a 'registered' property which is a promise which
fulfills when the language registration (including fetching the
manifest and querying the langpack service) completes.

In my next email, I'll talk about supported languages and env.request.

-stas

--
@stas

[Staś Małolepszy]

The Env API: Do we need env.request?
====================================

Next up, env.request and the supported languages. Similar to
env.register, we run into the ordering problem:

var env = new Env('http://example.com', manifest);

// env.request hasn't been called yet; should env.require throw
// or wait internally?
var ctx = env.require(['res.{locale}.l20n']);

// or, maybe ctx.get should wait for env.request internally?
ctx.get('foo').then(…);

Again, that's a bit too much internal magic for my taste. I see two
potential solutions:

1. pass the user preferred languages as the third argument to the
Env's constructor, or

2. pass the user preferred languages as an argument to env.require.

Let's look at each of those solutions in detail.


#1. An argument to the Env constructor
--------------------------------------

Imagine the following signature of the Env constructor:

var env = new Env('http://example.com', manifest, userLangs);

Now we have all the necessary information to create a fully-operational
env. userLangs is a list of user's preferred languages which is passed
to the language negotiator function which in turn returns the list of
supported languages. This list is stored in the Env instance.

We can now safely create contexts; the env's supported languages (or
a promise to have them) will be used to determine which resources will
be fetched:

var ctx = env.require(['res.{locale}.l20n']);

Since the supported languages are now part of env's state, we also need
a way to modify this state in case the user changes their preferred
languages.

env.request(newUserLangs);

As I mentioned before, I'd like env.request to be async for the sake of
forward-compatibility and also because it could then accept an array of
user preferred languages or a promise which resolves to such an array
(in case user prefs are stored in the Settings API or IndexDB or such
like).

However, an asynchronous env.request method which changes the env's
internal state is a sure recipe for race conditions:

var env = new Env('http://example.com', manifest, ['de']);

var ctx = env.require(['res.{locale}.l20n']);
ctx.get('foo').then(…); // supported languages are: de, en-US

env.addEventListener('languagechange', function() {
ctx.get('foo').then(…); // supported languages are: pl, en-US
});

env.request(['pl']);

ctx.get('foo').then(…); // supported languages at the time
// this line is run are still: de, en-US.

Another problem of this approach is that it's only possible to have one
list of supported locales per env. I'm not sure how much of an
edge-case multi-lingual interfaces are, but I wouldn't like to dismiss
them from the get-go either.


#2. An argument to env.require
------------------------------

As a thought excercise, I tried to imagine a stateless API in which
contexts are bound to a single immutable list of supported languages.

Branch: https://github.com/stasm/l20n.js/tree/immutable
Diff: https://github.com/stasm/l20n.js/compare/85ca771...9e7d466

I experimented with completely removing env.request as well as the
env's languagechange event. Instead, solutions native to the platform
can be used (e.g. navigator.languages and window.onlanguagechange in
browsers).

var env = new Env('http://example.com', manifest);

var ctx = env.require(['de'], ['res.{locale}.l20n']);
ctx.get('foo').then(…); // ctx's supported languages are: de, en-US

In this approach, contexts are immutable: once they're ready for the
list of supported languages they're passed, they remain ready forever.
If the user changes their language preferences, a new context must be
created:

var env = new Env('http://example.com', manifest);

window.addEventListener('languagechange', function retranslate() {
var ctx = env.require(navigator.languages, ['res.{locale}.l20n']);
myApp.render(ctx);
});

Note that since we're removing env.request, there's no need for
a custom languagechange event and instead we can use the platform's
native one.

env.require could also accept a promise as the first argument, which
would play nicely with async storages like mozSettings and IndexDB:

var userLangsPromise = PromisifiedSettings.get('languages.current');
var ctx = env.require(userLangsPromise, ['res.{locale}.l20n']);

I like this solution for its explicit nature and immutability which
makes it impossible to run into race conditions. It also allows to
create contexts with different lists of supported languages.

In my next email I'll talk about the context API.

-stas

--
@stas

[Staś Małolepszy]

The Context API
===============

In my previous emails I proposed the following API to create
localization contexts:

var ctx = env.require(userLangs, resourceNames);

For instance:

var env = new Env('http://example.com', manifest);

var ctx = env.require(['at', 'de'], ['res.{locale}.l20n']);

Let's take a look at what the Context API could look like in detail.

Context have immutable state represented by:

- the list of supported languages (order matters),
- the list of localization resources (order matters).

They also change from 'loading' to 'ready' once during their lifetime.
Since this happens only once, it can be hidden from the API user inside
of ctx.get calls.

Sometimes, however, it's useful to execute code when we know all the
resources have finished loading. We can express this with a Promise
which fulfills when the env has registered the available languages,
when user preferred languages have been retrieved and when the
resources in the first supported language have been fetched.

ctx.ready.then(…);

For retrieving translations, two methods will be helpful:

- retrieve a string value (no attributes):

ctx.format('foo').then(function(val) {
// val is guaranteed to be a string
assert.strictEqual(val, 'Foo');
});

- retrieve a resolved entity with its value and attributes:

ctx.get('foo').then(function(entity) {
assert.strictEqual(entity.value, 'Foo');
assert.deepEqual(entity.attrs, {
title: 'Foo Title'
});
});

Lastly, it should be possible to clean up after a context when it's not
needed anymore. The following method makes sure to remove all cached
resources which are not used by any other existing context associated
with the env:

ctx.destroy();

I implemented a POC of this API onmy branch:

https://github.com/stasm/l20n.js/tree/immutable


* * *

I'm looking for feedback on these ideas. Keep in mind that most of the
localization should be done via the declarative data-l10n-id attribute
in HTML, so what we're dealing with here are rare cases where the JS
API needs to be used imperatively.

Thanks,
-stas

--
@stas

[Axel Hecht]

Hi stas,

the overview looks good, renaming the thread for easier reference.

On 9/18/14 8:50 PM, Staś Małolepszy wrote:
> G2. An app can register a number of languages with the l10n framework
> to say 'here are the languages I'm available in'; this list can
> be extended by a language package service, so the registration
> needs to be asynchronous.

I think I know what you mean here, but folks that didn't dial into our
calls for the last year probably need more detail to understand this.

Maybe that's in email #4, I'll keep on reading.

Axel

[Axel Hecht]

On 9/18/14 8:57 PM, Staś Małolepszy wrote:
> The Env API: Do we need env.register?
> =====================================
>
> The initialization process of an env can be described in three stages:
>
> 1. create an Env instance:
>
> var env = new Env('http://example.com');
>
> 2. register the available and the default languages taken from
> a manifest:
>
> env.register(manifest).then(…);
>
> This step is async because in the future we will want to query the
> language package service about any additional languages available
> for this app, or updates to the existing ones.

These two seem to have a very narrow focus on gaia-style web apps. It'd
be good to understand how these would work in plain-websites, or even
server-side js.

> 3. negotiate the user's preferred languages and compute the list of
> supported languages:
>
> env.request(navigator.languages);
>
> This step can be sync if we're using our own langnego shim
> (Intl.prioritizeLocales); or it could be forced to be async for a)
> consistency with the rest of the API and b) future compatibility
> with an asynchronous language negotiator (Worker thread, Settings
> API etc.).

This is the API where apps and web sites would hook in that want to
manage their own language choice?

Also, how would they trigger a runtime switch of language?

> Apart from the above actions, it should also be possible to create new
> contexts once we have an Env instance. Contexts should have access to
> env's supported languages so that the correct resources can be fetched.
>
> var ctx = env.require(['res1.{locale}.l20n', 'res2.{locale}.l20n']);
> ctx.get('foo').then(function(val) {
> console.log(val);
> });

I'm not a big fan of single Array arguments, unless there's optional
arguments involved.

I'm wondering, which API names have you thrown out for require()? Mostly
asking because it's such an overloaded term in JS already.

> My first doubts are about step #2 and env.register. If we expose it as
> a separete method, we open up ways to call it out of order. There are
> three solutions to this:
>
> - mantain a state of language registration and throw when env.request
> and env.require are called before env.register,
>
> - internally wait for env.register to complete when env.request or
> env.require are called (e.g. listen to the
> 'availablelanguageschange' event or have a promise which fulfills
> when the manifest is successfully registered,
>
> - pass the manifest (or a promise resolving to the manifest) to the
> Env constructor and not expose env.register as part of the API.
>
> In my WIPs so far I went for the third solution which I think is
> elegant: it simplifies the code and avoids hidden magic.
>
> https://github.com/stasm/l20n.js/tree/newapi
>
> The env objects are created like this:
>
> var env = new Env('http://example.com', manifest);
>
> 'manifest' can be a JSON or a promise resolving to a JSON. A newly
> created env has a 'registered' property which is a promise which
> fulfills when the language registration (including fetching the
> manifest and querying the langpack service) completes.
>
> In my next email, I'll talk about supported languages and env.request.
>

I think I prefer #3, too, also because it's not just a matter of calling
methods in different order, but also one of calling methods multiple
times with different arguments.

Axel

[Staś Małolepszy]

Quoting Axel Hecht (2014-09-19 14:26:28)

> On 9/18/14 8:57 PM, Staś Małolepszy wrote:

> > The Env API: Do we need env.register?
> > =====================================
> >
> > The initialization process of an env can be described in three stages:
> >
> > 1. create an Env instance:
> >
> > var env = new Env('http://example.com');
> >
> > 2. register the available and the default languages taken from
> > a manifest:
> >
> > env.register(manifest).then(…);
> >
> > This step is async because in the future we will want to query the
> > language package service about any additional languages available
> > for this app, or updates to the existing ones.
>

> These two seem to have a very narrow focus on gaia-style web apps. It'd
> be good to understand how these would work in plain-websites, or even
> server-side js.

The vernacular might be Gaia-inspired, but this should work the same
for regular websites and server-side. The manifest is just a JSON
describing the available and the default languages, with their
versions.

Take a look at the tests, which are run in node:

https://github.com/stasm/l20n.js/blob/newapi/tests/lib/env/langs_test.js
https://github.com/stasm/l20n.js/blob/immutable/tests/lib/env/langs_test.js

> > 3. negotiate the user's preferred languages and compute the list
> > of
> > supported languages:
> >
> > env.request(navigator.languages);
> >
> > This step can be sync if we're using our own langnego shim
> > (Intl.prioritizeLocales); or it could be forced to be async for a)
> > consistency with the rest of the API and b) future compatibility
> > with an asynchronous language negotiator (Worker thread, Settings
> > API etc.).
>

> This is the API where apps and web sites would hook in that want to
> manage their own language choice?
>
> Also, how would they trigger a runtime switch of language?

So this is what my email #3 is about :) I've had this idea of letting
the developer handle language switching entirely: either by listening
to window.onlanguagechange, or by maintaining their own setting store
and keeping the user's preferred language there.

If the env doesn't have the 'supported language' state, then we don't
have to worry about it in the API.

> > Apart from the above actions, it should also be possible to create
> > new
> > contexts once we have an Env instance. Contexts should have access to
> > env's supported languages so that the correct resources can be fetched.
> >
> > var ctx = env.require(['res1.{locale}.l20n', 'res2.{locale}.l20n']);
> > ctx.get('foo').then(function(val) {
> > console.log(val);
> > });
>

> I'm not a big fan of single Array arguments, unless there's optional
> arguments involved.

Email #3 adds another argument.

> I'm wondering, which API names have you thrown out for require()?
> Mostly asking because it's such an overloaded term in JS already.

True. It used to be called getContext. I considered createContext to
emphasize the fact that there's no caching of contexts going on.
I chose the 'require' name to comply with my self-imposed prototyping
method: methods should describe what they do in one word.

> I think I prefer #3, too, also because it's not just a matter of
> calling methods in different order, but also one of calling methods
> multiple times with different arguments.

Good point!

Overall, I'm liking the "X or a promise" polymorphism; promises excel
at it and I think it makes the API cleaner.

-stas

--
@stas

[Staś Małolepszy]

Quoting Axel Hecht (2014-09-19 13:05:54)

> Hi stas,
>
> the overview looks good, renaming the thread for easier reference.
>
> On 9/18/14 8:50 PM, Staś Małolepszy wrote:

> > G2. An app can register a number of languages with the l10n framework
> > to say 'here are the languages I'm available in'; this list can
> > be extended by a language package service, so the registration
> > needs to be asynchronous.
>

> I think I know what you mean here, but folks that didn't dial into our
> calls for the last year probably need more detail to understand this.

You're right, let me try to describe the scoop.

There are three actors involved:

- the user whose preferred langs are [A, B],
- the app, version v2, which is available in [B v1, C v2, D v2]
(i.e., the B language is outdated),
- the langpack service, which has the following langs for this app
available: [A v2, B v2, E v1]

The app registers the langs its bundled with. The Env queries the
langpack service to check if there are any additional languages it
could use. The final list of available langs for this app is:

[A v2, B v2, C v2, D v2, E v1]

The Env proceeds to language negotiation against user's preferred
languages [A, B].

The result is the list of supported languages:

[A v2, B v2]

…which is much better coverage than the langs the app originally
shipped with: [B v1, C v2, D v2].

HTH,
-stas

--
@stas

[Staś Małolepszy]

Quoting Staś Małolepszy (2014-09-19 15:49:49)

> So this is what my email #3 is about :) I've had this idea of
> letting the developer handle language switching entirely: either by
> listening to window.onlanguagechange, or by maintaining their own
> setting store and keeping the user's preferred language there.
>
> If the env doesn't have the 'supported language' state, then we don't
> have to worry about it in the API.

That said, I think it could make sense to include language-switching
facilities in the HTML bindings, for instance.

-stas

--
@stas

[Staś Małolepszy]

Quoting Staś Małolepszy (2014-09-19 17:29:24)

I wanted to see how easy it would be to implement HTML bindings using
the new API. So I did it :)

You can find the code in the bindings/ directories of my branches:

https://github.com/stasm/l20n.js/tree/newapi
https://github.com/stasm/l20n.js/tree/immutable

Open examples/regular.html in the browser to see it in action.

Since the current HTML bindings logic is rather complex, I simplified
it significantly by making the following changes and consessions:

- the synchronous mozL10n.get method returns 'xxx',
- the synchronous mozL10n.readyState property is always 'complete',
- removed waiting for document.readyState === 'interactive' in favor
of using the defer attribute for l10n.js,
- the mutation observer works,
- language switching works,
- no support for pretranslation via GAIA_PRETRANSLATE=1,
- no support for JSON resources nor inline <meta> manifests via
GAIA_INLINE_LOCALES=1,
- mozL10n.once and mozL10n.ready wait for the default ctx to fetch the
resources for the first supported language; (I also experimented
with just waiting for the language negotiation, but the perf seemed
worse; I'll keep checking if it makes sense to change).

The two most interesting implementation differences were:

- language switching on newapi: I needed to add a way to reset
contexts associated with the env,
- language switching on immutable: each language change creates a new
context,
- mutation observers: it's necessary to pass the observer all the way
down to the function which inserts the translated value in order to
disconnect it temporarily for each translated element.

Overall, I really liked working with this API. It's atomic; it doesn't
do too much and the things it does are limited to small parts of the
logic. It plays nicely with other Web APIs. My current preference
goes to the 'immutable' branch for its greater simplicity and even less
magic.


Performance testing
-------------------

I made the same changes in the current shared/js/l10n.js to even out
the odds in perf testing. I tested with the following commands with
RUNS=5:

GAIA_OPTIMIZE=0 GAIA_CONCAT_LOCALES=0 APP=settings make install-gaia
APP=settings make test-perf

The baseline is the current l10n.js with the above changes applied:

settings (ms) Mean Stdev
------------------------- ---- -----
moz-chrome-dom-loaded 1925 466
moz-chrome-interactive 1925 466
moz-app-visually-complete 3391 458
moz-content-interactive 3391 459
moz-app-loaded 5565 472


settings Base newapi Δ
------------------------- ---- ------ ----
moz-chrome-dom-loaded 1925 1789 -135
moz-chrome-interactive 1925 1790 -135
moz-app-visually-complete 3391 3625 234
moz-content-interactive 3391 3626 234
moz-app-loaded 5565 6028 463


settings Base immutable Δ
------------------------- ---- --------- ---
moz-chrome-dom-loaded 1925 1927 2
moz-chrome-interactive 1925 1927 2
moz-app-visually-complete 3391 3517 126
moz-content-interactive 3391 3517 126
moz-app-loaded 5565 5886 321

I'm not sure yet why immutable scores slower on moz-chrome-* events,
but appears to perform better than newapi on others. I'll continue to
investigate the performance.

-stas

--
@stas

[Axel Hecht]

That's a lot of data, but I'll cherry pick one.

Looking at coding patterns in js land, making navigator.mozL10n.ctx
immutable doesn't feel right. It's just to popular to short-hand that,
to the extent that there'll be a

var _ = navigator.mozL10n.ctx.get.bind(navigator.mozL10n.ctx)

Breaking that code pattern wouldseem to be sad-facing.

Axel

[Staś Małolepszy]

Quoting Axel Hecht (2014-09-22 22:56:39)

> That's a lot of data, but I'll cherry pick one.
>
> Looking at coding patterns in js land, making navigator.mozL10n.ctx
> immutable doesn't feel right. It's just to popular to short-hand that,
> to the extent that there'll be a
>
> var _ = navigator.mozL10n.ctx.get.bind(navigator.mozL10n.ctx)
>
> Breaking that code pattern wouldseem to be sad-facing.

I've come to believe that this is one of the long-standing patterns
that we'll need to break in order to take the API to the next level.

In the async world, having that shortcut around is not very useful
anymore. In the vast majority of cases, you should not be using the
get() method at all. Instead, just assign data-l10n-id. Consider the
following code:

https://github.com/mozilla-b2g/gaia/blob/f911b47f1acfb8e229851d443692115caf8369e0/apps/music/js/music.js#L46-L59
https://github.com/mozilla-b2g/gaia/blob/f911b47f1acfb8e229851d443692115caf8369e0/apps/ftu/js/navigation.js#L175-L299

Both of these files make extensive use of the get() method with
a single goal of assigning the value to textContent or innerHTML. We
want to phase out such use completely.

Next up, we have cases where get() is used deeper in the abstraction
layer and then used by other code. Consider this example:

https://github.com/mozilla-b2g/gaia/blob/f911b47f1acfb8e229851d443692115caf8369e0/apps/communications/contacts/js/tag_options.js

This should probably define "l10nIds" for later resolution instead of
"values" which are resolved when this code is run. Both Gandalf and
I have been advocating for using l10n ids as much as possible and only
resolving them to actual translations at the last possible moment (i.e.
never, just set the data-l10n-id attribute :).

Last but not least, in some rare cases you actually might need to do
a few consecutive get() calls. I think the following code illustrates
this well:

https://github.com/mozilla-b2g/gaia/blob/f911b47f1acfb8e229851d443692115caf8369e0/apps/communications/dialer/js/mmi.js#L137-L145

Here we are definining a set of known possible messages from the
carrier which will be shown to the user once. The translations live in
the Dialer app and they are transmitted via window.postMessage to
somewhere else (which coincidentally already moves us into the async
world, so we're good!).

With an async get() method, it's easy to achieve the same result with
Promise.all, which refers to the get() method only once:

var l10nIds = [
['call-forwarding-status'],
['call-forwarding-voice', { voice: voice || inactive }],
['call-forwarding-data', { data: data || inactive }],
…
];

return Promise.all(
l10nIds.map(
Function.prototype.apply.bind(
navigator.mozL10n.get, navigator.mozL10n)));

-stas


--
@stas

[Staś Małolepszy]

Quoting Axel Hecht (2014-09-22 22:56:39)
> That's a lot of data, but I'll cherry pick one.
>
> Looking at coding patterns in js land, making navigator.mozL10n.ctx
> immutable doesn't feel right. It's just to popular to short-hand that,
> to the extent that there'll be a
>
> var _ = navigator.mozL10n.ctx.get.bind(navigator.mozL10n.ctx)

I'm not sure if something.mozL10n.ctx should be our primary API in the
HTML bindings.

Here's one way I thought this could be laid out:

- document.mozL10n is a DocumentBindings object which is available
right away.

- document.mozL10n instantiates its Env to handle language
registration once the manifest data is available. Since this is
completely l10n-format agnostic, it makes a good candidate for our
first native API; maybe `new navigator.mozLocalization` (to avoid
naming conflicts with the current navigator.mozL10n).

- document.mozL10n also has its default context (or a default resource
bundle) which uses the resources defined in <head>. If contexts are
immutable it might be a good idea to make this default context
private; in the future we'll want to make it possible to add new
resources to <head> declaratively and have document.mozL10n pick
them up automatically, which will require recreating the default
context. The same goes for language switching on my 'immutable'
branch.

As you can see, I'd like Env and Context to be a low-level web API
which can be used to build bigger abstractions. I think our HTML
bindings should be a consumer of those low-level APIs.

One more thing that I've been thinking about is this:

- document.mozL10n could also have its own get() and format() methods
which delegate to the default ctx if it's available or intercept the
calls and wait for the Env to be created.

I'm not sure if I like this last idea or not. Part of me thinks that
we should allow developers to use the document.mozL10n API even before
the manifest is fetched and the Env is created. Another part prefers
an API design which assumes some knowledge of how browsers parse HTML
and relies on the correct use of the defer attribute, a meaningful
ordering of <script> tags and maybe a special event called
'languagesregistered'.

-stas

--
@stas

[Richard Olsson]

I've been reading through this thread after a chat with stas, and although (or perhaps, because) I'm new to L20n and have very little insight (or interest, tbh) in the FxOS/Gaia project, I'd like to provide some feedback.

I came to L20n via the tutorial on l20n.org and the documentation linked to from there (in the GitHub master branch). I've only just started out, but coming from gettext I'm already in love with the l20n language and am looking at implementing it in a major product of ours that's in the tech planning stage right now.

My main takeaway from this thread however is that things are getting overly complicated. I like the current synchronous API, which is completely enough for our needs. More importantly, since we won't actually be doing much HTML (using React.js and building a virtual DOM from JS) the HTML bindings are uninteresting to us.

The current API (GitHub master, which I guess is 1.0?) currently allows me to very easily request a locale, wait for it to load, translate (and render) all the strings. If the user switches languages, I will request a new locale and repeat. Each component can render the strings sychronously once they have been loaded.

Since we are in complete control of our application, we don't need the amount of runtime fallback logic that this API is being designed to accommodate. Quite the opposite really, if a string is missing I want to fail in some way (exception, or return entity name) so that we can catch that in our testing.

With this new API proposal, I reckon this simple (and for web apps fairly common?) use case will be bloated for the sake of a less common use case. I guess this approach is based on the needs of FxOS/Gaia? Maybe I'm just misunderstanding.

I'm not sure what would be the best way to accommodate both use cases. Maybe two completely separate SDKs? I guess a strength with a project like this is that the core value of it is the ubiquitous l20n language, and it's entirely ok to have a ton of SDKs developed on top of it.

[Brandon Paton]

Hi Richard,

Agreed that L20n is a bit complicated for most use cases.

About a year ago I created Localize.js for this reason (https://localizejs.com <https://localizejs.com/>). We use simple HTML tag markers to accomplish interpolation (variables) and pluralization. I’d love to see L20N adopt this approach as well — it’s worked well for our users so far: https://localizejs.com/docs/usage/variables <https://localizejs.com/docs/usage/variables>

Brandon

> _______________________________________________
> tools-l10n mailing list
> tools...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/tools-l10n

[Richard Olsson]

Hi Brandon,

To clarify: I did not say that I think L20n (the language) is too complicated. On the contrary I think it's a very well designed language. I do however think that the _javascript API_ as it's evolving is getting too complicated. This is not a huge problem, because I could write my own API around the open parser.

I did however provide this feedback as input coming from a fresh pair of eyes, to start a discussion amongst the contributors about what would be the best way forward. I did not post it as an opportunity for you to shamelessly plug your proprietary, unfree (beer and freedom) SaaS, although I wish you the best of luck with it.

Because I'm very new I don't know about the rules or etiquette of this list, so excuse me if I'm being overly harsh towards this kind of thing. I'd prefer if this discussion going forward would get back to being about the API, and would appreciate any thoughts on the feedback I provided in my previous e-mail.


Cheers
/R

On Jan 24, 2015, at 2:01 AM, Brandon Paton <b...@brandonpaton.com> wrote:

> Hi Richard,
>
> Agreed that L20n is a bit complicated for most use cases.
>

> About a year ago I created Localize.js for this reason (https://localizejs.com). We use simple HTML tag markers to accomplish interpolation (variables) and pluralization. I’d love to see L20N adopt this approach as well — it’s worked well for our users so far: https://localizejs.com/docs/usage/variables

[Zibi Braniecki]

On Friday, January 23, 2015 at 5:03:09 AM UTC-8, Richard Olsson wrote:
> My main takeaway from this thread however is that things are getting overly complicated. I like the current synchronous API, which is completely enough for our needs. More importantly, since we won't actually be doing much HTML (using React.js and building a virtual DOM from JS) the HTML bindings are uninteresting to us.
>
> The current API (GitHub master, which I guess is 1.0?) currently allows me to very easily request a locale, wait for it to load, translate (and render) all the strings. If the user switches languages, I will request a new locale and repeat. Each component can render the strings sychronously once they have been loaded.
>
> Since we are in complete control of our application, we don't need the amount of runtime fallback logic that this API is being designed to accommodate. Quite the opposite really, if a string is missing I want to fail in some way (exception, or return entity name) so that we can catch that in our testing.

This is an awesome piece of feedback.

We need this. The problem we face is that we are designing L20n to be platform and system agnostic, but the first platform we're shipping on is FirefoxOS which means vanilla webapps.

It's good because it involves modern platform, and front-end work which L20n helps most with, but it also means that we keep things like python/django/node/angular/react etc. implementations only in the back of our heads and we kind of run our mental model experiments each time we update the API design with "Will that work for django? Hmm, it should" - done.

Lack of experience with JS frameworks makes it harder for us to understand the interactions between l20n and those frameworks especially in the areas where they may overlapp (our responsive l10n vs. two-way data bindings or live retranslation case).

I believe that this is artificial and l20n should be able to plug into those frameworks easily. L20n as a library should be standalone, pluggable and bindings agnostic. Then, bindings should be enabled when l20n is localizing vanilla webapp and bindings should be the part that we're pushing on WebAPI track.

This separation of concerns would help us tremendously and make us a good citizen in the frameworks world.

zb.

[Richard Olsson]

Thanks for your feedback (to my feedback ;))

First of all I just want to be clear that the L20n language as it stands is absolutely fine IMO. I'm just talking about the JS API (and about the design of APIs for L20n in general).

Maybe it would make sense to think of the API as two-tier, one low level layer and another higher-level. The lower level is essentially the parser/compiler, with a synchronous get() method to retrieve an entity by ID from a loaded resource (where resource is a first class object).

The higher level deals with the bits that need to be asynchronous, such as locale negotiation and loading, "responsive l10n", HTML bindings and other things that need to be implemented as callbacks, wrapping it all in a environment/context. Still, underneath, they will just be invoking that same get() method on the currently active resource.

The two tiers would probably live in the same repository, but would need to be separated such that the low-level API could be used without the need for the high level module to be included.

This would make it possible for someone who's just getting started, and people not using frameworks, to use the high-level, asynchronous APIs. At the same time, it would allow people who use frameworks that prevent them from modifying the DOM directly for example (and who thereby can't hand that responsibility over to L20n.js callbacks) to use the get() method directly within their framework callbacks.

It would also allow for people to take the parser/compiler and wrap it in something that makes sense for their framework of choice, and redistribute it as react-l20n or whatever. The same approach could be taken to any language, within the constraints of that language/environment.

Because all I really want to do is load a LOL file, and get entities from it. I might even bundle the LOL in the HTML (based on a user default language setting available to the server code) and load it from there. I still completely see the use case for all of that other stuff, but I don't want it to get in the way of the core functionality. :)

So basically the core, lower-level tier would just be this:

// Loading, which obviously needs to be async
var loadedResource = L20n.loadResource('/locale/en/strings.lol').then(function() {
var entity = loadedResource.get('myStringId');
});

// Compiling from text (could be from HTML, XHR or whatever)
var lolText = document.getElementById('default-lol').innerText;
var stringResource = L20n.compile(lolText);
var entity = stringResource.get('myStringId');


Need negotiation and fallbacks, HTML bindings and the likes? Use the high-level API or build it yourself. Need responsive l10n? Use the high-level API or just get() on the window resize event. Etc.

How about it?


Cheers
/R