HTML page localization proposal
Myk Melez
I have written the initial draft of a proposal for localization of HTML
pages in SDK-based addons:
https://github.com/mozilla/addon-sdk/wiki/HTML-Page-Localization
The proposal intends to build on and complement the existing work on
localization and provide a simple mechanism for addons to localize
static and dynamic text in their pages.
Please take a look, and let me know what you think!
I'm particularly interested in feedback from folks who use template
processors like jQuery Templates or mustache.js in addon pages, as I
want to minimize conflicts with such processors to enable the broadest
possible use of popular JS libraries alongside localization in addon pages.
-myk
Jeff Griffiths
functionality. My only concern would be that the use of the
gettext-ish '_' character might complicate use of the underscore.js
Javascript library.
This doesn't seem terrible however, considering that underscore.js has
functionality built-in to work around naming conflicts.
Jeff
> --
> You received this message because you are subscribed to the Google Groups
> "mozilla-labs-jetpack" group.
> To post to this group, send email to mozilla-la...@googlegroups.com.
> To unsubscribe from this group, send email to
> mozilla-labs-jet...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/mozilla-labs-jetpack?hl=en.
>
Myk Melez
That looks excellent, particularly regarding the static processing functionality. My only concern would be that the use of the gettext-ish '_' character might complicate use of the underscore.js Javascript library. This doesn't seem terrible however, considering that underscore.js has functionality built-in to work around naming conflicts.
let str = require("l10n").get;
console.log(str("Hello, world!"));
Doing this complicates plans to auto-generate the default .properties file by scanning the script for string retrieval call sites, at least until we integrate a real script parser (bug 683788). But it would allow us to resolve conflicts with underscore.js, if we expose the API to the pages via the CommonJS module.
-myk
Alexandre poirot
So overall It seems like a good approach to first propose static localization and then dynamic one.
- One way to fix conflicts with web framework translation would be to give a void to disable this l10n parsing.
Don't you think we should have a specific protocol for localization?
Or a pattern in addon:// URLS to enable/disable localization parser?
It sounds like we should gather all features we would like to put into addon: URLS and see how to mix those.
- Do you consider using "gettext" pattern in static localization. i.e accept localized strings in the template. So that an HTML page can be localizable without having to create any properties file and match same features/behavior than dynamic API?
Something like: <head>${Simple page}</head> (Needs a rule to escape key delimiter, here `}`)
- References within references. It is a great idea and it should be part of the common code between content and addon main l10n modules!
- So if I got it correctly, what you are describing in second paragraph "dynamic localization" is part of non-goals? It would be clearer if you remove this non-goals note and put a comment in this paragraph.
- Currently, there is only one big property file. Do you think we will need more than one for the use of static localization?
I have some concerns with the existing pattern about:
* code sharing (being able to share a piece of code with its related properties)
* keys conflicts (if two static files use the same key)
- Do you think we will be able to automatically fetch keys used for static localization with cfx?
Would it be reasonnable to fetch all ${.*} patterns in all html files from data folder?
Same question applies to dynamic localization. My main concern is that I'd really like to implement a feature that will automatically build the list of all used `keys`!
Myk Melez
- One way to fix conflicts with web framework translation would be to give a void to disable this l10n parsing.
Don't you think we should have a specific protocol for localization?
Or a pattern in addon:// URLS to enable/disable localization parser?
And I would enable localization by default, as we want addons to be localizable, and addon developers want that too. Plus we need to make sure that the localization processor doesn't conflict with common JavaScript libraries anyway; making localization optional wouldn't excuse us from identifying and addressing such conflicts.
However, I can imagine making localization processing disablable if we find that users encounter conflicts we cannot easily resolve. I have added a note to that effect.
- Do you consider using "gettext" pattern in static localization. i.e accept localized strings in the template. So that an HTML page can be localizable without having to create any properties file and match same features/behavior than dynamic API?
Something like: <head>${Simple page}</head> (Needs a rule to escape key delimiter, here `}`)
- References within references. It is a great idea and it should be part of the common code between content and addon main l10n modules!
- So if I got it correctly, what you are describing in second paragraph "dynamic localization" is part of non-goals? It would be clearer if you remove this non-goals note and put a comment in this paragraph.
So this:
document.body.appendChild(_("Hello, World!"));But not this:
document.body.innerHTML = _("<p>${helloWorld}</p>");
I do think dynamic localization of HTML is a valuable feature, and we should consider implementing it at some point; I just don't think it's necessary to do so in the first iteration.
I have updated the document to clarify this difference.
- Currently, there is only one big property file. Do you think we will need more than one for the use of static localization?
I have some concerns with the existing pattern about:
* code sharing (being able to share a piece of code with its related properties)
* keys conflicts (if two static files use the same key)
- Do you think we will be able to automatically fetch keys used for static localization with cfx?
Would it be reasonnable to fetch all ${.*} patterns in all html files from data folder?
Same question applies to dynamic localization. My main concern is that I'd really like to implement a feature that will automatically build the list of all used `keys`!
One other thought I had is that we could make the processor identify all text nodes (and localizable attribute values, like the "alt" attribute to the "img" tag) and use their values as keys automatically, so users wouldn't have to specify delimited keys.
I'm not sure how feasible or preferable that is, but I've added it to the doc as a possible alternative.
-myk
Irakli Gozalishvili
<html>
<head>
<title data-l10n>title</title>
</head>
<body data-l10n>
Hello <span data-l10n>world</span>
</body>
</html>
kazé
I’m following your work because we need something very similar for Gaia. FWIW, I think Irakli’s proposal makes a lot of sense for the two points he’s raised:
- the underscore could be a “synonym” of a more explicit/namespaced function, but we probably can’t keep it as the only way to access localized strings;
- using data-l10n attributes seems just right (“more htmly” indeed), and that’s close to what the l20n guys are working on.
It relies on the good old *.properties file format. To include those l10n resources in an HTML document I propose using a <link /> element: https://github.com/fabi1cazenave/webL10n/blob/master/index.html#L6
where “demo/data.properties” is the default l10n resource (fallback), and translation files use a [.lang] suffix — e.g. demo/data.properties.fr for the French translation. That’s the simplest solution I could think of, but I’m open to suggestions here (and I’d like to have a more explicit “rel” attribute).<link rel="resource" type="text/l10n" href="demo/data.properties" />
As you can see, I’ve used data-l10n-id attributes because I prefer using specific ID strings instead of using the innerText as implicit identifiers (and because it’s more l20n-friendly), but I guess both approaches could be complementary. I like the idea of nesting localizable nodes but how would it work?
I also suggest using an [identifier.attribute] notation to fill some localizable attributes. In this example, “Phone.title” fills the “title” attribute of the HTML element with the data-l10n-id="Phone" identifier.
I’m not proposing any JS API yet. I confess I’m not familiar yet with the JetPack l10n API, nor with the l20n API.
I hope I’m not interfering with your work. I’d really love it if we (= jetpack + gaia + l20n) could come up with a common HTML format that could work with *.properties files right now and with the l20n format when it’s ready. Please ping me if I can help in any way.
Cheers!
Zbigniew Braniecki
I apologies for entering discussion so late. We've been tracking this
thread and working very hard on our proposal which is based on our
generic approach to XML client side localization based on L20n
framework.
Now that we believe we have the whole API wrapped up I'd like to
suggest it for Jetpack HTML localization.
The major difference from the proposals that you have been discussing
here so far is that ours, in the broad sense operates on the XML DOM
structure rather than templating the text representation of it.
This is a major change and I'd like to elaborate a bit on the
reasoning behind it.
All proposals listed here so far reuse the concept of treating HTML
code as a plain text and templating it using some more or less complex
templating language.
Such approach dominates the server-side HTML UI localization, and it
works really well there, but we believe that using on the client side
will significantly increase the complexity of the cross-technology
operations and introduce a whole new set of potential break points
between JS, CSS, DOM and localization stack.
We believe that from the point where HTML is a DOM stack we should
avoid thinking about it in terms of source code and instead localize
DOM nodes directly.
That's the direction of L20n XML API that binds the L20n framework
into XML/HTML5.
Staś and I wrote a short etherpad document on how L20n approaches
HTML5 localization[1] and we also created a set of examples/tests[2]
that use our clientside l20n and l20n-xml libraries[3] to localize
HTML5 documents.
1) https://etherpad.mozilla.org/html-page-localization
2) http://zbraniecki.github.com/l20n/
3) https://github.com/zbraniecki/l20n
We're working on improving the descriptions on the examples to make it
easier to understand what's going on there, but I hope that looking
into HTML sources will be enough to grasp the concept.
Please, let me know if you have any questions and feedback :)
Cheers,
g.
Ben Bucksch
> operates on the XML DOM structure rather than templating the text representation of it.
+1
Myk Melez
> 1. I am also concerned with a conflict with underscore or any other
> libraries. I think we should either expose either l10n module (then we
> might run into conflict with requirejs cc jrburke to see check if such
> conflicts could be easily solved) or namespace injected functionality
> similar to how js libs do today. addon._ or jetpack._
> Alternatively we could use combination of both `var _ =
> addon.require('l10n').get`
> 2. I do not particularly mind `${key}` approach, but I think it may
> have same / bigger conflict implications as mustaches `{{key}}`, since
> ES.next uses them for quasi
> literals: http://wiki.ecmascript.org/doku.php?id=harmony:quasis
It isn't clear to me that quasi-literals will show up in the same places
these keys will, but in any case the delimiter can be anything, so we
could surely find a non-conflicting one.
> Also, I do like slightly modified third alternative in the proposal:
...
> Elements that have `data-l10n` attribute set will have their innerHTML
> replaced with localized version and existing innerHTML will be used as
> keys (Note support for the nesting). In the future data-l10n attribute
> value may be used to provide additional information for localization
> similar to l10n.get. This solution avoids all the conflicts with
> existing / future js libs and feels more htmly to me.
An approach like this runs into the problem that it isn't possible to
trigger localization at parse time, so it must instead do so after
parsing is completed, as the l20n examples do, perhaps hiding content
until localization is complete to avoid a flicker effect as
non-localized content becomes localized.
The template-like approach also seems more webby to me, even if it is
less HTMLy. ;-)
Nevertheless, the l20n folks do raise valid concerns about the ease of
performing localization with a template-like approach. It isn't clear to
me that those concerns are unresolvable, but I'm certainly open to an
attribute-based approach. Note that such an approach should allow
localization of attribute values in addition to node descendants, i.e.:
<a title="localize me too">localize me</a>
In any case, Alex has expressed an interest in driving this proposal
forward, so I'll let him decide how to evolve it given the latest round
of feedback!
-myk
Zbigniew Braniecki
> An approach like this runs into the problem that it isn't possible to
> trigger localization at parse time, so it must instead do so after
> parsing is completed, as the l20n examples do, perhaps hiding content
> until localization is complete to avoid a flicker effect as
> non-localized content becomes localized.
block the parser, so the patches against HTML5 parser collect
localizable nodes and asynchronously load resources.
> The template-like approach also seems more webby to me, even if it is
> less HTMLy. ;-)
front-end devs who worked with server-side templates.
But it's definitely less HTMLy :)
> Note that such an approach should allow
> localization of attribute values in addition to node descendants, i.e.:
>
> <a title="localize me too">localize me</a>
>
<foo "localize me"
title: "localize me too">
done :)
Cheers,
g.
ZER0
>> - References within references. It is a great idea and it should be
>> part of the common code between content and addon main l10n modules!
> Yes, good idea!
I thought was a good idea too when I did my localization code in my
previous jobs. However, it was proved that it was good just because I
looked at it with developer's eyes. Working side by side with pure
translator - no any kind of developers skill � proved that for them was
hard to follow nested references. Also, nested reference already
included some assumption about how the sentence should be translated, we
implicit assume it will contains that specific "term". Maybe that could
be true in latin-like languages, but not in all of them. In some cases,
it was just better re-phrase.
So, I was thinking that having a "nested reference" to the time unit in
was good, but it was actually better duplicated the terms.
At the end I had to disable that feature.
So, I'm not sure who we're going to target � developers? Translators? �
neither which languages, but I will be very careful to add and push this
option. It could be easily end up in a mess for translators.
Of course, all I said is limited to my personal experience.
Ben Bucksch
I guess it depends on how far you take it. If you get all excited about
it and try to use it in your logic, it produces the problems you described.
However, I found the ability to nest references invaluable for branding.
Often, the brand name appears in the string. In earlier Mozilla times,
DTDs didn't support nesting, which meant that these strings with a brand
had to use .properties and be manually set from a script. Extremely
painful. Now that I can just say "Thanks for using &brandname;", this is
no problem anymore and I have less fights with the Product Manager. ;-)
Ben
ZER0
>>>> - References within references. It is a great idea and it should be
>>>> part of the common code between content and addon main l10n modules!
>>> Yes, good idea!
>> I thought was a good idea too when I did my localization code in my
>> previous jobs. However, it was proved that it was good just because I
>> looked at it with developer's eyes. Working side by side with pure
>> translator - no any kind of developers skill � proved that for them was
>> hard to follow nested references. Also, nested reference already
>> included some assumption about how the sentence should be translated, we
>> implicit assume it will contains that specific "term". Maybe that could
>> be true in latin-like languages, but not in all of them. In some cases,
>> it was just better re-phrase.
>> So, I was thinking that having a "nested reference" to the time unit in
>> was good, but it was actually better duplicated the terms.
>>
>> At the end I had to disable that feature.
>> So, I'm not sure who we're going to target � developers? Translators? �
>> neither which languages, but I will be very careful to add and push this
>> option. It could be easily end up in a mess for translators.
>>
>> Of course, all I said is limited to my personal experience.
>
> I guess it depends on how far you take it. If you get all excited about
> it and try to use it in your logic, it produces the problems you described.
>
> However, I found the ability to nest references invaluable for branding.
Branding was probably the only case where I didn't have the problem I
described. It's pretty unique. But in most of the case the translator
didn't have the visibility of the whole file as you have as developer.
Usually the sentences were imported is some Translator tool (via DITA
files) so what they saw was just the sentence they're going to
translate. Having cross reference inside makes the things messier for
them, because they can't see what &open.label; or &time.unit; really
means, and they also could have different meaning depends by the context
or language.
As developer I agree that nesting references looks great and less
painful. But from a translators point of view could be different. Of
course I don't say that we shouldn't have this feature: I just wanted to
share my personal experience about some dangers that could be bring if
abused, especially if we're targeting translators (and translator tools)
as well, and not just developers.
Ben Bucksch
> Branding was probably the only case where I didn't have the problem I
> described. It's pretty unique.
Branding was the only case (that I remember) where I used nested refs,
yes. I do have several brand labels, though, not just one, e.g.
"Mozilla", "Mozilla Seamonkey", "Mail and news" (long), "Mailnews"
(short), "Composer" etc, all in one app.
Almost all other cases where you construct sentences might be useful for
a programmer, but are too dangerous from a linguistic standpoint,
because the sentence or words (common or weird forms of conjugation)
change in some languages. That is why translators tell us to always use
whole sentences. It's not because they don't understand (well, not
only), but because the languages are so widely different.
ZER0
> Almost all other cases where you construct sentences might be useful for
> a programmer, but are too dangerous from a linguistic standpoint,
> because the sentence or words (common or weird forms of conjugation)
> change in some languages. That is why translators tell us to always use
> whole sentences. It's not because they don't understand (well, not
> only), but because the languages are so widely different.
Exactly, as I described in my first post. I added also the limitation of
the tool / context because I remember it during the reply. :)
Irakli Gozalishvili
parsing is completed:
Alexandre poirot
I finally jumped into this feature. And I come with some concrete experience about how we could implement this.
== network layer implementation ==
I first tried to hack around protocols and channels, in order to be able to operate in the network layer.
But I faced bug 736046, that prevents us for implement this feature today. A patch is submitted and waiting for review,
but at the very least, it will slow down when we are going to be able to use this new feature. (i.e. not until FF 14)
We might be able to workaround it by writting localized HTML files on disk, but we will have to do it synchronously
on the main thread, or, ship localized files in the xpi. Both options are quite bad. The first one may introduce some hangs in FF UI
and the second one will increase XPI size and complexify locales updates.
== DOM/dynamic/gaia approach ==
At the end, I'm not really confident that hacking in network layer is a good idea.
It may introduce some slowdown. Reading a file from JS will most likely be slower than letting C++ HTML parser (or whatever c++ component) read it by itself.
As template base solution depends on using such practice, I started looking at a DOM based solution.
Especially the one already implemented in Gaia.
They basically wait for DOMContentLoaded event to fire and do a querySelector matching all nodes with a 'data-l10n-id' attribute.
Then textContent of these nodes is replaced with translated strings.
Here is an example:
# index.html
... <span data-l10n-id="foo">Hello</span> ...
# fr-FR.properties
foo = Bonjour
And I was really surprised how simple it is to implement!
(https://github.com/ochameau/addon-sdk/blob/c9549010b44ee0da122b89fbe51333833d3dd1c0/packages/api-utils/lib/l10n/html.js)
== Blink effect ==
Then, you may face some blinking content switching from original text to translated one,
but it seems that in most cases you won't ever see such effect.
I gave a try on Collusion addon, and except if you keep the F5 key down, you will most likely never see strings being updated!
Here is two versions of Collusion addon, with same localization algorithm than Gaia, but with fake values.
All strings will be translated to "Translated value".
1/ This first version use a CSS trick in order to hide the document until we finished the translation
http://dl.free.fr/mvdUBrcjc
2/ A alternative version without such trick.
http://dl.free.fr/qbtj2GuMw
You will quickly see that it is quite hard to see strings being updated.
And even if we want to be sure that it never happen, we can hide the document until it is translated.
So if we are confortable with this approach, I'd like to update wiki page, open a related bug
and discuss about particular choices we can make around this approach.
The good points are:
- I think it will end up being more efficient,
- We do not hit any platform issue that would prevent us from using this feature in current firefox releases,
- Closer to l20n, gaia and the DOM.
++
Alex
Dave Townsend
Alexandre poirot
You can find an overall description on the wiki:
https://github.com/mozilla/addon-sdk/wiki/HTML-Page-Localization
And look at the possible implementation here:
https://github.com/mozilla/addon-sdk/pull/387
Some alternatives/possible-improvements are listed on the wiki.
Any feedback is welcomed!
2012/3/21 Dave Townsend <dtow...@oxymoronical.com>:
> --
> You received this message because you are subscribed to the Google Groups
> "mozilla-labs-jetpack" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/mozilla-labs-jetpack/-/Jc-YQjf8tiQJ.
Alexandre poirot
Here is how you would use this feature to localize its HTML document displayed in a tab:
https://github.com/ochameau/collusion/compare/master...html-localization
You just have to put `data-l10n-id` attribute everywhere where there is a string to translate,
and then create a new .properties file for each new supported language.
It ends up working really well. But it highlights the need of supporting inner HTML. Although I'm convinced that the current feature set would already be super-helpfull. So that I'm suggesting to get this first feature set landed ASAP and open a followup discussion, bug and pull request in order to implement inner HTML feature.
By "inner HTML", I mean supporting nicely HTML fragment like this one:
<p>When you visit <a class="domain"></a>, it informs the following websites about you.</p>Currently you can only inject text content from properties files, so that you will have to split this html in pieces into:
<p><span data-l10n-id="descRefereeStart">When you visit</span> <a class="domain"></a>, <span data-l10n-id="descRefereeEnd">it informs the following websites about you.</span></p>From localization perspective it is awfull and may disallow translating in some languages or only with weird sentences.
descRefereeStart = When you visit
descRefereeEnd = it informs the following websites about you.
We would prefer doing something similar to what stas and gandalf suggest:
<p data-l10n-id="descRefereeStart">When you visit <a class="domain"></a>, it informs the following websites about you.</p>I'd like to synchronize this work with Gaia and l20n projects in order to try ending up with a similar feature/implementation.
descReferee = When you visit <a></a>, it informs the following websites about you.
Mingyi Liu
It might seem dumb, but is it possible to simply replace any string enclosed by $%% and %%$ (or some other unique paired strings)? At least it's consistent and easy to use.
Mingyi