Waiting for the 'localized' event before showing content with text

[Margaret Leibovic]

Hi all,

I'd like to surface an issue about l10n that's come up in two recent bugs
[1][2]. The issue is that we're showing content in some apps before
receiving the 'localized' event, meaning that we're first showing the
English strings that are hard-coded into the app's HTML before
translateFragment() is called in l10n.js. In some cases, such as the
settings app, it's really noticeable that we're showing English strings
before showing strings for the correct locale.

I'm worried that developers and testers working in English haven't noticed
issues related to this because the "translated" text for en-US is generally
the same as the text in the .en-US.properties files. However, the vast
majority of our users are not going to have their phones set to en-US, so
we need to fix this. I'd really like to encourage non-English speakers to
test in their locale if it's available and pay attention to issues like
this.

Unfortunately, the other half of this problem is that l10n.js is really
slow to fire the 'localized' event right now [3]. This hasn't been raised
as a blocker because it's not noticeable for apps that don't wait for the
event, but if we decide that's the Right thing to do, we'll need to find a
solution here as well.

Margaret

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=815852
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=816151
[3] https://bugzilla.mozilla.org/show_bug.cgi?id=809600

[Donovan Preston]

Maybe it would be good to replace the original content with lorem ipsum
or something so that even english users can see the effect of the delay?

> _______________________________________________
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia

[will kahn-greene]

The email app does FunnY StringS in the html, so if the l10n stuff
kicked in slowly or was missing strings, you'd probably notice.

[Margaret Leibovic]

Stas actually filed a bug [1] about removing all of the hard-coded strings
from the HTML, but no decision was made about that.

No matter what the solution, I'm strongly in favor of some common
practice/guidelines across all Gaia apps, since it's clear some apps do
things to deal with this but others don't.

Margaret

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=812993

>>> [1] https://bugzilla.mozilla.org/**show_bug.cgi?id=815852<https://bugzilla.mozilla.org/show_bug.cgi?id=815852>
>>> [2] https://bugzilla.mozilla.org/**show_bug.cgi?id=816151<https://bugzilla.mozilla.org/show_bug.cgi?id=816151>
>>> [3] https://bugzilla.mozilla.org/**show_bug.cgi?id=809600<https://bugzilla.mozilla.org/show_bug.cgi?id=809600>
>>> ______________________________**_________________
>>> dev-gaia mailing list
>>> dev-...@lists.mozilla.org
>>> https://lists.mozilla.org/**listinfo/dev-gaia<https://lists.mozilla.org/listinfo/dev-gaia>
>>>
>>
>>
>> ______________________________**_________________
>> dev-gaia mailing list
>> dev-...@lists.mozilla.org
>> https://lists.mozilla.org/**listinfo/dev-gaia<https://lists.mozilla.org/listinfo/dev-gaia>
>>
> ______________________________**_________________
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/**listinfo/dev-gaia<https://lists.mozilla.org/listinfo/dev-gaia>
>

[Zbigniew Braniecki]

----- Original Message -----
> Maybe it would be good to replace the original content with lorem
> ipsum
> or something so that even english users can see the effect of the
> delay?

Hi guys,

I just got back to working on performance metrics that will mostly focus on localization.

In a few days I'll provide you numbers wrt. the bugs.

I didn't comment cause I don't have hard data yet, but I'm working on performance metrics for gaia apps now.

Cheers,
g.

[Vivien]

On 30/11/2012 01:15, Margaret Leibovic wrote:
> Hi all,
>
> I'd like to surface an issue about l10n that's come up in two recent bugs
> [1][2]. The issue is that we're showing content in some apps before
> receiving the 'localized' event, meaning that we're first showing the
> English strings that are hard-coded into the app's HTML before
> translateFragment() is called in l10n.js. In some cases, such as the
> settings app, it's really noticeable that we're showing English strings
> before showing strings for the correct locale.

Hey,

What about the plan to precompile html files? I think this is really the
best option, there is no good reason to create a penalty for our
brazilian (or others) users because the main code is in English. Is it
too complicated?

>
> I'm worried that developers and testers working in English haven't noticed
> issues related to this because the "translated" text for en-US is generally
> the same as the text in the .en-US.properties files. However, the vast
> majority of our users are not going to have their phones set to en-US, so
> we need to fix this. I'd really like to encourage non-English speakers to
> test in their locale if it's available and pay attention to issues like
> this.
>
> Unfortunately, the other half of this problem is that l10n.js is really
> slow to fire the 'localized' event right now [3]. This hasn't been raised
> as a blocker because it's not noticeable for apps that don't wait for the
> event, but if we decide that's the Right thing to do, we'll need to find a
> solution here as well.
>
> Margaret
>

> [1] https://bugzilla.mozilla.org/show_bug.cgi?id=815852
> [2] https://bugzilla.mozilla.org/show_bug.cgi?id=816151
> [3] https://bugzilla.mozilla.org/show_bug.cgi?id=809600
> _______________________________________________

> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia

[Margaret Leibovic]

On Fri, Nov 30, 2012 at 2:19 AM, Vivien <vnic...@mozilla.com> wrote:

> On 30/11/2012 01:15, Margaret Leibovic wrote:
>
>> Hi all,
>>
>> I'd like to surface an issue about l10n that's come up in two recent bugs
>> [1][2]. The issue is that we're showing content in some apps before
>> receiving the 'localized' event, meaning that we're first showing the
>> English strings that are hard-coded into the app's HTML before
>> translateFragment() is called in l10n.js. In some cases, such as the
>> settings app, it's really noticeable that we're showing English strings
>> before showing strings for the correct locale.
>>
>
> Hey,
>
> What about the plan to precompile html files? I think this is really the
> best option, there is no good reason to create a penalty for our brazilian
> (or others) users because the main code is in English. Is it too
> complicated?
>

The short-term plan for precompiling the HTML files would be to just
include the JS data structure necessary to do the translation with the
existing translateFragment object. This would speed things up a lot because
the really slow part right now is doing XHR to get the .ini/.properties
files.

Completely translating the HTML at build time would be a lot trickier right
now because the data-l10n-id system we have is designed to do the
translation with DOM manipulation. Because we don't have a real template
system, I'm worried that doing build-time translation like this would be
error prone.

However, both of these approaches would require bug 811540. In bug 809600,
Axel felt strongly opposed to this change, but I don't know who makes the
final say about whether or not we should go forward with that. I suppose
that's the blocker at the moment.

Margaret

[Axel Hecht]

I can't say that I'm a fan, yes.

I think preprocessing states a pretty utter failure of our architecture,
and I'd prefer to show that our architecture can work.

I'm concerned about the level of magick involved in tweaking the app
protocol, and the amount of support that will cost us going forward once
external apps figured out that that's what they should do.

If we'd inline the 'ini data', I'm less concerned.

If we'd inline all the data for the localizations, I'm not sure we'd win
a lot, but also we don't need to do things like generate and magick an
html per language.

Generating localized html at build time is really scary, and
baby-bathwater stuff, IMHO.

The real challenge to me is accepting that we can't fix reading from a
zip file, and instead need to put all our zip entries into the html file
itself. That sounds like a bug that should have been fixed in another way.

Axel

[Zbigniew Braniecki]

----- Original Message -----
> On Fri, Nov 30, 2012 at 2:19 AM, Vivien < vnic...@mozilla.com >
> wrote:
>
>
>
>
> On 30/11/2012 01:15, Margaret Leibovic wrote:

> However, both of these approaches would require bug 811540. In bug
> 809600, Axel felt strongly opposed to this change, but I don't know
> who makes the final say about whether or not we should go forward
> with that. I suppose that's the blocker at the moment.

I spent some time with Axel today on this and I believe we have an idea on how we may want to solve that.

I got first numbers for l10n costs, and I'll try to get more details early next week and compare them to inlining JS/CSS/L10n resources into HTML document.

btw. the real issue here is the amount of files that we load - 10 for localization of Settings app for the moment - and due to resources constrains this becomes less deterministic since a file load time variance is rather significant (the file load time has a spread from 33ms to 159ms)

Cheers,
g.

[David Flanagan]

On 11/30/12 7:44 PM, Zbigniew Braniecki wrote:
> btw. the real issue here is the amount of files that we load - 10 for localization of Settings app for the moment - and due to resources constrains this becomes less deterministic since a file load time variance is rather significant (the file load time has a spread from 33ms to 159ms)
>
> Cheers,
> g.
>

I wonder how much the slow file load time is affecting the rest of the
app startup process. Do we have plans to combine and minify our js and
css files before we release v1? In addition to its localization files,
the Gallery app (for example) loads 3 css files and 11 scripts. Is
everything slow to load, or is it worse for the XHR requests that
l10n.js does?

Does the app:// protocol have any of the sort of 'only 2 connections at
a time' throttling that we used to apply to http? That is, we're not
artificially limiting the speed of our requests somehow, are we?
Conversely, are we overloading our poor little CPU by throwing too many
requests at it at once and causing it to thrash or something? (I'm just
throwing these questions out there: I don't have any real understanding
of how this under the hood stuff works...)

For localization, keep in mind that JavaScript now has robust binary
data support. We use this to good effect for the word suggestions
dictionaries, and could alter l10n.js to work with binary data if that
would help. It might decrease file size and processing time.

I assume that l10n.js uses querySelector() to find the element with the
matching value of data-l10n-id. Has anyone looked at the performance of
that? Would it make a performance difference if we used id (which is, I
assume, very highly optimized) instead of data-l10n-id? Could we hack
gecko to add an internal optimization for looking up data-l10n-id?

David

[Zbigniew Braniecki]

> I wonder how much the slow file load time is affecting the rest of
> the
> app startup process. Do we have plans to combine and minify our js
> and
> css files before we release v1? In addition to its localization
> files,
> the Gallery app (for example) loads 3 css files and 11 scripts. Is
> everything slow to load, or is it worse for the XHR requests that
> l10n.js does?

No, everything should be similarly slow.

The good news is that we're fairly good with parallelism so loading 10 files does mean 100ms times 10, rather ~200ms.

I'll combine css/js/l10n for an app and measure the difference early next week.

> Does the app:// protocol have any of the sort of 'only 2 connections
> at
> a time' throttling that we used to apply to http? That is, we're not
> artificially limiting the speed of our requests somehow, are we?

I don't know.
The numbers I see from performance.* API and XHR loading times suggest that synchronous loading is faster (40-60ms vs 90-150ms), but it blocks the main thread (obviously).

> Conversely, are we overloading our poor little CPU by throwing too
> many
> requests at it at once and causing it to thrash or something? (I'm
> just
> throwing these questions out there: I don't have any real
> understanding
> of how this under the hood stuff works...)

I'll investigate it more, but so far I don't see any impact of parallel file load on performance. I might have not tested enough files in parallel yet, but up to 10 files the parallelism of async file loading looks good.

> For localization, keep in mind that JavaScript now has robust binary
> data support. We use this to good effect for the word suggestions
> dictionaries, and could alter l10n.js to work with binary data if
> that
> would help. It might decrease file size and processing time.

File size is not an issue. The difference in load time between 18k file and 18byte file is almost zero.
Also, I/O is not an issue, the read perf is fairly good and exceeds what we observe by far.

I'd like to avoid turning to such methods.

> I assume that l10n.js uses querySelector() to find the element with
> the
> matching value of data-l10n-id. Has anyone looked at the performance
> of
> that? Would it make a performance difference if we used id (which
> is, I
> assume, very highly optimized) instead of data-l10n-id? Could we
> hack
> gecko to add an internal optimization for looking up data-l10n-id?

The perf. of this is fairly good. The whole DOM manipulation (search, inject, reflow) takes ~40ms for the whole document.

It feels (guessing here!) that the slowest we have is the time that it takes to access the file. Loading it is fast.

I tested if we close the ZIP file after each load (that's what Taras suggested) but it doesn't seem to be the case.
The high latency in file loading together with high number of files loaded brings high variance to all timings.

I suspect that it also affects pure index.html file load, since it is surprisingly slow as well. (empty template app on Unagi takes over 400ms to load).

So, my current plan is to test how much time can we shave by inlining resources into HTML files and update the bugs about file access latency next week.

Cheers,
g.

[Fabrice Desre]

On 12/01/2012 12:27 AM, Zbigniew Braniecki wrote:

> I suspect that it also affects pure index.html file load, since it is surprisingly slow as well. (empty template app on Unagi takes over 400ms to load).
>

There's much more happening when you launch an app OOP than just
retrieving the index.html. It's very easy to see that just by disabling
OOP in the settings. Also, Jason Duell is working on deep changes to the
app:// protocol implementation in bug 815523 and that will probably have
some consequences on the perfs. That could be good, since he found some
synchronous I/O that he's turning into asynchronous I/O.

Fabrice
--
Fabrice Desré
b2g team
Mozilla Corporation