l10n ease of entry and build process
Axel Hecht
one of the things that pops up every now and then is how hard it is to
enter localizations of Mozilla apps.
That in part is blamed (rightfully, IMHO) on the fact that we require a
full localization to begin with.
I was thinking about revamping the l10n build process to get rid of that
by doing something like:
- run compare-locales first
- for each green/orange locale (only obsolete strings):
-- do a regular repackaging
-- upload to tinderbox ftp (and nightly)
- for each red locale:
-- create a temporary localization from en-US, merging in existing
strings from the localization
-- upload to a non-complete-l10n dir in the ftp tinderbox dir
That had the benefit that localizers wouldn't start to insert en-US
strings for parts they're not done with yet and can simply leave those
out and still get builds for testing.
Comments?
Axel
Benjamin Smedberg
> - run compare-locales first
> - for each green/orange locale (only obsolete strings):
> -- do a regular repackaging
> -- upload to tinderbox ftp (and nightly)
> - for each red locale:
> -- create a temporary localization from en-US, merging in existing
> strings from the localization
> -- upload to a non-complete-l10n dir in the ftp tinderbox dir
>
> That had the benefit that localizers wouldn't start to insert en-US
> strings for parts they're not done with yet and can simply leave those
> out and still get builds for testing.
You could probably do this in one step: instead of creating a "temporary
locale", teach make-jars.pl enough logic to merge en-US at build time if
necessary.
It might be good to separate out a lot of the logic of compare-locales.pl
into a perl module so that we can reuse the same code when doing compares,
build-time fixup, and in your webtool.
--BDS
Axel Hecht
Well, it's a python module. For various reasons, mostly, because perl
data structures are too obfuscated for my taste. My brain just dies
after the third ${} or not. I just dropped perl for that.
If we used perl, we'd had to boost the requirements to 5.8 to handle
unicode. (Note to self, unicode escapes in perl look more evil than in
.properties, even.)
As for the single pass, I'm not so sure that that is a smooth way to go.
It may speed up the build, but it seems to be pretty complex. The flow
of logic seems to fork, too, as you'd need to signal 'fell back to
english' on the one hand, signal plain errors, and warning output. Our
build process is usually pretty brittle when it comes down to that.
make-jars.pl already does a whole lot of stuff, I'm not sure that
shoveling l10n comparisons and the like somewhere in the middle of it is
going to make things nice to maintain.
Axel
Benjamin Smedberg
> Well, it's a python module. For various reasons, mostly, because perl
> data structures are too obfuscated for my taste. My brain just dies
> after the third ${} or not. I just dropped perl for that.
> If we used perl, we'd had to boost the requirements to 5.8 to handle
> unicode. (Note to self, unicode escapes in perl look more evil than in
> .properties, even.)
Since this process is not going to be on by default, I don't mind adding
either a python or a perl5.8 dependency for this process. preed, please
chime in if you prefer one or the other, in general.
> As for the single pass, I'm not so sure that that is a smooth way to go.
> It may speed up the build, but it seems to be pretty complex. The flow
> of logic seems to fork, too, as you'd need to signal 'fell back to
> english' on the one hand, signal plain errors, and warning output. Our
> build process is usually pretty brittle when it comes down to that.
> make-jars.pl already does a whole lot of stuff, I'm not sure that
> shoveling l10n comparisons and the like somewhere in the middle of it is
> going to make things nice to maintain.
I'm very concerned about adding another build process on top of the existing
one: you've got to tell the build machinery where the "merged" langpack is
(and you don't want it to be in the default location, since that would
overwrite files that are potentially managed by CVS).
Signaling "fell back to English" in the build logs is not hard. make-jars.pl
is indeed a long file, but it has made great strides in readability and
maintainability in the last cycle.
--BDS
Axel Hecht
> Axel Hecht wrote:
>
>> Well, it's a python module. For various reasons, mostly, because perl
>> data structures are too obfuscated for my taste. My brain just dies
>> after the third ${} or not. I just dropped perl for that.
>> If we used perl, we'd had to boost the requirements to 5.8 to handle
>> unicode. (Note to self, unicode escapes in perl look more evil than in
>> .properties, even.)
>
> Since this process is not going to be on by default, I don't mind adding
> either a python or a perl5.8 dependency for this process. preed, please
> chime in if you prefer one or the other, in general.
>
>> As for the single pass, I'm not so sure that that is a smooth way to
>> go. It may speed up the build, but it seems to be pretty complex. The
>> flow of logic seems to fork, too, as you'd need to signal 'fell back
>> to english' on the one hand, signal plain errors, and warning output.
>> Our build process is usually pretty brittle when it comes down to that.
>> make-jars.pl already does a whole lot of stuff, I'm not sure that
>> shoveling l10n comparisons and the like somewhere in the middle of it
>> is going to make things nice to maintain.
>
> I'm very concerned about adding another build process on top of the
> existing one: you've got to tell the build machinery where the "merged"
> langpack is (and you don't want it to be in the default location, since
> that would overwrite files that are potentially managed by CVS).
My current thinking was to copy things over to
l10n/ab-[x|CD]-(mac)?test
I'd consider the modified locale code to actually be a feature, you can
see in the user agent whether someone would report a real l10n bug or
just a known problem.
But this is really just my two-minute guess at this particular problem.
> Signaling "fell back to English" in the build logs is not hard.
> make-jars.pl is indeed a long file, but it has made great strides in
> readability and maintainability in the last cycle.
>
> --BDS
Axel
J. Paul Reed
> Since this process is not going to be on by default, I don't mind adding
> either a python or a perl5.8 dependency for this process. preed, please
> chime in if you prefer one or the other, in general.
We should just bite the bullet and add both as a requirement.
We should attempt to include any other libraries that we might need (I
found out, for instance, that the Macs don't install libwww by default;
I'm not as up-to-date on what is useful in the Python space, but which
we don't include).
If you have details on versions of various packages, that'd be a start.
Later,
preed
--
J. Paul Reed
Build/Release Engineer - The Mozilla Corporation
smtp://pr...@mozilla.com
irc://irc.mozilla.org/preed
pots://650.903.0800/x256
Damjan Georgievski
Great. This will be a big boost to testing l10n before it's 100% finished.
btw +1 on the python choice :)
--
damjan