Quality of localized builds and process requirements
Axel Hecht
I guess I'll start a new thread on the topic of where I want localized
builds to be.
The guiding light should be the user experience, and, as we care about
the gaining new users, the public image of that.
Now, how does the public image of user experience get built? Hard
process, let me model it somewhat. Each of the following steps may be
wrong, utterly, but I don't think so.
I would personally put a significant amount of blame on bloggers on this.
Let's just for the sake of the argument assume that the user experience
of the en-US firefox is great. We know it's not bugfree, but still.
So we'd focus on bloggers in a native language, which are likely going
to have heard rave comments about Firefox elsewhere. If the localization
is great, they join the club.
If the translation is bad, they won't, and they likely are going to say so.
Now, to the next step in modeling, I would think that it takes a certain
constant amount of bloggers to turn the image of UE down, that is, the
numbers of negative "reviews" needed is probably largely independent of
the size of the locale. I guess there would be mesh theories to back
that up (think about that 7 folks to know everyone on the world, broken
as that paper is).
So, yet another assumption, bloggers are power users. They're going to
look at more stuff, they're going to be more picky, they're going to see
more edge cases due to their surfing habits.
In addition to this, the amount of people interested in testing Firefox
is orders of magnitude larger than with any other FOSS project. Thus
gaining that critical mass of bad reviews is way more likely to happen
to Firefox than to, say KDE, or even OpenOffice.org.
Next assumption. Users like these will see the difference between a
hacked translation and a translation that follows good software
engineering processes. They will see things like randomly sprinkled
English strings, dialog buttons that say "Restart later..." instead of
"Update later..." or others. And I doubt they'll say "cool to have this
in my language", I'd rather expect them to say "wow, someone didn't
bother looking at what he did, I wonder which other crap is in there.
Firefox probably sucks."
That is why I emphasize completeness of localizations, why I want
localizers to be active members of the Mozilla community, and why I (and
other powers @mozilla) enforce our proven software engineering processes
on the localization community. This is why I will continue to focus on
that step, enabling more locales to get that step done. Of course
without arbitrarily breaking the entry level.
Axel
João Miguel Neves
> That is why I emphasize completeness of localizations, why I want
> localizers to be active members of the Mozilla community, and why I (and
> other powers @mozilla) enforce our proven software engineering processes
> on the localization community. This is why I will continue to focus on
> that step, enabling more locales to get that step done. Of course
> without arbitrarily breaking the entry level.
>
Completeness is obviously something you want in a localization that is
released. Let me just state this once more: it's not enough. Coherence,
good translations, a good reviewing process are also critical in this. A
QA infrastructure that helps identify problems with the translation, not
just if Firefox functionality is still there.
We've built part of that for pt-PT, but it's still not easy enough to do
it for all the teams (it does require technical skills to set up).
Making the building of language packs easier, reducing the overhead to
correct bugs (possibly separating in time some localized releases from
en-US ones) would help this effort. Creating translation memories,
glossary checking tools, several consistency/correction checks would
also improve the overall quality. Completeness is a great goal, but is
only the first step.
Best regards,
João Miguel Neves
Axel Hecht
> Qui, 2006-11-23 às 17:24 +0100, Axel Hecht escreveu:
>> That is why I emphasize completeness of localizations, why I want
>> localizers to be active members of the Mozilla community, and why I (and
>> other powers @mozilla) enforce our proven software engineering processes
>> on the localization community. This is why I will continue to focus on
>> that step, enabling more locales to get that step done. Of course
>> without arbitrarily breaking the entry level.
>>
> Just to repeat myself, but in the context, I think it's useful.
> Completeness is obviously something you want in a localization that is
> released. Let me just state this once more: it's not enough. Coherence,
> good translations, a good reviewing process are also critical in this. A
> QA infrastructure that helps identify problems with the translation, not
> just if Firefox functionality is still there.
Thanks for filling in some of the verbose part for "proven engineering
processes".
> We've built part of that for pt-PT, but it's still not easy enough to do
> it for all the teams (it does require technical skills to set up).
> Making the building of language packs easier, reducing the overhead to
> correct bugs (possibly separating in time some localized releases from
> en-US ones) would help this effort. Creating translation memories,
> glossary checking tools, several consistency/correction checks would
> also improve the overall quality. Completeness is a great goal, but is
> only the first step.
Let me take a bet. The documentation for those tools is sparse, and
written in Portuguese?
Share share share. What tools do you have?
Axel
Dwayne Bailey
<snip>
> That is why I emphasize completeness of localizations, why I want
> localizers to be active members of the Mozilla community, and why I (and
> other powers @mozilla) enforce our proven software engineering processes
> on the localization community. This is why I will continue to focus on
> that step, enabling more locales to get that step done. Of course
> without arbitrarily breaking the entry level.
OK I'm at a loss to understand how any of this address the issues of
translation you mention. How exactly does good software engineering
process address:
- poor terminology choice, lack of terminology
- poorly skilled localisers
- badly structured source text
I do agree the perception can play an important role and can scuttle
good intentions. But engineering does not address: poor, non-existant
and conflicting terminology. Power users, certainly where I live, are
not the best at terminology - preferring anglicisms when native language
constructs exist.
But onto solutions :) Lets rather talk about good localisation process:
1) Define terminology - we would need a subset of terms and definitions
used by Mozilla products. This should be the starting point and guiding
light for the translators. So hopefully no confusion around 'Restart'
and 'Update'. In our experience we translated Spreadsheet in
OpenOffice.org 3 different ways, all correct, so consistency can be just
as important as correctness. Thus established languages can not ignore
this step.
2) Style guidelines - starting from a simple set of rules related to the
translation, from the style of translation to rules about whether the
language will follow the en-US title case convention. Small is best.
3) Focus effort - knowing which parts are critical to the visual
experience so that more effort, better review can be focused on that is
more important then trying to review 100%. And how do we know that
these have even been reviewed or where they just translated?
4) Examine conflicts - conflicts in translation: same target word used
of 2 English words or same English word translated in 2 ways are often
an indication of poor translation. Finding these, correcting or
validating them is an important step in improving consistency.
5) Reuse existing work - translation memory is an important part of
ensuring consistency across products and multiple translators. Without
it we are continually retranslating. And there is the risk of products
not aligning or parts of products not aligning in terminology use.
6) Tracking review - it would be great if we had some track that
critical sections have been reviewed. Yes I guess we could say lets do
the 100% of the product but I think we're then ignoring the fact that
only about 40% is probably critical for the UI experience, the rest
being 'important'. So ensuring that the critical part are actually
reviewed is more important then the 100% target. It would also be a way
to track untranslated items, in that items that remain untranslated and
have been reviewed we then know are meant to be untranslated.
7) Live review - often the hardest to perform. But if we can get a
situation where we can quickly go to the relevant screen, that would be
amazing. Any chance we can do away with manual screen resizing, as this
is I guess a large source of bugs.
I've tried to avoid defining how any of these stages should be addressed
but I hope I've highlighted the bits that I think are critical to
improving the process.
--
Dwayne Bailey
Translate.org.za
+27-12-460-1095 (w)
+27-83-443-7114 (cell)
Cédric Corazza
> OK I'm at a loss to understand how any of this address the issues of
> translation you mention. How exactly does good software engineering
> process address:
>
> - poor terminology choice, lack of terminology
> - poorly skilled localisers
> - badly structured source text
Dwayne, am I wrong or are you insulting for the second time Mozilla l10n
teams?
> But onto solutions :) Lets rather talk about good localisation process:
>
> 1) Define terminology - we would need a subset of terms and definitions
> used by Mozilla products. This should be the starting point and guiding
> light for the translators. So hopefully no confusion around 'Restart'
> and 'Update'. In our experience we translated Spreadsheet in
> OpenOffice.org 3 different ways, all correct, so consistency can be just
> as important as correctness. Thus established languages can not ignore
> this step.
>
> 2) Style guidelines - starting from a simple set of rules related to the
> translation, from the style of translation to rules about whether the
> language will follow the en-US title case convention. Small is best.
I'm talking about the French locale here (but I guess that most other
locales do so), we have such "tools" and glossary.
Though the consistency you're talking about should be applied most of
the time, some terms *do* need to be translated differently depending
where there are used. This is related to the culture of each people (by
people I mean the French, the Pole, etc.). So though this consistency is
useful, it can't be blindly followed.
> 3) Focus effort - knowing which parts are critical to the visual
> experience so that more effort, better review can be focused on that is
> more important then trying to review 100%. And how do we know that
> these have even been reviewed or where they just translated?
Agreed. The current l10n Litmus tests are almost useless. We definitely
need some set of essentials strings to be mandatory reviewed : menus,
buttons, common error or information messages, etc.
> 4) Examine conflicts - conflicts in translation: same target word used
> of 2 English words or same English word translated in 2 ways are often
> an indication of poor translation. Finding these, correcting or
> validating them is an important step in improving consistency.
I partly disagree on that (cf. 2) section comment
> 5) Reuse existing work - translation memory is an important part of
> ensuring consistency across products and multiple translators. Without
> it we are continually retranslating. And there is the risk of products
> not aligning or parts of products not aligning in terminology use.
>
> 6) Tracking review - it would be great if we had some track that
> critical sections have been reviewed. Yes I guess we could say lets do
> the 100% of the product but I think we're then ignoring the fact that
> only about 40% is probably critical for the UI experience, the rest
> being 'important'. So ensuring that the critical part are actually
> reviewed is more important then the 100% target. It would also be a way
> to track untranslated items, in that items that remain untranslated and
> have been reviewed we then know are meant to be untranslated.
This meets 3) section. The most seen (common) strings do need to be
tracked and reviewed. Though, I think that the 100% target must be
reached. And the current rule about 'release can be done even if the
help section is not translated' might be changed. Let's say the first
time a locale is released, the Help section can be left untranslated,
but it has to be for the next major release; this lets time lots of time
for translators, as this is the painful and long part. Once the initial
translation is done, it is very much less painful to maintain the changes.
> 7) Live review - often the hardest to perform. But if we can get a
> situation where we can quickly go to the relevant screen, that would be
> amazing. Any chance we can do away with manual screen resizing, as this
> is I guess a large source of bugs.
I proposed that in the other thread (some html sample pages which can
make appear the strings we are looking for in context). But it's a big
job to make I guess and we need to define the '40%' strings important
before.
> I've tried to avoid defining how any of these stages should be addressed
> but I hope I've highlighted the bits that I think are critical to
> improving the process.
Regards
Cédric
Ricardo Palomares Martinez
>
> 4) Examine conflicts - conflicts in translation: same target word used
> of 2 English words or same English word translated in 2 ways are often
> an indication of poor translation. Finding these, correcting or
> validating them is an important step in improving consistency.
This is funny, because I just started yesterday a discussion in es-ES
mailing list about which spanish words use to translate Delete and
Remove, giving some examples of en-US strings containing it, and a
consensus has quickly been reached on that we should really use just
one translation for both words, because examples show that there is no
really a semantic difference between both words that we need to
maintain (or can in spanish).
>
> 5) Reuse existing work - translation memory is an important part of
> ensuring consistency across products and multiple translators. Without
> it we are continually retranslating. And there is the risk of products
> not aligning or parts of products not aligning in terminology use.
Well, maybe because es-ES team is comprised of people that started
using Mozilla when it was in fact Netscape, there are a bunch of terms
that we don't really want to "normalize" or align across projects.
These terms are "brand identity" for Netscape/Mozilla vs. MSIE, for
instance (and that's not exclusive of es-ES: think, for instance, of
"Bookmarks" vs. "Favourites"). Aligning terminology between product
translations just as a rule would be bad.
Aligning terminology inside the same product (or product family, like
Mozilla) can be good usually, but still there are exceptions.
"Preview" in en-US for both viewing a web page printout before
actually sending it to the printer and for playing a sound can be
fine, keeping the same translation to spanish would be ridiculous.
And, BTW, while I know that it can be done, I haven't been able to see
in gettext manual how PO deals with this.
So, translation memory is, overall, a nice thing to have, but blindly
follow it can lead to bad translations, too.
Ricardo.
--
If it's true that we are here to help others,
then what exactly are the OTHERS here for?
pascal
>
> 4) Examine conflicts - conflicts in translation: same target word used
> of 2 English words or same English word translated in 2 ways are often
> an indication of poor translation. Finding these, correcting or
> validating them is an important step in improving consistency.
I don't agree with this one, Ricardo mentionned the example of
Delete/Remove (the French locale also uses a single word for that) but
there are many examples of words that have to be translated in several
different ways in the target languages.
For example "forward" in seamonkey is used both for forwarding an email
and going back one page, French and probably most languages use
different words for these 2 different actions.
There are also neologisms without established equivalents in your
language that need paraphrasing. "antiphishing" is an example in French
with different translations depending on where you see the word because
there is no consensus yet on a French word understood by everybody.
There is also the case of verb/nouns that are the same in English like
'bookmark' and also the case of words that probably have an obvious
meaning to an English speaker whatever the context but need rewording in
French. For example "location" is translated differently in French for
the strings "save location" and the "location" text field in a bookmark
properties panel. Using the same translation for both would be either
poor French or unclear.
To illustrate all that, an example. The sentence "where are you?" could
be translated in French as "où es-tu ?", "où tu es ?", "où est-ce que tu
es ?", "tu es où ?", "où êtes vous?"... all these translations are
correct French and would depend on the context where the original string
is located.
Pascal
Axel Hecht
This sound like total overengineering to me.
I really see no reason why Mozilla should decide with which files a
localizer should start, for example.
The industry experience is (and this is not just within Mozilla), that
consistency is something that volunteer localizers struggle with, but
that that's not a blocker. You file bugs on that, get them fixed, get
them verified (this is the important process, btw.). That's totally
different from commercial translation vendors, which actually have real
problems. Being not-consistent is just disturbing, being wrong is wrong.
And the latter happens with dedicated volunteers almost not at all,
whereas it seems to be a reoccurring pattern at places where you have to
go to commercial vendors, may that be for legal or NDA reasons.
As we're doing OpenSource software, it's not that essential to get it
right at once, but to get it done, and then improve it. Which governs
the way we look at localization teams, we want them to continuously
improve, their locale and the user experience on a global level.
The real challenge for the Mozilla side of things is to actually get
more builds out there earlier, so that folks can actually release early,
release often (though I'm only talking about nightlies here).
This does not exclude sharing experiences between teams. I would think
that somebody new to translating Firefox may be suprised to see how
often they should think about translating something consistently (or
not, on purpose). So generating a list of common terms and phrases is
definitely something we should do, and I started investigating here. Our
own Sherman Dickman did something similar a while back, I'll try to get
his mindshare to hook that process up to our sources.
Review is important, but from my experience from looking at how our
teams operate, they're either low on resources, or they're doing it
already. When in the development process that review happens depends
largely on the technical skills. For those teams that don't have the
manpower, I'm cutting some slack. The affected userbase is usually not
that big and still better off. For those individuals, the development
cycle usually takes longer, but that's about as far as I go in compromising.
Re 6-7, I'm admin at litmus now, so if folks propose some tests that
should be run to do some coverage testing, I'm happy to add them to the
system.
Axel
João Miguel Neves
> Let me take a bet. The documentation for those tools is sparse, and
> written in Portuguese?
>
http://gettext-lint.sourceforge.net/ The version that allows for
glossary checks is only on the CVS (and a working glossary builder (from
a translation).
The problem is that it works with PO (these tools were built originally
by the pt-PT KDE team). We then have a bunch of scripts that
automatically checkout/update the code from mozilla's CVS and our local
one, turn them into PO and run the tests.
The kind of results we're getting can be seen in:
http://server.intraneia.com/~mozilla/ff/2.0/rel/
The glossary we use can be found on our wiki (yet another script picks
the data from the wiki and generates the glossary file for testing).
We use this to detect errors and guide our reviews.
All the scripts we use are in those dirs (check the parent directories).
If someone is interested I could package an install script of some kind
(or a zip file with the right directory structure the scripts assume).
João Miguel Neves
> Dwayne Bailey a écrit :
>
> >
> > 4) Examine conflicts - conflicts in translation: same target word used
> > of 2 English words or same English word translated in 2 ways are often
> > an indication of poor translation. Finding these, correcting or
> > validating them is an important step in improving consistency.
>
> I don't agree with this one, Ricardo mentionned the example of
> Delete/Remove (the French locale also uses a single word for that) but
> there are many examples of words that have to be translated in several
> different ways in the target languages.
>
> For example "forward" in seamonkey is used both for forwarding an email
> and going back one page, French and probably most languages use
> different words for these 2 different actions.
>
our glossary at:
http://firefox.ansol.org/user-cgi/moin.cgi/Gloss%c3%a1rio?highlight=%
28gloss%C3%A1rio%29
Yes, those are 6 different translations for "check". And that's because
we haven't a bank check reference in Firefox. Even so, with all these,
checking the translation against the glossary (we use POFileGlossary
over an XML file generated from that wiki page) has allowed us to find
and correct several consistency errors or typos.
Damjan Georgievski
> builds to be.
>
> The guiding light should be the user experience, and, as we care about
> the gaining new users, the public image of that.
>
> Now, how does the public image of user experience get built? Hard
> process, let me model it somewhat. Each of the following steps may be
> wrong, utterly, but I don't think so.
>
> I would personally put a significant amount of blame on bloggers on this.
> So we'd focus on bloggers in a native language, which are likely going
> to have heard rave comments about Firefox elsewhere.
Bloggers (at least in my country) are all power users, and almost all of
them will NEVER use a localized Firefox. It's just that they are so used to
software in english, and they understand the english terminology much
better.
The main target group for a localized Firefox are older users and people
that don't understand english very well. For them it would be much more
beneficial if Firefox could use the mk translation as primary, but fallback
to serbian or croatian or bulgarian as secondary, than any other feature.
> If the localization is great, they join the club.
>
> If the translation is bad, they won't, and they likely are going to say
> so.
This is a problem with Firefox, you need to download a whole new installer
and reinstall just to switch languages (and don't tell that there's an
extension for locale switching - we are talking about first-comers).
BTW
I know you don't want to gear this AGAIN, but gettext solves both of the
above problems very elegantly... Mozilla MUST catchup here.
> So, yet another assumption, bloggers are power users. They're going to
> look at more stuff, they're going to be more picky, they're going to see
> more edge cases due to their surfing habits.
The ideal would be if they could just click and modify the localization
right there in the interface, just like a wiki page... I don't know if
there's been a system for that, but I'd say thats the holy grail of
localization.
> Next assumption. Users like these will see the difference between a
> hacked translation and a translation that follows good software
> engineering processes. They will see things like randomly sprinkled
> English strings, dialog buttons that say "Restart later..." instead of
> "Update later..." or others. And I doubt they'll say "cool to have this
> in my language", I'd rather expect them to say "wow, someone didn't
> bother looking at what he did, I wonder which other crap is in there.
> Firefox probably sucks."
>
> That is why I emphasize completeness of localizations, why I want
> localizers to be active members of the Mozilla community, and why I (and
> other powers @mozilla) enforce our proven software engineering processes
> on the localization community. This is why I will continue to focus on
> that step, enabling more locales to get that step done. Of course
> without arbitrarily breaking the entry level.
OTOH
Some of those bithchers would say: "this is awful. how can they do this with
my language. I must do something to correct this" and then they will try to
find how to contribute to the localization and that's when they'll be sure
that "...Firefox probably sucks". We will not only loose users but
contributors also.
And I must say, for all projects that depend on community, the contributors
come first, the users second... since if it is not for the contributors,
there wan't be software for the users anyway...
But since Mozilla is less and less a community project, maybe this doesn't
apply.
--
damjan
Axel Hecht
Not.
>> So, yet another assumption, bloggers are power users. They're going to
>> look at more stuff, they're going to be more picky, they're going to see
>> more edge cases due to their surfing habits.
>
> The ideal would be if they could just click and modify the localization
> right there in the interface, just like a wiki page... I don't know if
> there's been a system for that, but I'd say thats the holy grail of
> localization.
Only if you don't care about consistency. Just think about the recent
thing with wikipedia and that citizen-thing-fork.
Not that I see a way to implement this.
>> Next assumption. Users like these will see the difference between a
>> hacked translation and a translation that follows good software
>> engineering processes. They will see things like randomly sprinkled
>> English strings, dialog buttons that say "Restart later..." instead of
>> "Update later..." or others. And I doubt they'll say "cool to have this
>> in my language", I'd rather expect them to say "wow, someone didn't
>> bother looking at what he did, I wonder which other crap is in there.
>> Firefox probably sucks."
>>
>> That is why I emphasize completeness of localizations, why I want
>> localizers to be active members of the Mozilla community, and why I (and
>> other powers @mozilla) enforce our proven software engineering processes
>> on the localization community. This is why I will continue to focus on
>> that step, enabling more locales to get that step done. Of course
>> without arbitrarily breaking the entry level.
>
> OTOH
> Some of those bithchers would say: "this is awful. how can they do this with
> my language. I must do something to correct this" and then they will try to
> find how to contribute to the localization and that's when they'll be sure
> that "...Firefox probably sucks". We will not only loose users but
> contributors also.
This is what nightlies are for.
> And I must say, for all projects that depend on community, the contributors
> come first, the users second... since if it is not for the contributors,
> there wan't be software for the users anyway...
> But since Mozilla is less and less a community project, maybe this doesn't
> apply.
You are wrong. Firefox is very much a community project. And no, it's
not contributors first, users second. If you do that, only your
contributors will use it. Users first, always. That decision triggered
the fork of MB (to be phoenix ..) from the suite back then.
"Users first" is base of the success of Firefox and stands. Rock solid.
Axel
Peter Van der Beken
> Dwayne Bailey a écrit :
>> 4) Examine conflicts - conflicts in translation: same target word used
>> of 2 English words or same English word translated in 2 ways are often
>> an indication of poor translation. Finding these, correcting or
>> validating them is an important step in improving consistency.
>
> I don't agree with this one
Actually, you agree completely I think, "correcting or validating"
allows for multiple translations of the same string where necessary.
Peter
Benoit
I think the part most of us were strongly disagreeing was not the idea
of seeking and validating conflicts. What some of us were upset was the
part about the "indication of poor translation".
In the case of the French language, where every word should very
carefully chosen to blend in the context, the reverse could actually be
closer to the truth. If you end up with the same translation for
different uses of a word, you probably have a problem and the result may
look very unprofessional. (FWIW, that was the case in the Office 2007
and IE7 French betas. (un)fortunately, those were fixed in time for the
release :))