Error found in 36 locales with our localization tools script.
dynamis
l10n owners:
Check if your locale and fix errors if your locale have.
Axel and Seth:
Please let l10n owners know about their errors if they don't notice this mail.
In short:
We found some errors in these locale files:
ca, de, fr, gl, gu-IN, he, hi-IN, hu, id, ka, kk, kn, ko, lv, mk, ml, mn, mr,
nl, oc, pa-IN, pt-PT, si, sk, sl, sq, sr, sv-SE, ta-LK, te, th, tr, uk, uk,
zh-CN, zh-TW
If your locale is in this list, see:
http://people.mozilla.com/~dynamis/lot/20090427/errorlog.summary.txt
duplicate entity error means:
You have 2 or more entity with same id in the file.
placeholder error means:
If you use 2nd placeholder %2$S but not %1$S, firefox cannot handle the entity
correctly and you should insert dummy placeholder %1$0.S. but some locale just
use only 2nd one or typo like %1%S
syntax error means
Your file have syntax error (probably). Check the detected line and around.
Hope this help your l10n work. ;)
Details:
We japanese l10n team are using scirpt called called Localization Tools (or just
'lot 'in short) from Fx 1.0 days. The script have now syntax check, placeholder
check, word/char check, compare/merge locale files and make langpack features
and so on.
The script was ja/ja-JP-mac specific and required some hacks to use for other
locales. But today I made it possible to use it for other locales without
complex command line options and hacks. It's not yet perfect but almost working
correctly and I'm testing script with allmost every locale files (*) and many
many errors found in your l10n files.
(*) except bn-IN, mk, lv with which the script have some trouble (script's bug)
output log of the script are:
http://people.mozilla.com/~dynamis/lot/20090427/log/
and summary file containing only error messages of all log file:
http://people.mozilla.com/~dynamis/lot/20090427/errorlog.summary.txt
If you want to check your locale by yourselves, you can try:
svn export http://svn.mozilla.l10n.jp/trunk/lot/
cd lot
ant setupl10n
# input your locale and some other options
ant errorcheck
ant compare
With most locales you'll see too many output about accesskey matching unless
your locale's accesskey policy is to sync with en-US (like us). Ignore them and
see only other messages in the errorcheck/compare log file.
You can try merge and generate firefox langpack with merged files
ant merge
ant buildfx
You can see other targets and command examples with:
ant -p
ant examples
note:
The script is now in developing/refactoring phase and all interface/config are
subject to change. I'll let you know how to use the script for your daily l10n
work later (if someone are interesting in our localization tools script).
Thanks.
Alexander Ihrig
the "error" in German l10n is known:
https://bugzilla.mozilla.org/show_bug.cgi?id=462742
Regards
Alex Ihrig
Cédric Corazza
> Hi all, here are error report found in 36 locale found with my script.
> l10n owners:
> Check if your locale and fix errors if your locale have.
> Axel and Seth:
> Please let l10n owners know about their errors if they don't notice this mail.
> In short:
> We found some errors in these locale files:
> ca, de, fr, gl, gu-IN, he, hi-IN, hu, id, ka, kk, kn, ko, lv, mk, ml, mn, mr,
> nl, oc, pa-IN, pt-PT, si, sk, sl, sq, sr, sv-SE, ta-LK, te, th, tr, uk, uk,
> zh-CN, zh-TW
Axel, dynamis: does this mean there is also a duplicated string in en-US?
If this script is useful (I bet it is), so merging it with the current
compare-locales.py script sounds a good thing. Axel?
Regards
Cédric
dynamis
Probablly not in this case but I cannot read German and not sure how you're
going to fix it. So, this is just confirmation or FYI. If you're planning to fix
including 2nd placeholder, ignore this message.
Placeholder Error Found in
/Users/dynamis/mozdev/lot/l10n/de/toolkit/chrome/passwordmgr/passwordmgr.properties:
@line 46: saveLoginTextNoUsername = Soll %1$S dieses Passwort fï¿œr %3$S speichern?
If you intentionally dropping 2nd placeholder to make translation natural, you
can use dummy placeholer like:
saveLoginTextNoUsername = %2$0.SSoll %1$S dieses Passwort fï¿œr %3$S speichern?
Robert Kaiser
> Hi Alexander,
>
> Probablly not in this case but I cannot read German and not sure how
> you're going to fix it. So, this is just confirmation or FYI. If you're
> planning to fix including 2nd placeholder, ignore this message.
>
> Placeholder Error Found in
> /Users/dynamis/mozdev/lot/l10n/de/toolkit/chrome/passwordmgr/passwordmgr.properties:
>
> speichern?
>
> If you intentionally dropping 2nd placeholder to make translation
> natural, you can use dummy placeholer like:
>
> speichern?
It's not true that this ugly workaround is needed in all cases, in fact,
it works well without this in many cases as we recently discussed in the
Thunderbird development group.
Robert Kaiser
dynamis
I didn't know that. Thanks let me know it.
For like this case, my script will have white list feature to skip/ignore each
error check and I'll add saveLoginTextNoUsername as exception of de locale
placeholder check to avoid show useless error message again.
Since simpler rule makes less trouble, maybe ja team will continue to use ugly
workaround in all cases untill we need not use workaround in all case.
Robert Kaiser wrote:
> dynamis wrote:
>> Hi Alexander,
>>
>> Probablly not in this case but I cannot read German and not sure how
>> you're going to fix it. So, this is just confirmation or FYI. If you're
>> planning to fix including 2nd placeholder, ignore this message.
>>
>> Placeholder Error Found in
>> /Users/dynamis/mozdev/lot/l10n/de/toolkit/chrome/passwordmgr/passwordmgr.properties:
>>
>>
>> @line 46: saveLoginTextNoUsername = Soll %1$S dieses Passwort fï¿œr %3$S
>> speichern?
>>
>> If you intentionally dropping 2nd placeholder to make translation
>> natural, you can use dummy placeholer like:
>>
>> saveLoginTextNoUsername = %2$0.SSoll %1$S dieses Passwort fï¿œr %3$S
dynamis
The error check was against mozilla-1.9.1 branch files, not l10n-central.
http://hg.mozilla.org/releases/l10n-mozilla-1.9.1/
# to shankar: as you already noticed and fixed your locale ;)
I'll also make report for l10n-central later when I finished refactoring/debug
my script (not sure when it will be).
If you cannot wait my slow development and want to check l10n-central just now,
of course you can do it by yourselves:
svn export http://svn.mozilla.l10n.jp/trunk/lot/
cd lot
ant setupl10n
-Ddownload.en-US.url=http://ftp-developer.mozilla-japan.org/pub/mozilla-japan/l10n-tools/resources/central/en-US/en-US-latest.zip
-Dl10n.repo.base=http://hg.mozilla.org/l10n-central/
# note: above command is in a line with two long -D option
# input your locale and some other options
ant errorcheck
ant compare
Attachment is the sample output with fr locale.
http://people.mozilla.com/~dynamis/lot/20090429/log/fr.200904290511.compare
http://people.mozilla.com/~dynamis/lot/20090429/log/fr.200904290517.syntaxerror
Duplicate entities and syntaxerror (introduced by miss file copy or rename) are
found in this case.
Thanks.
Robert Kaiser
> For like this case, my script will have white list feature to
> skip/ignore each error check and I'll add saveLoginTextNoUsername as
> exception of de locale placeholder check to avoid show useless error
> message again.
Leave this one in, it's an actual bug, %3$S doesn't even exist in the
original there.
I just wanted to tell you that the %1$0.S workaround isn't needed in all
cases, if I have to guess, it's probably only needed where the string is
used by C++ code and not where it's used by JavaScript code - but then,
this is an educated guess only. There's work underway that may make it
not be needed anywhere, but that's not done for FF 3.5 in any case.
Robert Kaiser
dynamis
I see. Thanks detailed explanation.
I added a task to my todo list:
Compare placeholders with en-US, not only check within l10n locale to avoid
using invalid larger number placeholders which aren't defined in en-US.
Thanks.
dynamis
Cédric Corazza wrote:
> Le 28/04/2009 21:26, dynamis a écrit :
>> Hi all, here are error report found in 36 locale found with my script.
>> l10n owners:
>> Check if your locale and fix errors if your locale have.
>> Axel and Seth:
>> Please let l10n owners know about their errors if they don't notice
>> this mail.
>> In short:
>> We found some errors in these locale files:
>> ca, de, fr, gl, gu-IN, he, hi-IN, hu, id, ka, kk, kn, ko, lv, mk, ml,
>> mn, mr,
>> nl, oc, pa-IN, pt-PT, si, sk, sl, sq, sr, sv-SE, ta-LK, te, th, tr,
>> uk, uk,
>> zh-CN, zh-TW
>
> Axel, dynamis: does this mean there is also a duplicated string in en-US?
No, no duplicate entities in en-US. No errors in other 30+ locales.
# my script also check en-US syntax, duplicate entity errors at the same time.
Localizer update file many many times. Firefox, compare-locales script etc don't
show error even duplicate entity exists. My script detects common human error.
> If this script is useful (I bet it is), so merging it with the current
> compare-locales.py script sounds a good thing. Axel?
To tell the truth, compare-locales.py and compare/merge part of my script are a
kind of brother. Entity block definitions are (almost) same like:
http://hg.mozilla.org/build/compare-locales/file/tip/lib/Mozilla/Parser.py#l50
http://trac.mozilla.l10n.jp/browser/trunk/lot/res/const.conf#L37
# since I contributed for merge feature of compare-locales.py
The reason why I'm writing separate script (not writting patch) is, just it's
much easer and faster (for me) to use ant+groovy to implement all features,
flexibility, usability I want.
Axel:
My script are not compatible and still having some bugs but already including
almost all features compare-locale.py have. If you can accept ant+groovy (not
python), I can support to replace compare-locale.py.
If python is reuirements, I I cannot write complete patch but can explain what
my script does and how to update compare-locales.py to do same. Basic workflow
and data model are similar and not difficult to do same (just need longer
time/code).
> Regards
>
> Cédric
>
Дамјан Георгиевски
thanks, very useful .. one small note, maybe it would be better if you
could've generated a separate errorlog summary file for each locale ...
like mk.errorlog.txt or something
--
дамјан ( http://softver.org.mk/damjan/ )
"We think it's a great consumer win, and it's a great industry win,
to be able to ensure that with good copy protection,
you can have so much functionality for the user",
Jordi Rivas, Microsoft Director of Technology
dynamis
Of course I will.
To tell the truth, it was firstrun of my errorcheck with all-locales and summary
file was made by hand as a byproduct when I check the each log file to see if my
script works well or not (so may contain human error by me).
I of course never want to do it by hand again and I'll automate to generate
summary file per locale.
Thanks feedback.
arjun
> cd lot
> ant setupl10n
> # input your locale and some other options
> ant errorcheck
> ant compare
>
> With most locales you'll see too many output about accesskey matching unless
> your locale's accesskey policy is to sync with en-US (like us). Ignore them and
> see only other messages in the errorcheck/compare log file.
>
> You can try merge and generate firefox langpack with merged files
> ant merge
> ant buildfx
>
> You can see other targets and command examples with:
> ant -p
> ant examples
>
> note:
> The script is now in developing/refactoring phase and all interface/config are
> subject to change. I'll let you know how to use the script for your daily l10n
> work later (if someone are interesting in our localization tools script).
>
> Thanks.
Te fixed.
Thanks
Arjun
Robert Kaiser
> the "error" in German l10n is known:
> https://bugzilla.mozilla.org/show_bug.cgi?id=462742
FIXED now, btw, de should be clear now :)
Robert Kaiser
Filip Miletic
> Hi all, here are error report found in 36 locale found with my script.
>
> l10n owners:
> Check if your locale and fix errors if your locale have.
> Axel and Seth:
> Please let l10n owners know about their errors if they don't notice this mail.
Thanks for letting me know about the errors in sr. I corrected these in
my repo and will push the changes along with other major work. The
errors in sr concern an extension which is not fully translated yet.
f
Ankitkumar Rameshchandra Patel
Thanks!
--
Regards,
Ankit Patel
http://www.indianoss.org/
Axel Hecht
Sorry, but java isn't a requirement that we're likely to add. Let alone
ant and groovy on top of that.
It'd be good to see where in the design these hooks best go, and then
figure out if they should be part of compare-locales.py or silme.
Axel
dynamis
Your answer not accepting java is as I imagined. Then my suggestion is, porting
only important/useful feature from my script to comapre-locales and running on
our server separately for other minor checking feature etc.
I'll continue adding features into my script and will setup our server to check
all locales files periodically.
That is, critical problems are found with compare-locales and required to be
fixed for release, minor or unclear problems will not block release but can
check with our server when needed.
As for porting features, a little more strict syntax check for dtd (error like
"<ul<li>") and duplicate entity check are easy and worth to port IMHO.
# not sure but maybe should allow BOM on properties too (not only dtd)
Merge relating features are not and will not required for official release and
need not port to compare-locales I think. Those who want to use more flexible
merge feature for their daily l10n will be able to use my script.
sounds reasonable?
Axel Hecht
> Hi Axel,
>
> Your answer not accepting java is as I imagined. Then my suggestion is,
> porting only important/useful feature from my script to comapre-locales
> and running on our server separately for other minor checking feature etc.
> I'll continue adding features into my script and will setup our server
> to check all locales files periodically.
> That is, critical problems are found with compare-locales and required
> to be fixed for release, minor or unclear problems will not block
> release but can check with our server when needed.
>
> As for porting features, a little more strict syntax check for dtd
> (error like "<ul<li>") and duplicate entity check are easy and worth to
> port IMHO.
> # not sure but maybe should allow BOM on properties too (not only dtd)
>
> Merge relating features are not and will not required for official
> release and need not port to compare-locales I think. Those who want to
> use more flexible merge feature for their daily l10n will be able to use
> my script.
>
> sounds reasonable?
Yeah. Even though, for merge stuff, we're going to use that in the
nightlies rather sooner than later. So if there's goodness to get there,
we should have that on the list.
Axel