Trac translations and Transifex

[Christian Boos]

Hello,

Last weekend I refreshed our Trac project on Transifex.com, created new
"releases" to match our maintenance branches, renamed a few things,
experimented a bit, and documented all of this in the process:

- http://trac.edgewall.org/wiki/TracL10N#UsingTransifex
- http://trac.edgewall.org/wiki/TracL10N/Transifex

In short, Transifex can be very attractive for occasional translators or
even regular ones if you'd like to benefit from their nice editing
interface.

The only issue is to find an effective "workflow" and ways to integrate
changes from Transifex into the code, and back.

Currently, most of the time the changes are made directly by the
translators on the catalogs, and they commit those changes directly. In
the past those changes were propagated to Transifex, but now they don't
do that anymore and leave the burden of conflict resolution and merge
handling to the developers. So now we have *two* possible sources of
updates, and what we need is a clear picture of how to proceed in order
to keep them in sync and to minimize the risks of confusion.

The synchronization is not trivial due to the following reasons:

1. the differences between the automatic message formatting done by
Transifex and the one in the repository (done manually by the translator
or automatically by Babel),
2. the size of the catalog files and the resulting diffs,
3. the language barrier which doesn't make immediately obvious the
"semantic" of a change (is a deletion corresponding to a synchronization
problem or is it a rightful deletion of a wrong translation?),
4. the number of the translations, which makes it overwhelming to
maintain for a single person

I think I have now a way to address all of the above difficulties.


1) We need to do normalization, and the only practical way to do this is
to use Babel. A translator can decide which version he uses (Babel 0.9.6
or 1.0dev), but he must use the same version for updating the catalog in
the repository and for updating the catalog after pulling changes from
Transifex.

In short, the workflow should be:

$ make update-<lang> # to normalize the catalog
$ edit # eventually...

$ make check-<lang> # the usual quality assurance steps,
$ make compile-<lang> # and if no error is detected:

$ make update-<lang> # **always**, before committing
$ svn ci -m "<branch>/l10n/<lang>: hey, what a day (150%)"

At that point, you have a baseline against which you can compare with
the changes you will pull from Transifex:

$ tx pull -l <lang> [-f]

If you just committed, chances are that you need the "-f" flag to not
have tx "skip" the pull because the modified time of the local file is
more recent than the one on Transifex (tx is the transifex-client, see
the wiki page for installation details). It's also safe to do "-f" in
that case, as you have all your local changes committed, so you don't
risk to overwrite anything permanently.

Right after a tx pull, the *diff is meaningless", due to the automatic
formatting changes done by Transifex. Therefore, you need the second
normalization step:

$ make update-<lang>

Now we can do a diff...


2) But the diff might still be too big, especially if you didn't
update/commit recently before pulling from Transifex. If the differences
are due to the formatting, then you probably *didn't* use the same
version of Babel, go back to step 1. and recreate a new baseline. If the
differences are in the line numbers because an extraction happened in
the meantime, then you can use "make diff" which filters out the
irrelevant changes:

$ make diff-<lang> [vc=git,hg]

The recently added vc= enables you to create your baseline against a
local commit, made in a checkout from one of the Trac git or hg mirrors.


3) To make it easier to decide whether the changes coming from Transifex
are worth integrating, we can establish a simple rule which would enable
even a non speaker of the language to take action: a deletion is always
wrong. If someone thinks that a translation is bad, then she has to
*replace* it with a better one, not delete it. Following that rule, we
could always decide that removals are wrong and should be discarded.

Once the changes have been edited, do again:

$ make check-<lang> # the usual quality assurance steps,
$ make compile-<lang> # and if no error is detected:

$ make update-<lang> # to normalize the catalog
$ svn ci -m "<branch>/l10n/<lang>: integrated changes from Transifex"
$ tx push -t -l <lang>

(you normally won't need to add "-f" to the above push, as the local
changes are now newer than those on Transifex)

So always remember to push the changes back to Transifex.com after you
integrated the changes from there. But never blindly push to
Transifex.com after you committed your own edits (like in step 1.), as
this would risk to overwrite changes contributed by others; in that
situation, go through step 2. (and 3. if needed) before pushing.


4) Finally, as you might have guessed, my answer to the huge amount of
verification work due to the large number of translations we have is to
ask each translator coordinator to take his share ;-)
I hope it won't be too hard now, given the above explanations and the
complementary ones written in the wiki.

In fact, not all translators need to do this, only those for which
there's already a "Translation Team" set up on Transifex.

Note that ideally, the Trac committer for a given translation and the
Transifex Team coordinator should be the same person.

Also, at the present moment the situation is a bit shaky, we have
several issues:

- possible discrepancy between committer status and team coordinator status
- teams without committers
- teams without translations content
- "wrong" teams (tr_TR, while the correct tr one is marked as having
"no team yet")

In more details, along with the suggested course of action:

* The languages for which we have a committer and a Team on Transifex
(language: committer/coordinator):

- Chinese (China): zengjie/gnozil
- Chinese (Taiwan): zclin/zclin
- French: cboos/cboos
- German: fschwarz/fschwarz
- Dutch: lholst/trbs
- Hebrew: itamaro/Yaron
- Italian: lgaifax/seccanj
- Japanese: jomae/jun66j5 (same ;-) )
- Portuguese (Brazil): lwatter/mfcn
- Spanish: raragon/codeadict
- Spanish (Argentina): rmorales/cramm (same)
- Russian: aprudnikov/bishop (same?)
- Swedish: mrelbe/mrelbe

If the committer is not active anymore and the Translation coordinator
wants to get commit access, just ask. It's not a big deal to have more
than one people having access to the translations in the repository
(only the usual coordination and merge workflow), as long as they all
correctly follow the guidelines given above. Maybe in addition, they'll
have to settle on a given Babel version, in order to not have too many
spurious reformatting changes. It's more of a problem when there's *no
one* having commit access! Also, if an active committer is not the
Transifex coordinator, no big deal either: being coordinator there is
just about granting access to new translation contributors. If you think
someone should have received access and still hasn't, ask me directly as
I can also grant access.

* The languages for which we have a committer but no Team and therefore
no coordinator:

- Armenian: narine_martirosyan/-
- Catalan: jordi/-
- English (United Kingdom): mrelbe/-
- Esperanto: wdiestel/-
- Hungarian: nzoltan/-
- Norwegian Bokmål: osimons/-
- Slovenian: vezjakv/-
- Turkish: asurion/-

Creating a team is of course left at the discretion of the committer for
that language. If as a committer you don't have a Translation Team
there, you should think about creating one, as chances are that you'll
get some more contributions, and hopefully it's now clear how to manage
them! If someone else wants to step up as translation team coordinator,
then he should also become committer.

* The languages that we have in the repository, but for which the
contributor(s) only sent patches and are not committers, and for which
there's no team on Transifex yet:

- Galician
- Korean
- Persian
- Portuguese
- Vietnamese
- Romanian

Those are mostly translations coming from the early days of 0.12dev,
which have been contributed as patch and were not updated since a while.

* The language teams on Transifex for which there's no content, neither
in Transifex, nor in the repository:

- Afrikaans
- Hindi
- Indonesian (Indonesia)
- Panjabi (Punjabi)
- Turkish (Turkey) - i.e. "tr_TR", but we have a "tr" translation

Well, no content, nothing to do.

* The language teams on Transifex for which there's some content, for
which some version of the translation is in the repository, but there's
no committer yet:

- Czech: -/Tsbook
- Estonian: -/lkraav
- Finnish: -/mikkorantalainen
- Greek: -/Evropi
- Polish: -/wiget
- Ukrainian: -/sgalashyn

Here, the translation team coordinators of Transifex should step up to
get svn access, *iff* the translations are advanced enough (> 50% for a
given release).

* The language teams on Transifex for which there's some content, but
nothing yet in the repository:

- Bosnian -/kenan3008
- Danish: -/jenshyllegaard
- Spanish (Mexico): -/tzkmx

The "da" and "es_MX" ones ought to be in the repository, as they're at
100% of completion on 0.12-stable! Here as well, the translation team
coordinators should step up to get svn access (but I'll commit the
initial version).

Ok, I realize the categorization above is perhaps not crystal clear,
I'll update the wiki page with a nice table summarizing this ;-)

Don't hesitate to share your opinion on the above and your suggestions.

-- Christian