Julia documentation is ready for translation/internationalization on Transifex!

[Ismael VC]

Merry Christmas everyone!

I have an announcement and an invitation to make to the Julia community.

Yesterday I finally finished seting-up, julialang docs on Transifex (online crowdsourcing localization platform) something I wanted to do ever since last year, but sadly never had time to actually invest on it, my intention is to translate the docs to spanish, and also make an invitation to anyone interested in helping me out or translating them to their native language.

You can see my setup here: https://github.com/Ismael-VC/julia/tree/julia-translations/doc

Please don't hesitate to ask me any question or give me advice I really want to do this well. If any of you have experience with any of this I would really really love to have a chat with you!

! I'm still testing the current setup (I don't know why `make gettext` takes so much time, yesterday it was done in a breeze:

[ismaelvc@toybox doc]$ time ./update_resources.sh
Enter passphrase for key '/home/ismaelvc/.ssh/id_rsa':
From github.com:JuliaLang/julia
 * branch            master     -> FETCH_HEAD
Already up-to-date.
. /home/ismaelvc/julia/deps/julia-env/bin/activate && sphinx-build -b gettext   . _build/locale
Running Sphinx v1.2.3
loading pickled environment... done
building [gettext]: targets for 0 template files
building [gettext]: targets for 56 source files that are out of date
updating environment: 0 added, 2 changed, 0 removed
reading sources... [ 50%] manual/faq

It just stays there (half an hour already ...tried several times). Anyway that's only for updating resources, but I can get to work with what I already  have.

You can find the project here: https://www.transifex.com/projects/p/julia-doc/


Ismael VC

 

[Ismael VC]

Ok It really did finish, that's a relief!

[ismaelvc@toybox doc]$ time ./update_resources.sh
Enter passphrase for key '/home/ismaelvc/.ssh/id_rsa':
From github.com:JuliaLang/julia
 * branch            master     -> FETCH_HEAD
Already up-to-date.
. /home/ismaelvc/julia/deps/julia-env/bin/activate && sphinx-build -b gettext   . _build/locale
Running Sphinx v1.2.3
loading pickled environment... done
building [gettext]: targets for 0 template files
building [gettext]: targets for 56 source files that are out of date
updating environment: 0 added, 2 changed, 0

removed
reading sources... [100%] manual/strings                                        
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] stdlib/test                                            
writing message catalogs... [100%] devdocs                                      
build succeeded.

Build finished. The message catalogs are in _build/locale.
Not Changed: locale/es/LC_MESSAGES/devdocs.po
Not Changed: locale/es/LC_MESSAGES/index.po
Update: locale/es/LC_MESSAGES/manual.po +62, -58
Not Changed: locale/es/LC_MESSAGES/stdlib.po
Not Changed: locale/es/LC_MESSAGES/latex.po
[julia-translations 8c36236] updating resources
 9 files changed, 191 insertions(+), 226 deletions(-)
 rewrite doc/_build/locale/.doctrees/environment.pickle (94%)
 rewrite doc/_build/locale/.doctrees/manual/faq.doctree (72%)

Enter passphrase for key '/home/ismaelvc/.ssh/id_rsa':

Counting objects: 26, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (23/23), done.
Writing objects: 100% (26/26), 122.07 KiB | 0 bytes/s, done.
Total 26 (delta 17), reused 0 (delta 0)
To git@github.com:Ismael-VC/julia.git
   8166cbe..8c36236  julia-translations -> julia-translations
Pushing translations for resource julia-doc.devdocs:
Pushing source file (_build/locale/devdocs.pot)
Pushing translations for resource julia-doc.index:
Pushing source file (_build/locale/index.pot)
Pushing translations for resource julia-doc.manual:
Pushing source file (_build/locale/manual.pot)
Pushing translations for resource julia-doc.stdlib:
Pushing source file (_build/locale/stdlib.pot)
Pushing translations for resource julia-doc.latex:
Pushing source file (_build/locale/latex.pot)
Done.

real    59m11.596s
user    56m12.856s
sys    0m3.877s
[ismaelvc@toybox doc]$

If found this related issue: https://bitbucket.org/birkenfeld/sphinx/issue/1426/gettext-builder-is-very-slow-during-read

[Ismael VC]

Here is a screenshot of the transifex editor in action, I wonder if it works from a smartphone!

[Ivar Nesje]

This is great!

transifex looks good, and I wish you the best of luck. I'm no use in Spanish or German, but if I see a great (updated) manual in a few languages in 6 month time, I might look into translating to Norwegian.

I just want to give a word of warning that the Julia documentation isn't static, so you will need some simple (and partially automated) way of keeping everything up to date.

Ivar

[Ismael VC]

Thank you very much Ivar!

There has been requests for adding more languages to the project already, Russian and Portuguese, but actually translating them will be responsibility of whomever works for them, I can only focus in Spanish.

I'm well aware that the docs aren't static, which is in fact one of the reason that I have studied and planned the way to do it and waited up until now to actually start with the translation (I don't want to wait until it reaches 1.0 or something like that). You can check out the repo I'm working on here:

https://github.com/Ismael-VC/julia/tree/julia-translations/doc

And the sphinx internationalization page:

http://sphinx-doc.org/latest/intl.html


I'll try to describe the workflow briefly.

1. Periodically pull changes from Julia
2. use gettet to compile a catalog of pot iles
   
3. update the pot files in transifex from these files

All while one keeps translating the files and idealy, once one finishes translating "all", then it's just a matter of translating only the changed/new strings from the update at a time.

I do it with this script (which I'm still testing): https://github.com/Ismael-VC/julia/blob/julia-translations/doc/update_resources.sh

When the `pot` files are translated, they are pulled back again, transformed into `mo` files, and then it's the same (using `transifex-client`).

I'm using `sphinx-intl` and `transifex-client`, which let me work towards continuos internationalization and localization, and transifex enables us to work together in the same platform, without having each project doing their own setup.

I just want for people to start translating, and focus on that, instead on focusing on managing the resources, separately.

I decided to start now, because even while it's true that the docs are a moving target, it's not like it's a major rewrite of it all the time. And I want excellent and up to date documentation available in my language with the exact same look and feel as the original version, so pherhaps one day we could integrate it.

Sorry for my english I'm not that good at it. :P

Cheers

Ismael VC

[Waldir Pimenta]

Hi Ismael,

First of all, thanks for doing this :)

I am not sure if you announced your plans to work on this in advance, so sorry if I missed it and thus the opportunity to provide early feedback. The last time I saw the issue of translating the documentation using a translation platform being discussed, I mentioned my concerns about the choice of platform, particularly regarding two features I consider extremely useful: translation memory / machine translation (which ends up saving a lot of typing, as well as ensuring consistency), and the ability for users to choose their language fallback chain (e.g. a Portuguese user could pick Brazilian Portuguese, Galician, and Spanish as fallback languages), which again allows a much more efficient translation workflow and often help escaping ambiguity holes due to insufficient context.

IIRC Transifex doesn't provide these features, while e.g. Crowdin and TranslateWiki do. Have you considered either of them? Would it be feasible to change the platform at this time?

[Ismael VC]

Well I was told that historicaly people were encouraged not to spend too much energy translating the docs because they are not static (and I see that still hasn't changed). I thought there wasn't much interest, so I left the idea for a while and stoped looking for advance in this field.

But I've been planing this for a while, after all I want to be a programmer, and learn how to automate this kind of stuff the most I can and solve problems, etc (I don't mind about non-staticness, as a matter of fact I like the idea of keep on learning Julia at the same time I do this):

• How to translate the docs: http://bit.ly/13Kgt18
• Does anyone has any experience in computer aided translating?: http://bit.ly/144ZIyy
• Advice about git repository layout: http://bit.ly/1zncd0b

I'm just so tired of JIT translating from the manual when I'm in the classroom or when explaining what Julia is and why it is important to learn and use, to someone, over and over again, I think I can get a group of students from my school to aid me in the review process, I've talked to one of my profesors about giving them some extra credits or something.

Waldir I would love to get feedback from you, I just followed the sphinx instructions, but if there are better options that benefit all the non english people, I would like to implement them as soon as posible (while my vacations last), this would be the best time to do this since the project is just starting, but either way I'm not stoping this time, I just want to get it done and done well.

My ulitmate goal is that once most of the strings are translated and reviewed, make a program automatically update the resources, when the tracked files are changed and get a notification, if I can get some people to translate or review in trasifex (even if it is just one string a day, 5 minutes of their time) from their cellphones instead of watching them waste all day playing candycrush or whatever (most of the smarphone people I know do this), then that would make me very very happy.

I absoulutely want this manual on my school library!

[Ivan Ogasawara]

Hi Ismael, maybe I can help you with Brazilian Portuguese and Spanish translations.

this week I'm on vacation .. but the next week I can start it (maybe 2 hours per week).

I need to learn too the transifex editor .. but I think it not will be a problem :)

Translation Memory will be great ... but the only one I know is http://anaphraseus.sourceforge.net/ to openoffice :(

Probably I will need a help to start the project :)

My best regards

[SVAKSHA]

On Fri, Dec 26, 2014 at 9:15 PM, Ismael VC <ismael...@gmail.com> wrote:
>
> My ulitmate goal is that once most of the strings are translated and
> reviewed, make a program automatically update the resources, when the
> tracked files are changed and get a notification, if I can get some people

Would Apertium[1] help you automate the Spanish translation process?
IIRC, they support En<->Es and the rate was quite good - iirc, was
above 70% (sorry, its been many years so I'm not sure if that has
changed, which I'm sure it has since they have been a regular
participant in GSOC and the engine must have improved in leaps over
the years).

Ofcourse, you would still need to manually verify the strings but it
helps you automate most of the regular stuff you could perhaps train
it for any terms that are missing.

Francis is one of the lead devs among others and they are a helpful
bunch - if you prefer irc, hop over to #apertium on freenode.

[1] https://en.wikipedia.org/wiki/Apertium
HTH,
-SVAKSHA ॥ http://about.me/svaksha ॥

[Ivan Ogasawara]

I saw apertium 5 years ago on FISL (Brazil) ... It seems great... but i've never work with that ... this can be the first time :)

[Ivan Ogasawara]

***2 years ago

[Ismael VC]

Ivan any time you can invest into the project is very much apreciated, wellcome and let me know if you need anithing, I'll leave you some doc links about the editor here:

• http://docs.transifex.com/guides/txeditor
  
• https://www.youtube.com/watch?v=zw677Bp1En8
  

I will check out anaphrasues, thank you very much!

[Ismael VC]

Svanska thank you very much I will definetly check out apertium!

[Ismael VC]

I think my email adress if showing up truncated, everyone please feel free to contact me here: ismael...@gmail.com

[Ismael VC]

Just as a suggestion, I use this pomodoro app, to optimize my invested time into the project and abvoid procastination: http://tomato-timer.com/ :)

El jueves, 25 de diciembre de 2014 11:34:01 UTC-6, Ismael VC escribió:

[Waldir Pimenta]

On Friday, December 26, 2014 9:15:34 PM UTC, Ismael VC wrote:

Waldir I would love to get feedback from you, I just followed the sphinx instructions, but if there are better options that benefit all the non english people, I would like to implement them as soon as posible (while my vacations last), this would be the best time to do this since the project is just starting, but either way I'm not stoping this time, I just want to get it done and done well.

Well, my main feedback was the suggestion to use a different platform than Transifex (unless they support the features I mentioned, but they aren't enabled by default). If you want a specific recommendation, I'd suggest Crowdin. I've been using it recently (as a volunteer translator for another project) and it works really well! Could you give it a try?

[Ismael VC]

Thank you for your suggestions Waldir, I'll be checking out Crowdin today!

I've just found this transifex feature very useful; tranlsaltion of web pages, simply by adding the url of julialang.org,
Is there anything like this in Crowdin?

You can see my progress here (I'm almost finished with the spanish translation, I just hope there is enough interest to add support for internationalization to the webpage, I haven't recieved any message about it yet, but I have my fingers crossed):

https://groups.google.com/forum/#!topic/julia-users/UCKss8S9LB8

Is there a command line client for crowding, analog to the transifex-client, or how could I automate the update process?

I'd also like feedback about how to configure those disabled features by default, so I can keep focusing on the translation per se.

I'm gonna start uploadig the resources, that we already have translated to Crowdin and see if I can continue from there.

Cheers
Ismael VC

[Ismael VC]

I think it could take a hile before I can start to use it as an Open Source project I only have a 10 days free trial, but I have already sent a request:

Crowdin is completely free of charge for Open Source projects. We love Open Source and ready to support community projects.

Before submitting this form, please ensure your project meets the following criteria:

1. The project is licensed under an approved license from the Open Source Initiative.
2. Your project source code is available for download to the public.
3. You do not have Commercial products related to the Open Source project you are requesting a license for.
4. You are the project lead.
5. You have been working on your open source project for at least 3 months.
6. You have an active community of collaborators.
7. You keep the News section on your site up to date.
8. You release updated builds on a regular basis.

Let's see what happens, in the mean time, ...lets keep translating some more! :D

Crowdin is completely free of charge for Open Source projects. We love Open Source and ready to support community projects.

Before submitting this form, please ensure your project meets the following criteria:

1. The project is licensed under an approved license from the Open Source Initiative.
2. Your project source code is available for download to the public.
3. You do not have Commercial products related to the Open Source project you are requesting a license for.
4. You are the project lead.
5. You have been working on your open source project for at least 3 months.
6. You have an active community of collaborators.
7. You keep the News section on your site up to date.
8. You release updated builds on a regular basis.

El jueves, 25 de diciembre de 2014 11:34:01 UTC-6, Ismael VC escribió:

[Waldir Pimenta]

On Monday, December 29, 2014 6:57:43 PM UTC, Ismael VC wrote:

Thank you for your suggestions Waldir, I'll be checking out Crowdin today!

I've just found this transifex feature very useful; tranlsaltion of web pages, simply by adding the url of julialang.org,
Is there anything like this in Crowdin?

I don't know, sorry, I only ever used these platforms as a volunteer translator, never as a project admin.

Is there a command line client for crowding, analog to the transifex-client, or how could I automate the update process?

Same as above, I don't know :/

I'd also like feedback about how to configure those disabled features by default, so I can keep focusing on the translation per se.

I was perhaps unclear: what I meant was that I never saw the features I mentioned in my first message (machine translation/translation memory, and configuring fallback languages) in any Transifex project that I have participated in so far, so I assumed it doesn't support them. The alternative was that it does support them, but none of the projects I've seen had them activated, but that was just speculation.

I'm gonna start uploadig the resources, that we already have translated to Crowdin and see if I can continue from there.

Great, let us know when we can apply to join the project there :D