Keeping language packs from different languages in sync

[Alexander Obuhovich]

Here is some ideas how we can keep language packs from multiple languages synchronized with primary language pack for English language.

We invent such term as translatable unit. Units could consist of one or more fields (textareas) that could be translated. Also translatable unit will feature a Description field, that will tell translator, where exactly given text will be used, so translation could be more accurate and appropriate. Each translatable unit have unique identifier (phrase name for phrases and email event name + email event type for email events). Each translatable unit have version number (integer). Once primary language translation is updated or new translation unit is added, then it's version number is incremented.

When we perform translation of set of the translatable units (language pack) to other language, then each unit will keep it's version, version used to make initial translation). When primary unit translation (on English language) will be changed, then it's version number will changed and will differ from all other translated units in other language packs. This way we can detect translation units, that need to be updated to be in sync.

I suppose, that this will be some script on http://www.in-portal.com website. Each website user can register as "translator" for any count of languages so he'll get notified when his expertise is need to keep associated language pack in sync. Page with language packs will be automatically constructed based on finished translation for each language pack and translation progress bar will available for each language pack.

I'll try to look if something, that I've described is already available on the Internet, if not will write something myself.

Let me know If you have any ideas about this.

--
Best Regards,

http://www.in-portal.com
http://www.alex-time.com

[Phil ..:: domicilis.biz ::..]

Hello Alexander,

translation themselves don't really need any update as:

- when a word or phrase is translated, it's meaning won't change over the time

- in french, I adapt many french phrases and words from the original
french pack to fit the front-end needs and requests, thus any update
would destroy all custom work

- if a new module is released, then we'll maybe add some translations,
maybe remove other ones, this part could be updated: on install, we
remove tag named "xxx" and add tag "yyy". After this update, it'll be
very easy to find new tags, as we can sort all tags by modification
date in Regional menu

More pragmatically, did you had any request involving this kind of function?

Sorry if I'm not so much enthusiast, but to be honest I think we have
really a greater job to do with VAT, which is far more usefull for
spreading the soft outside US...
Please have look to my post, 6 monthes ago...
http://groups.google.com/group/in-portal-dev/browse_thread/thread/60b03403de998323/f86601343c167e51?hl=fr&lnk=gst&q=vat#f86601343c167e51

Phil.

2010/2/8 Alexander Obuhovich <aik....@gmail.com>:

> --
> You received this message because you are subscribed to the Google Groups
> "In-Portal Localization Team" group.
> To post to this group, send email to in-port...@googlegroups.com.
> To unsubscribe from this group, send email to
> in-portal-loc...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/in-portal-local?hl=en.
>

[Alexander Obuhovich]

I've found what I've imagined at http://localize.drupal.org/translate?no_cache=1265705125 .

So it's not common practice for international users to install In-Portal and then download language pack for other language and install it as well? In the last months we are really renaming/translating existing phrases and email events a lot and I think that current language packs on other languages are really outdated.

[Phil ..:: domicilis.biz ::..]

Of course installing new language is common, sorry if I didn't
understoud well your willing, I was saying that "updating" lang packs
is not usefull.

Il the goal is to make lang pack working better, I agree; for example,
we don't have any hint on how have been processed the import, how many
tags replaced....

2010/2/9 Alexander Obuhovich <aik....@gmail.com>:

[Alexander Obuhovich]

Assuming you mean phrase name as tag, then only way to keep phrase renames in sync is via upgrade scripts, as we already do. Whole proposed idea is only new installations, so when we change English language pack, then we will know what language packs on other languages should be updated as well.

[Phil ..:: domicilis.biz ::..]

ok, slightly different from what I've understoud first :)
we install language only 1 time in a website life, but why not...

2010/2/9 Alexander Obuhovich <aik....@gmail.com>:

[Alexander Obuhovich]

We can expand this idea to site upgrade so we will track what phrases, email events user have changed and what of there were changed on original language pack in new version of In-Portal and overwrite only not touched by the user.

[Phil ..:: domicilis.biz ::..]

this way it could be really usefull, as I needed to code my own
duplicate tag finder to move from 4,x to 5,x translations...

2010/2/9 Alexander Obuhovich <aik....@gmail.com>:

[Dmitry A.]

Hi Alex, Phil,

I just wanted to check if we are clear on this and it's ready to be
filed as Infrastructure Feature Request?

DA.

On Feb 9, 6:01 am, "Phil ..:: domicilis.biz ::.." <p...@domicilis.biz>
wrote:

> this way it could be really usefull, as I needed to code my own
> duplicate tag finder to move from 4,x to 5,x translations...
>

> 2010/2/9 Alexander Obuhovich <aik.b...@gmail.com>:

>
>
>
> > We can expand this idea to site upgrade so we will track what phrases, email
> > events user have changed and what of there were changed on original language
> > pack in new version of In-Portal and overwrite only not touched by the user.
>
> > On Tue, Feb 9, 2010 at 12:51 PM, Phil ..:: domicilis.biz ::..
> > <p...@domicilis.biz> wrote:
>
> >> ok, slightly different from what I've understoud first :)
> >> we install language only 1 time in a website life, but why not...
>

> >> 2010/2/9 Alexander Obuhovich <aik.b...@gmail.com>:

> >> > Assuming you mean phrase name as tag, then only way to keep phrase
> >> > renames
> >> > in sync is via upgrade scripts, as we already do. Whole proposed idea is
> >> > only new installations, so when we change English language pack, then we
> >> > will know what language packs on other languages should be updated as
> >> > well.
>
> >> > On Tue, Feb 9, 2010 at 11:34 AM, Phil ..:: domicilis.biz ::..
> >> > <p...@domicilis.biz> wrote:
>
> >> >> Of course installing new language is common, sorry if I didn't
> >> >> understoud well your willing, I was saying that "updating" lang packs
> >> >> is not usefull.
>
> >> >> Il the goal is to make lang pack working better, I agree; for example,
> >> >> we don't have any hint on how have been processed the import, how many
> >> >> tags replaced....
>

> >> >> 2010/2/9 Alexander Obuhovich <aik.b...@gmail.com>:

> >> >> >>http://groups.google.com/group/in-portal-dev/browse_thread/thread/60b...
>
> >> >> >> Phil.
>
> >> >> >> 2010/2/8 Alexander Obuhovich <aik.b...@gmail.com>:

[Phil ..:: domicilis.biz ::..]

Hi,

Alex, I think you have a clearer view than me on this feature...

Phil.

2010/3/25 Dmitry A. <dand...@gmail.com>:

[Alexander Obuhovich]

Yes. Ready.

[Dmitry Andrejev]

Hi Alex,

I have reviewed your description from top to the bottom. 

To be honest I think you'll do a better job converting it to the task, if you don't mind of course.

DA.

[Alexander Obuhovich]

Here is the task: http://tracker.in-portal.org/view.php?id=1023

Sorry for 1 year delay.

[Phil -- wbtc.fr --]

late is better than never :)

2011/3/13 Alexander Obuhovich <aik....@gmail.com>

[Alexander Obuhovich]

Some new ideas arrived (after 2 years of silence).

I've imagined that such (or alike) translation system would also be useful for In-Portal websites, who are translated into multiple languages to keep all phrases/e-mails in sync between languages.

Actually I've imagined new system for keeping all in sync.

1. Add "TranslateFrom" ML column into LanguageLabels (former "Phrases") and EmailEvents (former "Events") database tables
2. Add "Use as Primary" button on phrase/e-mail event edit page (with hint "Use this translation as primary for other languages")
3. Initially all TranslateFrom columns have 0 in them, that means - "I'm in sync".

As you probably noticed user then can press either "Save" or "Use as Primary" button after changing a translation on a specific language he is assigned to translate.

Pressing "Use as Primary" would save phrase/e-mail event translation and set this language ID to TranslateFrom field for all other languages except current one:

• l1_ TranslateFrom = 3
• l2_ TranslateFrom = 3
• l3_ TranslateFrom = 0
• l4_ TranslateFrom = 3
• l5_ TranslateFrom = 3

In example above you can easily see, that translation was changed on 3rd language and all other languages needs to be updated to be in sync.

Pressing "Save" would set 0 to TranslateFrom column at current language only, like in example below:

• l1_ TranslateFrom = 3
• l2_ TranslateFrom = 3
• l3_ TranslateFrom = 0
• l4_ TranslateFrom = 3
• l5_ TranslateFrom = 0

Here 5th language translation was fixed to be in sync and 0 was set in corresponding field.


Then in phrase/e-mail event list we create column "Translation in Sync" (Yes/No), where we would just compare TranslateFrom column from current language with 0. If it's 0, then all is translated.

On phrase editing page we'll show translation form language indicated in TranslateFrom column and not from primary language. Of course if TranslateFrom = 0, then we can fallback to translation from primary language.

Also I've noticed that we're not showing translation from primary language on e-mail event editing, which makes it harder to translate them.

[Phil]

Seems perfect. I'd just propose another grid column in Regional
Configuration menu, called "sync status", where we could display % of
synced labels, for reporting purposes.

On 4 mai, 12:53, Alexander Obuhovich <aik.b...@gmail.com> wrote:
> Some new ideas arrived (after 2 years of silence).
>
> I've imagined that such (or alike) translation system would also be useful
> for In-Portal websites, who are translated into multiple languages to keep
> all phrases/e-mails in sync between languages.
>
> Actually I've imagined new system for keeping all in sync.
>

>    1. Add "TranslateFrom" ML column into LanguageLabels (former "Phrases")

>    and EmailEvents (former "Events") database tables

>    2. Add "Use as Primary" button on phrase/e-mail event edit page (with

>    hint "Use this translation as primary for other languages")

>    3. Initially all TranslateFrom columns have 0 in them, that means - "I'm

>    in sync".
>
> As you probably noticed user then can press either "Save" or "Use as
> Primary" button after changing a translation on a specific language he is
> assigned to translate.
>

> Pressing "*Use as Primary*" would save phrase/e-mail event translation and

> set this language ID to TranslateFrom field for all other languages except
> current one:
>

>    - l1_ TranslateFrom = 3
>    - l2_ TranslateFrom = 3
>    - l3_ TranslateFrom = 0
>    - l4_ TranslateFrom = 3
>    - l5_ TranslateFrom = 3

>
> In example above you can easily see, that translation was changed on 3rd
> language and all other languages needs to be updated to be in sync.
>
> Pressing "Save" would set 0 to TranslateFrom column at current language
> only, like in example below:
>

>    - l1_ TranslateFrom = 3
>    - l2_ TranslateFrom = 3
>    - l3_ TranslateFrom = 0
>    - l4_ TranslateFrom = 3
>    - l5_ TranslateFrom = 0

>
> Here 5th language translation was fixed to be in sync and 0 was set in
> corresponding field.
>
> Then in phrase/e-mail event list we create column "Translation in Sync"
> (Yes/No), where we would just compare TranslateFrom column from current
> language with 0. If it's 0, then all is translated.
>
> On phrase editing page we'll show translation form language indicated
> in TranslateFrom column and not from primary language. Of course
> if TranslateFrom = 0, then we can fallback to translation from primary
> language.
>
> Also I've noticed that we're not showing translation from primary language
> on e-mail event editing, which makes it harder to translate them.
>

> On Sun, Mar 13, 2011 at 4:43 PM, Phil -- wbtc.fr -- <p...@wbtc.fr> wrote:
>
>
>
> > late is better than never :)
>

> > 2011/3/13 Alexander Obuhovich <aik.b...@gmail.com>

>
> >> Here is the task:http://tracker.in-portal.org/view.php?id=1023
>
> >> <http://tracker.in-portal.org/view.php?id=1023>Sorry for 1 year delay.
>

> >> On Fri, Mar 26, 2010 at 5:27 PM, Dmitry Andrejev <dandre...@gmail.com>wrote:
>
> >>> Hi Alex,
>
> >>> I have reviewed your description from top to the bottom.
>
> >>> To be honest I think you'll do a better job converting it to the task,
> >>> if you don't mind of course.
>
> >>> DA.
>

> >>> On Thu, Mar 25, 2010 at 2:58 PM, Alexander Obuhovich <aik.b...@gmail.com

> >>> > wrote:
>
> >>>> Yes. Ready.
>
> >>>> On Thu, Mar 25, 2010 at 11:09 AM, Phil ..:: domicilis.biz ::.. <
> >>>> p...@domicilis.biz> wrote:
>
> >>>>> Hi,
>
> >>>>> Alex, I think you have a clearer view than me on this feature...
>
> >>>>> Phil.
>

> >>>>> 2010/3/25 Dmitry A. <dandre...@gmail.com>:

> ...
>
> plus de détails »

[Alexander Obuhovich]

Since in new system we no longer stick to primary language as source for all translations, then how do we calculate that percent?

Maybe SUM(IF(l5_TranslateFrom = 0, 0, 1)) / COUNT(l5_TranslateFrom).

• SUM(IF(l5_TranslateFrom = 0, 1, 0)) - phrase count, that doesn't require to be synced
• COUNT(l5_TranslateFrom) - total phrase count

--
You received this message because you are subscribed to the Google Groups "In-Portal Localization Team" group.
To post to this group, send email to in-port...@googlegroups.com.
To unsubscribe from this group, send email to in-portal-loc...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/in-portal-local?hl=en.

[Phil]

seems good :)

[Dmitry A.]

Nice idea Alex!

Are we still want to get In-Portal.com involved to allow users to Synchronize languages from DB of language packs or something like this. I am not sure how necessary this is at present time.

DA 

[Alexander Obuhovich]

Nice idea means, that we can try it out in 5.2.0?

--

You received this message because you are subscribed to the Google Groups "In-Portal Localization Team" group.

To view this discussion on the web visit https://groups.google.com/d/msg/in-portal-local/-/QabKtQ4tvtQJ.

To post to this group, send email to in-port...@googlegroups.com.
To unsubscribe from this group, send email to in-portal-loc...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/in-portal-local?hl=en.

[Alexander Obuhovich]

Dmitry suggested another idea, that might help us with content block translation tracking a bit.

Create a "Scheduled Task" (former "Agent"), that would count % of translated content blocks for each page, which has them. It of course won't allow to track outdated translations as with phrases, but it at least would tell what content blocks aren't translated at all on other languages.

Add multilingual (one column per language) TranslatedContentBlocks column. For each language execute this SQL to fill it:

SELECT SUM( IF(COALESCE(l1_Content, '') = '', 0, 1) ), PageId

FROM PageContent

GROUP BY PageId

Then just display field in category list, that is calculated using this formula:

(l2_TranslatedContentBlocks / l1_TranslatedContentBlocks) * 100

Then you can perfectly see how much content blocks require initial translation on other languages.

[Phil]

content blocks... I'd say that before adding extra feature such as the one you describe, it'd be really more useful to create a tool to import/export them.

It's a good idea to think to adapt Phrases features to Content blocks anyway.

But as a daily user of In-Portal ML stuff, I'd say that I miss more backup tool, rather than a checking tool.

For example, import/export can be used to work offline on contents, and upload them once completed.

[Dmitry A.]

Thanks for details everyone.

I guess at this point we should update already existing task since it's not fully describes what we want to do now?

http://tracker.in-portal.org/view.php?id=1023 

DA 

[Phil]

weird, I can't access this task on Mantis, my access is denied... normal?

-- 
You received this message because you are subscribed to the Google Groups "In-Portal Localization Team" group.

To view this discussion on the web visit https://groups.google.com/d/msg/in-portal-local/-/xvGNKXHlSRwJ.

[Dmitry A.]

Try now.

DA

[Phil]

works now, thanks.

Le mardi 15 mai 2012 à 23:45, Phil a écrit :

weird, I can't access this task on Mantis, my access is denied... normal?

Envoyé avec Sparrow

Le mardi 15 mai 2012 à 22:38, Dmitry A. a écrit :

--

You received this message because you are subscribed to the Google Groups "In-Portal Localization Team" group.

To view this discussion on the web visit https://groups.google.com/d/msg/in-portal-local/-/xvGNKXHlSRwJ.

[Alexander Obuhovich]

Current task describes same result, but different way of reaching it.

[Dmitry A.]

Updated current task to describe newly discussed things.

DA 

[Alexander Obuhovich]

Fixed some minor description details in task, but it's good to go otherwise.

On Thu, May 17, 2012 at 7:05 AM, Dmitry A. <dand...@gmail.com> wrote:

Updated current task to describe newly discussed things.

DA 

--

You received this message because you are subscribed to the Google Groups "In-Portal Localization Team" group.

To view this discussion on the web visit https://groups.google.com/d/msg/in-portal-local/-/Qf41MnjUtvUJ.

To post to this group, send email to in-port...@googlegroups.com.
To unsubscribe from this group, send email to in-portal-loc...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/in-portal-local?hl=en.

[Alexander Obuhovich]

Don't remember if I wrote this before, but I think that if new phrases/e-mail events are created during language pack import, then they should have "TranslateFrom" column set to language that is currently imported. Since they only exist on that language and other languages needs to be adapted to have their translation as well.

[Dmitry A.]

Good idea and thanks for clarifying, Alex.

DA

[Alexander Obuhovich]

Currently keeping language packs from different languages in sync is major issue of localization team. I even doubt, that we have 5.1.3 version compatible language packs from at least 2 different languages.

Using language sync engine we've talked about earlier in this discussion (that one with TranslateFrom columns involved) we can create a separate In-Portal installation with a theme that would allow (on front-end) to:

1. register user as a translator for a language (or multiple languages)
2. see translation (on a given language) completion %
3. translate language pack in an easy ajax-based interface 

Of course if 2+ users have registered to translate Spain language for example, then what 1st user changes in it will be visible to 2nd user too and he can correct it if there is a need.

This separate In-Portal installation will be upgrade to last stable version and this way any new phrases/e-mail events would be available for translation. Then at any time user who wants to get last Spanish language pack just would generate it real-time from last version of the translation.

--

Since we will upgrade this In-Portal installation used for language pack management there would be no way to support translations of language packs, used with older In-Portal versions. I think we can live with that, especially since user better do an upgrade to latest version, rather then upgrading language pack of their out-dated In-Portal installation.

On Sat, May 26, 2012 at 6:27 PM, Dmitry A. <dand...@gmail.com> wrote:

Good idea and thanks for clarifying, Alex.

DA

--

You received this message because you are subscribed to the Google Groups "In-Portal Localization Team" group.

To view this discussion on the web visit https://groups.google.com/d/msg/in-portal-local/-/cRGDCICS-8sJ.

To post to this group, send email to in-port...@googlegroups.com.
To unsubscribe from this group, send email to in-portal-loc...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/in-portal-local?hl=en.

[Alexander Obuhovich]

This is how translation progress page could look like  http://localize.drupal.org/translate/languages . I like the idea, how Drupal did this very much.