Filter plugin v. 1.13-1.45 breaks backward compatibility for ID-bound alternative translations

[Manuel Souto Pico]

Dear all,

I have reported the problem in ticket #273 and below follows a summary and some questions.

Summary: When we use an Okapi filter in OmegaT, we can bind alternative/ICE matches to the segments key/id or resname, which is (supposed to be) more robust than relying on the actual co-text (previous and next segments).

For example, in the screenshot below you can see the second segment has an alternative translation bound to ID NFDBB2FA9-tu2_0:

The problem is that versions of the filter plugin up to 1.12-1.44 produced a different ID from the one produced by filter plugin versions starting at 1.13-1.45.

What this means in practice is that we (in my team) cannot upgrade to the latest version of the filters plugin (at least not until we are 100% sure that any project using the OpenXML filter is closed, and it might not be easy to be 100% sure). I'm assuming the issue is likely to happen with other Okapi filters, such as HTML, etc. but haven't tested it.

I don't maintain this plugin for myself, but for the entire team in my company (including a large base of freelancers). This issue, together with other similar ones like #272 that break backward compatibility of translations, make me grow increasingly wary of using Okapi filters in OmegaT. I fear translations might get broken with any new version of the plugin.

I wonder why this happens. Is it because Okpai developers don't have enough information about what filter/plugin's features users rely on? Is it perhaps because the Okapi team is not aware of the OmegaT code that interacts with the plugin (or the Okapi filter)? 

What matters most now: Can the problem be fixed? 

What matters for later: Would it be possible to ensure that this kind of issue is avoided (perhaps by means of thorough unit tests changes)? Is there any way that I or OmegaT devs can help prevent this kind of issue going forward?

Thanks.
Cheers, Manuel

[Manuel Souto Pico]

Dear all, 

I have updated the ticket with more info (reproduced here) that hopefully helps understand why this is a critical issue.

We've seen the impact of this issue in two different scenarios:

• An OmegaT project where auto-propagation is disabled, which means that all translations are ICE (or alternative) translations, bound to the segment ID. If the Okapi filter version used to produce the translations and their segment IDs is different from the Okapi filter version used by other people working on the same project later on (a reviser, the project manager, etc) or some programmatic solution, then all the translations will be missing. The problem in this case can be noticed and resolved by harmonizing the Okapi filter versions for all users (which is annoying but feasible).

• An OmegaT project where auto-propagation is enabled (far more common than the situation above), where alternative/ICE translations are normally created only when a default translation is not a good fit for a certain context. When the problem happens because two users have different versions of the Okapi filter, the match of ICE/alternative translations will be broken and those translations will not be used, however most likely a default translation exists and (since it matches based on source text only and not on ID) will used instead without anyone noticing (which is very DANGEROUS).

Cheers, Manuel