Internationalization of Arches

[ape...@fargeo.com]

Hello Arches developers.  We’ve heard from a few members of the community that they would like Arches to support multiple languages so that their users can interact with resource information in languages other than English.

Generally, localization of an application simply involves translating the user interface. In Arches this is complicated by the fact that the application contains a mixture of both static strings as well as dynamically created strings (such as those created as part of model and card configuration).  Translating static strings is accomplished by using standard .po files and well understood text replacement logic.  Translating dynamically defined strings is a bit trickier.

Adding to the complexity, is the fact that users might want to edit resource instances in multiple languages, which is another kind of dynamic string that needs to be localizable.  Finally, RTL (right-to-left) languages need to be considered (eg: Arabic, Hebrew, etc...), which is another non-trivial task.

We’ve spent a bit of time identifying (at a high level) portions of the Arches code that would need to be updated to support multiple languages.  These include:

> All static labels in the UI

> Arches Designer (Dynamic labels and UI)

    - nodes

    - cards

    - widgets

    - help files

    - String data types (e.g.: string, domain, boolean)

> Data import/export

    - graph import

        - any localized elements should be importable

        - all language codes should be preserved (even those not currently in the system)

        - old style graphs should be importable (backward compatibility)

    - graph export

        - exported graph should include all translations of all localized elements

> Arches Data

    - all different localized strings need to reference the same resource instance (for search)

    - users would like to see if there is data in alternate languages (need api)

    - backward compatibility?

        - data migration from old datatype

        - update primary descriptor function

            - refactor to be an external overridable function

        - update/add unit tests

        - jsonld import

            - use lang attribute on json nodes

        - jsonld export

    - search

        - remove term index and port to resource index (compatible with new RDM)

            - use search-as-you-type datatype for string data

            - add “languageid” to strings property in resource index

            - update term search filter component

        - add language filter search component (within new “advanced filters” expandable section)

> RTL integration

    - confirm images make sense in RTL

    - html templates need to be confirmed

    - confirrm 3rd party modules (eg: dropzone, chosen, select2, moment, etc...) work in RTL

    - absolutely positioned elements in UI need to be updated

    - css floats

> Documentation/Testing

    - documentation of how to implement internationalization for users and developers

    - general testing/QA

        - QA

        - testing

> When considering possible technical solutions it would be nice if they had the following attributes:

    - Minimize the use of database migrations

    - Be backward compatible with existing Arches instances and datafacts like exported graphs and datafiles(csv, json, etc..)

    - Minimize the impact on any custom components that users have already created

    - Be able to easily add or remove languages

> We’ve started testing some possible technical approaches to address these issues, including:

    - Using well known django packages that address translating dyanmic strings

        - django-modeltranslation (https://django-modeltranslation.readthedocs.io/en/latest/index.html) 

            - relies on updating the database schema with additional columns suffixed with the language bidi

            - presents a single language to the UI

            - JSONField not supported by default - this would need to be patched

        - django-translated-fields (https://github.com/matthiask/django-translated-fields/)

            - relies on updating the database schema with additional columns suffixed with the language bidi

            - presents a single language to the UI

    - Creating our own system of translating dynamic strings

        - We've already tested a system of using a single JSONField to replace the dyanmic field

            - Stores localized data in a single json object keyed with the languages bidi

            - Can present the json object to the UI or can be configured to present a single language to the UI 

            - Database migrations might not be reqired when languages are added

However, we would really like to open the discussion of how to implement support for multiple languages up to the full developer community.  Please let us know how you have dealt with internationalization in other projects, pros and cons to specific patterns, and libraries that you feel might help in supporting mutlitple languages in Arches.  We are also very interested in hearing how you have solved the data representation, import/export, and indexing issues in your work.

Finally, we would definitely benefit from specfic use cases for multi-language Arches applcations.  If you have real-world requirements for multiple languages, please share them with us!

Thanks in advance for any input that you might have in supporting multiple languages in Arches.

[Phil Weir]

Hi Alexei,

Very much interested here. We have been doing a project over the last year with researchers in Saudi Arabia with one of our own SaaS products, which has involved full implementation of Arabic support. As we have monolingual schoolstudents as part of the test group, this has set a high bar for clarity and completeness. To facilitate this, we have been running a dedicated Weblate instance for non-technical translation, and have the translator tools linked into continuous integration, to streamline iterative improvement and allow us to transform output as necessary for the different translation systems that we use in different parts of the stack. Originally, this was a single-language platform with little internationalization, so we have been adding that support progressively across the components.

This has been a fascinating process, and as you mention, third-party libraries have given us plenty to think about, but no complete blockers. We did have to move to Mapbox to get translatable maps (which fits with Arches nicely) and timeline support was actually fine, although we had to answer related UX questions in RTL - having local stakeholders was critical for this.

Given some of our areas of work, the ability to localize Arches would be very desirable, but our existing projects would not support a lot of our team's time on the initial internationalization piece - however, we are waiting to hear back on one larger project that might enable us to allocate some time, if there was a combined effort?

All the best,

Phil Weir

This email was sent from Flax & Teal Limited, a company registered in NI, with number NI617545

at Farset Labs, Weavers Court, Linfield Industrial Estate, Linfield Road, Belfast, BT12 5GH

Flax & Teal is a member of the Avata Industries business syndicate

--
You received this message because you are subscribed to the Google Groups "Arches Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to arches-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/arches-dev/63885f48-644a-44b2-a44f-672f319d4620n%40googlegroups.com.

[Alexei Peters]

Hi Phil,

Thank for letting us know about your interest and potential for involvement.  I hadn't heard of Weblate before but we have used Transifex in the past.  It'll be interesting to see how useful those products will be in the context of Arches, because so much of the site is dynamically driven.

Good to hear the Mapbox supports the localization efforts you encountered (and presumably ours as well).

Cheers,

Alexei

Director of Web Development - Farallon Geographics, Inc. - 971.227.3173

You received this message because you are subscribed to a topic in the Google Groups "Arches Development" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/arches-dev/0ma4YqjXfoQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to arches-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/arches-dev/CAB1PGKa9jaoOpqdT9w_%2BFzpQeYo9a4S4ymzE0i7GK7t42wEaoQ%40mail.gmail.com.

[ape...@fargeo.com]

I just posted a brief review (https://github.com/archesproject/arches/issues/6956#issuecomment-841495220) of 3 potential solutions to the problem of localizing dynamic data in arches.  I would love some feedback about the solutions and the ultimate verdict.  Any solution will have a fairly sizable impact on the core structure of how data in Arches is stored so it would be nice to get this right the first time.