Manage multi language content on a nosql database

[Daniel Arita]

Hi,

One of the features of our site is the support for viewing content on three languages (English, Spanish and Portuguese). This not only means having labels being translated to another language, but also the content being stored on the database. We have been considering many approaches to achieve this and ended up considering two. For example considering one of the content are Events we have a model something like this:

class Event(BasicModel):

    name = ndb.StringProperty(required=True)

    description = ndb.StringProperty(required=True)

    privacy = ndb.StringProperty(choices=PRIVACY_CHOICES)

    start_datetime = ndb.DateTimeProperty()

The dependencies it handles are managed like this:

Where the speakers and resources are a repeated KeyProperty

Repeated Entities

With this approach we would store three records for each event the user creates, one for each language. So we can distiguish them we were thinking of setting the key ourselves, so if the event has an acronym like 'gio' we would append to the acronym the language it was created ( 'gio-en', 'gio-es', 'gio-pt' ) so searching for the event on a specific language becomes easier. So if we translated the cgi event to english and spanish we would have these two records:

1. gio-en
1. name = Google IO
2. description = Anual event hosted by Google
3. privacy = public
4. start_datetime = 2015-05-23
2. gio-es
1. name = Google IO
2. description = Evento anual realizado por Google
3. privacy = public
4. start_datetime = 2015-05-23

Advantages

• Simpler to control.
• More scalable when it comes to translate for a new language

 Disadvantages

• Repeated Data, aside from having three records for everything the data that won't be translated will be repeated.
• Manage dependencies with the keyproperties becomes more difficult if the child entity needs to be translated too.

Strutctured Properties

This approach considers a significant change to the properties that are going to be translated, changing them to a structured property. For example if the name is going to be translated it would change from a single StringProperty to a StructuredProperty with this fields

• en
• es
• pt

So in each field it would contain the translated name, and when trying to access the appropiate translation we could just use 'name.english'. This would only leave a single record for each event:

1. gio
1. name
1. en= Google IO
2. es= Google IO
3. pt =
2. descrtiption
1. en= Anual Event hosted by Google
2. es= Evento anual realizado por Google
3. pt=
3. privacy= public
4. start_datetime = 2015-05-23

Advantages

• More structured.
• All the data is on one record.

Disadvantages

• Adds an extra level for all entities
• Manage new languages becomes more complicated

We would like to know what can be the best approach for this, or if is better an combined implementation of the both or is there another approach we could take?

Thanks in advance for thehelp.