[Question] jsi18n - get_javascript_catalog a bit obscure?

[Cristiano Coelho]

https://github.com/django/django/blob/master/django/views/i18n.py#L204

Can someone explain me why does it have to always load english as the first fallback?

Also, line 248:

 # If the currently selected language is English but it doesn't have a
 # translation catalog (presumably due to being the language translated
 # from) then a wrong language catalog might have been loaded in the
 # previous step. It needs to be discarded.

Looks unnecessary hard.

On a project, I had to monkey patch this method because I didn't add translations at all for my default language (not english) since the identifiers was enough, and for all the server side translations it worked fine. However, for javascript translations, since english was one of my translations, I would end up getting english translated strings when I was expecting the default identifier to be returned. So to fix it I just removed all the english and default languages loading and everything worked as expected. What would be the issues with this? Was there actually another approach?

Below is my slightly changed code of this method (I did it quite a few ago but surfaced again for some reason), I would like to know if it makes sense and the original code really has some obscure logic or I'm missing something important about it.

def get_javascript_catalog(locale, domain, packages):
    default_locale = to_locale(settings.LANGUAGE_CODE)
    app_configs = apps.get_app_configs()
    allowable_packages = set(app_config.name for app_config in app_configs)
    allowable_packages.add('django.conf')
    packages = [p for p in packages if p in allowable_packages]
    t = {}
    paths = []
 
    # paths of requested packages
    for package in packages:
        p = importlib.import_module(package)
        path = os.path.join(os.path.dirname(upath(p.__file__)), 'locale')
        paths.append(path)
    # add the filesystem paths listed in the LOCALE_PATHS setting
    paths.extend(list(reversed(settings.LOCALE_PATHS)))
 
       
    locale_t = {}
    for path in paths:
        try:
            catalog = gettext_module.translation(domain, path, [locale])
        except IOError:
            catalog = None
        if catalog is not None:
            locale_t.update(catalog._catalog)
 
    if locale_t:
        t = locale_t
    plural = None
    if '' in t:
        for l in t[''].split('\n'):
            if l.startswith('Plural-Forms:'):
                plural = l.split(':', 1)[1].strip()
    if plural is not None:
        # this should actually be a compiled function of a typical plural-form:
        # Plural-Forms: nplurals=3; plural=n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2;
        plural = [el.strip() for el in plural.split(';') if el.strip().startswith('plural=')][0].split('=', 1)[1]
 
    pdict = {}
    maxcnts = {}
    catalog = {}
    for k, v in t.items():
        if k == '':
            continue
        if isinstance(k, six.string_types):
            catalog[k] = v
        elif isinstance(k, tuple):
            msgid = k[0]
            cnt = k[1]
            maxcnts[msgid] = max(cnt, maxcnts.get(msgid, 0))
            pdict.setdefault(msgid, {})[cnt] = v
        else:
            raise TypeError(k)
    for k, v in pdict.items():
        catalog[k] = [v.get(i, '') for i in range(maxcnts[msgid] + 1)]
 
    return catalog, plural

[Cristiano Coelho]

Maybe I wasn't clear neither, but the main issue is this: when using a language equals to the default one, and if that language does not define any translation text (because ids are the same as values so it is not necessary), the server side translations will always correctly return the translated texts, while the javascript won't because it always has an english fallback.
Also the code I provided should actually load the default language translations as well (but not the english ones!) to make it behave exactly as the server side translations.

[Tim Graham]

Have you tried looking through history with git blame to find related tickets? Another tip is to search Google with a query like "javascript_catalog site:code.djangoproject.com". This will let you find tickets to see if the issue was raised before. This is how I try to answer questions like this since many of the original authors are no longer active (or at least, you can find the person who authored the code in question and try a more directed query like a ping in IRC).

[Cristiano Coelho]

Looking through git history seems like the "always load english translations" code is quite a few years old.

There's a 5 y.o ticket in here: https://code.djangoproject.com/ticket/16284

Which leads to here: https://code.djangoproject.com/ticket/3594 with a fix that adds the "discard if english not found" which doesn't resolve the issue completely if you actually have english translations but it is not the default language.

There's also a link to here: https://groups.google.com/forum/#!topic/django-developers/1X_tPbhG_NQ proposing a change with no replies at all.

I couldn't really understand why is it that the hardcoded english language can not be removed from here.

In my opinion the code should be slightly changed to only load the default language rather than english (and as long as the requested language is not the same as the default one) as fallback, so it can match the actual server side behaviour, which will load the configured language (at settings) as a fallback language as long as it is different from it (an from english, which makes sense because django text ids are all in english).
This change could however affect people that relies on always having an english translation as fallback when the configured default language is not english (does this even make sense? Would anyone do that?)

After this change is done, there could be an improvement (for both js and server side translations) that some people might find useful, rather than always falling back to the default configured language, you could have a map of fallbacks, for example, if the user requests a portuguese language, but you only have english (default) and spanish (secondary), it makes more sense to fallback to spanish rather than english, but if the user requests russian, it makes more sense to fallback to english.

[Cristiano Coelho]

Would a pull request be accepted? Does it need to be a new branch even if the change is just a few lines? Does it need an open ticket first?

[Tim Graham]

Sure, I mean... my impression as indicated by the lack of response to the thread is that no one provides much expertise in javascript_catalog() these days (although it's only been a couple days). I don't use it myself so I do my best to review proposed changes and then we wait for bug reports. Maybe this will be the start of your journey to being that expert.

Quoting https://github.com/django/django/blob/master/CONTRIBUTING.rst, "non-trivial pull requests (anything more than fixing a typo) without Trac tickets will be closed! Please file a ticket to suggest changes."

[Cristiano Coelho]

Where do I get an account for Trac tickets? :) I guess I should still open a ticket first

[Tim Graham]

Please read https://code.djangoproject.com/

[Tim Graham]

For posterity, ticket is https://code.djangoproject.com/ticket/26328.