[Django] #16284: djangojs uses en as fallback language rather than projects language code
Django
---------------------------+-------------------------------
Reporter: anonymous | Owner: nobody
Type: Uncategorized | Status: new
Milestone: | Component: Uncategorized
Version: 1.3 | Severity: Normal
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Easy pickings: 0
UI/UX: 0 |
---------------------------+-------------------------------
In views/i18n.py there is code:
{{{
# first load all english languages files for defaults
for package in packages:
p = importlib.import_module(package)
path = os.path.join(os.path.dirname(p.__file__), 'locale')
paths.append(path)
try:
catalog = gettext_module.translation(domain, path, ['en'])
t.update(catalog._catalog)
except IOError:
pass
else:
# 'en' is the selected language and at least one of the
packages
# listed in `packages` has an 'en' catalog
if en_selected:
en_catalog_missing = False
}}}
I believe django should first check if LANGUAGE_CODE setting is not set
and use this for defaults. Our main language is polish and while we have
english translations available, we don't want django to fallback to
english.
--
Ticket URL: <https://code.djangoproject.com/ticket/16284>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: anonymous | Owner: aaugustin
Type: | Status: assigned
Cleanup/optimization | Component:
Milestone: | Internationalization
Version: 1.3 | Severity: Normal
Resolution: | Keywords:
Triage Stage: Design | Has patch: 0
decision needed | Needs tests: 0
Needs documentation: 0 | Easy pickings: 0
Patch needs improvement: 0 |
UI/UX: 0 |
-------------------------------------+-------------------------------------
Changes (by aaugustin):
* status: new => assigned
* needs_better_patch: => 0
* component: Uncategorized => Internationalization
* needs_tests: => 0
* owner: nobody => aaugustin
* needs_docs: => 0
* type: Uncategorized => Cleanup/optimization
* stage: Unreviewed => Design decision needed
Comment:
Indeed, it's weird to have a special case for English.
This dates back to r1560. Unfortunately, the commit message is not
explicit, and it doesn't reference a ticket.
That code was touched several times recently (r14901, r15404, r15441).
#13726 is a good entry point, but I still don't understand very well
what's going on there.
I'll mark this ticket as DDN, because I think we need to check if the
special case is useful, or if it's just an historic artifact.
----
While investigating this ticket, I noticed there is another explicit
reference to English as the "untranslated language" in
`django.core.management.base.execute()`. The comment dates back to #5898,
which was a refactoring, so it may be even older. It would be nice to
check what's happening there too, and add an helpful comment for future
developers :)
--
Ticket URL: <https://code.djangoproject.com/ticket/16284#comment:1>
Django
Reporter: anonymous | Owner: nobody
Type: | Status: new
Milestone: | Internationalization
Version: 1.3 | Severity: Normal
Resolution: | Keywords:
Triage Stage: Design | Has patch: 0
decision needed | Needs tests: 0
Needs documentation: 0 | Easy pickings: 0
Patch needs improvement: 0 |
UI/UX: 0 |
-------------------------------------+-------------------------------------
Changes (by aaugustin):
* owner: aaugustin => nobody
* status: assigned => new
--
Ticket URL: <https://code.djangoproject.com/ticket/16284#comment:2>
Django
Reporter: anonymous | Owner: nobody
Type: | Status: new
Cleanup/optimization | Version: 1.3
Component: | Resolution:
Internationalization | Triage Stage: Design
Severity: Normal | decision needed
Keywords: | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by jeroen.pulles@…):
This piece of code also causes an annoying bug, as I see it:
I have a project with ''nl'' as default language, the source strings are
in ''en_US'' and there are ''en_GB'' translations. As a result this piece
of code has "en_selected" but no catalog named "en" (it's named en_GB,
obviously); Meaning it ends up with an empty translations dictionary for
no good reason.
To my mind, this should be as simple as ''got language code, got catalog
with that language code, i'm done.''
--
Ticket URL: <https://code.djangoproject.com/ticket/16284#comment:3>
Django
Reporter: anonymous | Owner: nobody
Type: | Status: new
Cleanup/optimization | Version: 1.3
Component: | Resolution:
Internationalization | Triage Stage: Design
Severity: Normal | decision needed
Keywords: | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by ramiro):
See (in that order and with their associated commits): #3594, #13388,
#13726 for the rationale of the treatment and to help you research if it's
possible to remove the special treatment 'en' is given in the JS i18n
code. See also this comment:
https://code.djangoproject.com/ticket/13726#comment:3
--
Ticket URL: <https://code.djangoproject.com/ticket/16284#comment:4>
Django
Reporter: anonymous | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Internationalization | Version: 1.3
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by aaugustin):
* stage: Design decision needed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/16284#comment:5>
Django
Reporter: anonymous | Owner: KJ
Type: Cleanup/optimization | Status: assigned
Component: Internationalization | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by KJ):
* owner: nobody => KJ
* status: new => assigned
* has_patch: 0 => 1
* version: 1.3 => master
Comment:
I’ve created a patch: https://github.com/django/django/pull/1338 . There
are two things it removes:
1. Loading of English translation. As there is a priori no reason to treat
English differently than other languages, this is a no-brainer.
2. Loading of language specified in LANGUAGE_CODE setting. This has to be
done because there is no way to determine what language we’re translating
from. For example, if we have source texts written in Polish and
LANGUAGE_CODE is set to “es”, we want to get no translations for users
with language set to “pl”. If Spanish translation was present, this would
be impossible with current solution.
The patch makes the javascript_catalog function simpler and more
consistent with gettext behaviour.
The present behaviour had one test which I had to remove. Fortunately it
doesn’t seem to be documented, so probably we don’t have to worry that
much about backwards compatibility (although a small release note may be
worth considering).
--
Ticket URL: <https://code.djangoproject.com/ticket/16284#comment:6>
Django
Reporter: anonymous | Owner: KJ
Type: Cleanup/optimization | Status: assigned
Component: Internationalization | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by ramiro):
Replying to [comment:6 KJ]:
> 1. Loading of English translation. As there is a priori no reason to
treat English differently than other languages, this is a no-brainer.
We can discuss about this. I think it might be possible to give no special
treatment to English in JS i18n because it's not tied to the GNU gettext
runtime translation machinery (through the Python gettext stdlib module)
that is the one known to have some en-us preference hardcoded behavior.
Replying to [comment:6 KJ]:
> 2. Loading of language specified in LANGUAGE_CODE setting. This has to
be done because there is no way to determine what language we’re
translating from. For example, if we have source texts written in Polish
and LANGUAGE_CODE is set to “es”, we want to get no translations for users
with language set to “pl”. If Spanish translation was present, this would
be impossible with current solution.
Well, that would change the behavior. By definition LANGUAGE_CODE is the
language to which runtime translation fallbacks when no translation is
available for a given literal in the user-preferred language. Also, it's
the way to dictate which global translation to use when no per-user
preference language detection mechanism is in effect (localmiddleware). As
you've well described this LANGUAGE_CODE can be different from the
language of the Python/JS source code files/templates.
We need to find a way to cover your use case (fallback to source code
language without taking in account LANGUAGE_CODE) but without breaking
existing use and deployments.
--
Ticket URL: <https://code.djangoproject.com/ticket/16284#comment:7>
Django
Reporter: anonymous | Owner: KJ
Type: Cleanup/optimization | Status: assigned
Component: Internationalization | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by KJ):
* needs_better_patch: 0 => 1
Comment:
Replying to [comment:7 ramiro]:
> Replying to [comment:6 KJ]:
> > 1. Loading of English translation. As there is a priori no reason to
treat English differently than other languages, this is a no-brainer.
>
> We can discuss about this. I think it might be possible to give no
special treatment to English in JS i18n because it's not tied to the GNU
gettext runtime translation machinery (through the Python gettext stdlib
module) that is the one known to have some en-us preference hardcoded
behavior.
I didn’t know about that. I have done some googling and grepping in
gettext’s source code for that en-us-centric behavior and found nothing,
which may mean that it is not a feature that gettext is very proud of.
> Replying to [comment:6 KJ]:
> > 2. Loading of language specified in LANGUAGE_CODE setting. This has to
be done because there is no way to determine what language we’re
translating from. For example, if we have source texts written in Polish
and LANGUAGE_CODE is set to “es”, we want to get no translations for users
with language set to “pl”. If Spanish translation was present, this would
be impossible with current solution.
>
> Well, that would change the behavior. By definition LANGUAGE_CODE is the
language to which runtime translation fallbacks when no translation is
available for a given literal in the user-preferred language. Also, it's
the way to dictate which global translation to use when no per-user
preference language detection mechanism is in effect (localmiddleware). As
you've well described this LANGUAGE_CODE can be different from the
language of the Python/JS source code files/templates.
>
> We need to find a way to cover your use case (fallback to source code
language without taking in account LANGUAGE_CODE) but without breaking
existing use and deployments.
One way would be to provide more flexible way of specifying language
preference (like a setting which points to a function which returns an
iterable of language codes, in order of preference). However this would
still leave some potential holes, for example if:
- App A is written in English and translated into Polish.
- App B is written in Polish and translated into English.
, then I see no way to make a satisfactory global language preference
order. In this case it would be better to introduce a setting which allows
programmer to specify “source code natural language”, but this setting
should be assigned per app, which is currently not possible in an elegant
way. So I propose to cut my pull request to only removing English-centric
behavior for now, as the second problem doesn’t seem to be an easy one to
handle.
--
Ticket URL: <https://code.djangoproject.com/ticket/16284#comment:8>
Django
Reporter: anonymous | Owner: KJ
Type: Cleanup/optimization | Status: assigned
Component: Internationalization | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Comment (by ramiro):
Replying to [comment:8 KJ]:
> Replying to [comment:7 ramiro]:
> > Replying to [comment:6 KJ]:
> > > 1. Loading of English translation. As there is a priori no reason to
treat English differently than other languages, this is a no-brainer.
> >
> > We can discuss about this. I think it might be possible to give no
special treatment to English in JS i18n because it's not tied to the GNU
gettext runtime translation machinery (through the Python gettext stdlib
module) that is the one known to have some en-us preference hardcoded
behavior.
>
> I didn’t know about that. I have done some googling and grepping in
gettext’s source code for that en-us-centric behavior and found nothing,
which may mean that it is not a feature that gettext is very proud of.
I'll need to search for this myself too. I rememeber reading about it...
And here it is, it was in this same issue tracker: #8626
--
Ticket URL: <https://code.djangoproject.com/ticket/16284#comment:9>
Django
Reporter: anonymous | Owner: KJ
Type: Cleanup/optimization | Status: assigned
Component: Internationalization | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Comment (by KJ):
I’ve started [https://groups.google.com/forum/#!topic/django-
developers/1X_tPbhG_NQ a thread on django-developers]. The thread contains
solution proposal.
--
Ticket URL: <https://code.djangoproject.com/ticket/16284#comment:10>
Django
Reporter: anonymous | Owner: KJ
Component: Internationalization | Version: master
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Changes (by claudep):
* status: assigned => closed
* resolution: => fixed
Comment:
With commit [11c60b529816c5d29c], Django master (1.10) should treat JS
catalogs like other translations. I hope this fixes the concerns of this
ticket. If not, please reopen.
--
Ticket URL: <https://code.djangoproject.com/ticket/16284#comment:11>