[Django] #33078: Internationalisation didn't support language locale containing both script and region.
Django
and region.
-----------------------------------------+------------------------
Reporter: A-hông | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 3.2
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------
The {{{i18n_patterns}}} didn't work with locale contains both script and
region, like {{{en-latn-us}}}.
Given {{{settings.py}}}
{{{
LANGUAGE_CODE = 'en-us'
LANGUAGES = [
('en-us', "English"),
('en-latn-us', "Latin English"),
('en-Latn-US', "BCP 47 case format"),
]
}}}
{{{urls.py}}}
{{{
from django.conf.urls.i18n import i18n_patterns
from django.http import HttpResponse
def bangiah(request):
return HttpResponse('U!')
urlpatterns += i18n_patterns(
path('', bangiah),
)
}}}
- The response of {{{http://localhost:8000/en-us/}}} is 200 {{{U!}}}.
- The response of {{{http://localhost:8000/en-lat-us/}}} is 404 not found.
- The response of {{{http://localhost:8000/en-Latn-US/}}} is 404 not
found.
**Steps to Reproduce**
1. Start a new project with {{{django-admin startproject tshi}}} and {{{
cd tshi/}}}
2. Append to {{{tshi/settings.py}}} as follows
{{{
LANGUAGES = [
('en-us', "English"),
('en-latn-us', "Latin English"),
('en-Latn-US', "BCP 47 case format"),
]
MIDDLEWARE += [
'django.middleware.locale.LocaleMiddleware',
]
}}}
3. Edit {{{tshi/urls.py}}} by appending follows
{{{
from django.conf.urls.i18n import i18n_patterns
from django.http import HttpResponse
def bangiah(request):
return HttpResponse('U!')
urlpatterns += i18n_patterns(
path('', bangiah),
)
}}}
4. {{{python manage.py migrate}}}
5. {{{python manage.py runserver}}}
6. The results
- The response of {{{http://localhost:8000/en-us/}}} is 200 {{{U!}}}.
- The response of {{{http://localhost:8000/en-lat-us/}}} is 404 not found.
- The response of {{{http://localhost:8000/en-Latn-US/}}} is 404 not
found.
** Expect to happen instead**
The response of {{{http://localhost:8000/en-latn-us/}}} and
{{{http://localhost:8000/en-Latn-US/}}} should be 200 {{{U!}}}.
The {{{en-Latn-US}}} tag follows format defined in [https://www.rfc-
editor.org/info/rfc5646 RFC 5646]. It's
[https://docs.djangoproject.com/en/dev/topics/i18n/#term-language-code
documented] that the language part is always in lowercase, following
[https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Language
Accept-Language]. [https://developer.mozilla.org/en-
US/docs/Web/HTTP/Headers/Accept-Language Accept-Language] is following
[https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-
Language Content-Language Header], which is following [https://www.rfc-
editor.org/info/rfc5646 RFC 5646]. The [https://www.rfc-
editor.org/info/rfc5646 RFC 5646] defined {{{langtag}}} as follow:
{{{
langtag = language
["-" script]
["-" region]
*("-" variant)
*("-" extension)
["-" privateuse]
language = 2*3ALPHA ; shortest ISO 639 code
["-" extlang] ; sometimes followed by
; extended language subtags
/ 4ALPHA ; or reserved for future use
/ 5*8ALPHA ; or registered language subtag
extlang = 3ALPHA ; selected ISO 639 codes
*2("-" 3ALPHA) ; permanently reserved
script = 4ALPHA ; ISO 15924 code
region = 2ALPHA ; ISO 3166-1 code
/ 3DIGIT ; UN M.49 code
}}}
I have confirmed that this issue can be reproduced as described on a fresh
Django project
* Python version: 3.7.5
* Django version: 3.2.7
--
Ticket URL: <https://code.djangoproject.com/ticket/33078>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
and region.
Reporter: A-hông | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 3.2
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Claude Paroz):
What's the use case of using `en-latn-us`?
--
Ticket URL: <https://code.djangoproject.com/ticket/33078#comment:1>
Django
and region.
Reporter: A-hông | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 3.2
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by A-hông):
Replying to [comment:1 Claude Paroz]:
> What's the use case of using `en-latn-us`?
Our language, Taigi in Taiwan, is denoted by {{{nan}}} in ISO 639-3.
{{{nan}}} denoted other languages which are different from Taigi although
we are submitting the proposal for new language code this year. So that
the language code is added the region, {{{ nan-tw }}}. Taigi contains many
writing systems, like Latin, Hanji(Chinese Chars), a mixed written form
combining Latin and Hanji. The language code list of the website is {{{
nan-latn-tw }}} and {{{ nan-hani-tw }}} following the format {{{ lang-
script-region }}} in RFC 5646.
--
Ticket URL: <https://code.djangoproject.com/ticket/33078#comment:2>
Django
and region.
Reporter: A-hông | Owner: nobody
Type: New feature | Status: new
Component: Internationalization | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Claude Paroz):
* type: Uncategorized => New feature
* version: 3.2 => dev
* component: Uncategorized => Internationalization
* stage: Unreviewed => Accepted
Comment:
Thanks for the explanation. It makes much more sense now!
After a very quick glance, an idea could be to take `settings.LANGUAGES`
into account somehow in `get_language_from_path`. Extending the regex
might catch way too much non-language stuff.
--
Ticket URL: <https://code.djangoproject.com/ticket/33078#comment:3>
Django
and region.
Reporter: A-hông | Owner: Maxim
| Piskunov
Type: New feature | Status: assigned
Component: | Version: dev
Internationalization |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Maxim Piskunov):
* owner: nobody => Maxim Piskunov
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/33078#comment:4>
Django
and region.
Reporter: A-hông | Owner: Maxim
| Piskunov
Type: New feature | Status: assigned
Component: | Version: dev
Internationalization |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Maxim Piskunov):
* has_patch: 0 => 1
Comment:
Replying to [comment:3 Claude Paroz]:
> Thanks for the explanation. It makes much more sense now!
>
> After a very quick glance, an idea could be to take `settings.LANGUAGES`
into account somehow in `get_language_from_path`. Extending the regex
might catch way too much non-language stuff.
Hey. I was checking the code for get_language_from_path. Not sure how to
include `settings.LANGUAGES` in the flow there.
I made a PR with regex version.
[https://github.com/django/django/pull/15098 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/33078#comment:5>
Django
and region.
Reporter: A-hông | Owner: Maxim
| Piskunov
Type: New feature | Status: assigned
Component: | Version: dev
Internationalization |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/33078#comment:6>
Django
and region.
Reporter: A-hông | Owner: Maxim
| Piskunov
Type: New feature | Status: assigned
Component: | Version: dev
Internationalization |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Maxim Piskunov):
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/33078#comment:7>
Django
and region.
Reporter: A-hông | Owner: Maxim
| Piskunov
Type: New feature | Status: assigned
Component: | Version: dev
Internationalization |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* needs_tests: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/33078#comment:8>
Django
and region.
Reporter: A-hông | Owner: Maxim
| Piskunov
Type: New feature | Status: assigned
Component: | Version: dev
Internationalization |
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* needs_tests: 1 => 0
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/33078#comment:9>
Django
and region.
Reporter: A-hông | Owner: Maxim
| Piskunov
Component: | Version: dev
Internationalization |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak <felisiak.mariusz@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"d3f4c2b95d2a13a5d9bc0e6413dfdbab21388822" d3f4c2b9]:
{{{
#!CommitTicketReference repository=""
revision="d3f4c2b95d2a13a5d9bc0e6413dfdbab21388822"
Fixed #33078 -- Added support for language regions in i18n_patterns().
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33078#comment:11>
Django
and region.
Reporter: A-hông | Owner: Maxim
| Piskunov
Component: | Version: dev
Internationalization |
Severity: Normal | Resolution:
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"4f7bbc61386ec89bf664905609ae6e93e41bd6bf" 4f7bbc61]:
{{{
#!CommitTicketReference repository=""
revision="4f7bbc61386ec89bf664905609ae6e93e41bd6bf"
Refs #33078 -- Added extra assertions to
MiscTests.test_get_language_from_path_real().
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33078#comment:10>