[Django] #23689: Django detects HTTP Accept-Language header in case-sensitive manner
Django
--------------------------------------+--------------------
Reporter: wayneye | Owner: nobody
Type: Bug | Status: new
Component: Internationalization | Version: 1.7
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 1 | UI/UX: 0
--------------------------------------+--------------------
This issue was originally discussed in django-developers:
https://groups.google.com/forum/#!topic/django-developers/1Y9LZSAOSnE
Per [http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.4 w3c],
[http://tools.ietf.org/html/rfc2616#page-104 rfc2616] and
[http://tools.ietf.org/html/bcp47 bcp47], Language tags should be parsed
in case-insensitive, however, I noticed that Django detects HTTP Accept-
Language headers in case-sensitive manner.
For example, the following headers:
{{{
Chrome: Accept-Language: zh-TW,zh;q=0.8,en-US;q=0.6,en;q=0.4
Firefox: Accept-Language: zh-tw,zh;q=0.8,en-us;q=0.5,en;q=0.3
}}}
Django will correctly display Traditional Chinese for Chrome, but won't
for Firefox because of lower-cased TW.
The fix contains two parts:
1. Fix potential case-sensitive places in code to follow case-insensitive
(for example parse_accept_lang_header())
2. Fix [https://docs.djangoproject.com/en/dev/topics/i18n/#definitions
documentation], correct the sentence "Browsers send the names of the
languages they accept in the Accept-Language HTTP header using this
format. Examples: it, de-at, es, pt-br. Both the language and the country
parts are in lower case. ", which obviously incorrect, Chrome uses tags
like zh-TW, pt-BR.
--
Ticket URL: <https://code.djangoproject.com/ticket/23689>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: wayneye | Owner: nobody
Type: Bug | Status: new
Internationalization | Resolution:
Severity: Normal | Triage Stage:
Keywords: | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
Changes (by claudep):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
Case sensibility should have been resolved by
2bab9d6d9ea095c4bcaeede2df576708afd46291
I have done some local tests and couldn't reproduce your issue. Having a
failing test case would help.
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:1>
Django
Reporter: wayneye | Owner: nobody
Type: Bug | Status: closed
Component: | Version: 1.7
Internationalization | Resolution: needsinfo
Severity: Normal | Triage Stage:
Keywords: | Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 1 | UI/UX: 0
Changes (by carljm):
* status: new => closed
* resolution: => needsinfo
Comment:
Closing as needsinfo.
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:2>
Django
Reporter: wayneye | Owner: nobody
Type: Bug | Status: closed
Component: | Version: 1.7
Internationalization |
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 1 | UI/UX: 0
Comment (by Daniel Samuels):
I've just hit this same problem today, here's some example code:
views.py:
{{{#!python
class ExampleView(TemplateView):
template_name = 'example.html'
def get_context_data(self, **kwargs: Any) -> Dict[str, Any]:
context = super().get_context_data(**kwargs)
context['language_code'] = translation.get_language()
# ^-- should be pt-BR, but is pt
return context
}}}
test_views.py
{{{#!python
def test_example_view(db, client):
language_code = 'pt-BR'
resp = client.get(reverse('example'),
HTTP_ACCEPT_LANGUAGE=language_code)
assert resp.context_data['language_code'] == language_code
# ^-- AssertionError: pt-BR != pt
}}}
The code path that's going wrong is:
* `LocaleMiddleware.process_request` calls
`translation.get_language_from_request`
* `get_language_from_request` calls `parse_accept_lang_header` which turns
`pt-BR` ito `pt-br`
* `get_language_from_request` then calls `get_supported_language_variant`,
passing `pt-br` as the `lang_code`
* `get_supported_language_variant` then runs `if code in
supported_lang_codes`, which is `False` (note that `'pt-BR' in
supported_lang_codes == True`)
* `get_supported_language_variant` then returns the fallback lang_code
`pt`
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:3>
Django
Reporter: wayneye | Owner: nobody
Type: Bug | Status: new
Internationalization |
Severity: Normal | Resolution:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 1 | UI/UX: 0
Changes (by Daniel Samuels):
* status: closed => new
* resolution: needsinfo =>
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:4>
Django
Reporter: wayneye | Owner: nobody
Type: Bug | Status: new
Internationalization |
Severity: Normal | Resolution:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 1 | UI/UX: 0
Comment (by Daniel Samuels):
I've created a simple reproduction using the above example code as a
basis. You can find it here: https://github.com/danielsamuels/django_23689
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:5>
Django
Reporter: wayneye | Owner: zainab-
| amir
Type: Bug | Status: assigned
Component: | Version: 1.7
Internationalization |
Severity: Normal | Resolution:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Easy pickings: 1 | UI/UX: 0
Changes (by zainab-amir):
* owner: nobody => zainab-amir
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:6>
Django
Reporter: wayneye | Owner: zainab-
| amir
Type: Bug | Status: assigned
Component: | Version: 1.7
Internationalization |
Severity: Normal | Resolution:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):
* cc: Claude Paroz (added)
* easy: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:7>
Django
Reporter: wayneye | Owner: Zainab
| Amir
Type: Bug | Status: assigned
Component: | Version: 1.7
Internationalization |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Unreviewed => Accepted
Comment:
Thanks for the test project!
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:8>
Django
Reporter: wayneye | Owner: Zainab
| Amir
Type: Bug | Status: assigned
Component: | Version: 1.7
Internationalization |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Zainab Amir):
Created a fix for it on my branch:
https://github.com/django/django/compare/main...zainab-amir:ticket_23689
I will create a PR on django if no one has any suggestions or comments.
1. The documentation already mentions that the header should be case
sensitive, so I fixed that
2. Also noticed that running django server with these settings throw an
error as mentioned below. This is also fixed.
{{{
LANGUAGES = (
('EN-US', 'English (US)'),
('De', 'Deutsche'),
('ar', 'عربى'),
)
LANGUAGE_CODE = 'en-us'
}}}
ERROR:
{{{
ERRORS:
?: (translation.E004) You have provided a value for the LANGUAGE_CODE
setting that is not in the LANGUAGES setting.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:9>
Django
Reporter: wayneye | Owner: Zainab
| Amir
Type: Bug | Status: assigned
Internationalization |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* version: 1.7 => 4.0
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:10>
Django
Reporter: wayneye | Owner: Zainab
| Amir
Type: Bug | Status: assigned
Component: | Version: 4.0
Internationalization |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak):
> I will create a PR on django if no one has any suggestions or comments.
You don't need to wait for an approval. PR is the right place for
suggestions and comments.
> Also noticed that running django server with these settings throw an
error as mentioned below. This is also fixed.
I don't see anything to fix here 🤔
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:11>
Django
Reporter: wayneye | Owner: Zainab
| Amir
Type: Bug | Status: assigned
Component: | Version: 4.0
Internationalization |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Zainab Amir):
Replying to [comment:11 Mariusz Felisiak]:
> > Also noticed that running django server with these settings throw an
error as mentioned below. This is also fixed.
>
> I don't see anything to fix here 🤔
This is the ripple effect of fixing the header parsing to be case
insensitive.
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:12>
Django
Reporter: wayneye | Owner: Zainab
| Amir
Type: Bug | Status: assigned
Component: | Version: 4.0
Internationalization |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Daniel Samuels (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:13>
Django
Reporter: wayneye | Owner: Zainab
| Amir
Type: Bug | Status: assigned
Component: | Version: 4.0
Internationalization |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* has_patch: 0 => 1
Comment:
[https://github.com/django/django/pull/15774 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:14>
Django
Reporter: wayneye | Owner: Zainab
| Amir
Type: Bug | Status: assigned
Component: | Version: 4.0
Internationalization |
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:15>
Django
Reporter: wayneye | Owner: Zainab
| Amir
Component: | Version: 4.0
Internationalization |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"901a1691982cab76349d33e51b72c40120ec927a" 901a1691]:
{{{
#!CommitTicketReference repository=""
revision="901a1691982cab76349d33e51b72c40120ec927a"
Fixed #23689 -- Made parsing HTTP Accept-Language header case-insensitive.
Thank you Daniel Samuels for test project.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:16>
Django
Reporter: wayneye | Owner: Zainab
| Amir
Type: Bug | Status: closed
Component: | Version: 4.0
Internationalization |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"f741dd5fab53e7db7c045ce392fbf99b9ee24d2a" f741dd5]:
{{{
#!CommitTicketReference repository=""
revision="f741dd5fab53e7db7c045ce392fbf99b9ee24d2a"
[4.1.x] Fixed #23689 -- Made parsing HTTP Accept-Language header case-
insensitive.
Thank you Daniel Samuels for test project.
Backport of 901a1691982cab76349d33e51b72c40120ec927a from main
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/23689#comment:17>