[Django] #20530: Incorrect QUERY_STRING handling on Python 3
Django
-----------------------------+--------------------
Reporter: mitsuhiko | Owner: nobody
Type: Bug | Status: new
Component: Core (URLs) | Version: 1.5
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------+--------------------
Certain browsers (IE cough) will not fully encode the path in the request
in all situations. As such you will encounter non ascii letters in the
request line. Currently the QueryDict does not handle that properly. In
addition to that it also means that the WSGI `QUERY_STRING` variable needs
to be handled the same way as `PATH_INFO` and `SCRIPT_NAME`.
Here is what is necessary to handle the case properly:
1. the `environ['QUERY_STRING']` attribute needs to go through the PEP
3333 dance on Python 3 that creates a bytes object
2. unquoting happens on the bytes
3. finally everything is done to the intended encoding (UTF-8)
The logic currently employed by QueryDict in combination with the
WSGIRequest object is double wrong:
1. the WSGIRequest object is not properly doing the dance and passes a
(potentially mangled) unicode string to query dict
2. the query dict decodes that incorrectly formatted unicode string (WSGI
on 3.x intentionally incorrectly encodes information) causing invalid data
to show up in request.args
Independently of that if bytes are passed to the QueryDict it does not do
proper decoding unless the bytes are a subset of ASCII.
--
Ticket URL: <https://code.djangoproject.com/ticket/20530>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: mitsuhiko | Owner: aaugustin
Type: Bug | Status: assigned
Component: Core (URLs) | Version: 1.5
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by aaugustin):
* status: new => assigned
* needs_better_patch: => 0
* needs_tests: => 0
* owner: nobody => aaugustin
* needs_docs: => 0
* stage: Unreviewed => Accepted
Comment:
Thanks for the report. I'll take care of that.
--
Ticket URL: <https://code.djangoproject.com/ticket/20530#comment:1>
Django
Reporter: mitsuhiko | Owner: aaugustin
Type: Bug | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by aaugustin):
Item 1 above was fixed by
https://github.com/django/django/commit/8aaca651cf5732bbf395d24a7d9f2edfab00250c#L0L136
--
Ticket URL: <https://code.djangoproject.com/ticket/20530#comment:2>
Django
Reporter: mitsuhiko | Owner: aaugustin
Type: Bug | Status: assigned
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Aymeric Augustin <aymeric.augustin@…>):
In [changeset:"7bb627936034c1b9500a8d250cce75b30f980b23"]:
{{{
#!CommitTicketReference repository=""
revision="7bb627936034c1b9500a8d250cce75b30f980b23"
Fixed an encoding issue in the test client.
Fixed
comment_tests.tests.test_comment_view.CommentViewTests.testCommentPostRedirectWithInvalidIntegerPK.
Refs #20530.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/20530#comment:3>
Django
Reporter: mitsuhiko | Owner: aaugustin
Component: Core (URLs) | Version: 1.5
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Aymeric Augustin <aymeric.augustin@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"65b6eff322a4a3331601e111934dee95c090961c"]:
{{{
#!CommitTicketReference repository=""
revision="65b6eff322a4a3331601e111934dee95c090961c"
Fixed #20530 -- Properly decoded non-ASCII query strings on Python 3.
Thanks mitsuhiko for the report.
This commit just adds a test since the problem was fixed in 8aaca651.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/20530#comment:4>
Django
Reporter: mitsuhiko | Owner: aaugustin
Type: Bug | Status: closed
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Aymeric Augustin <aymeric.augustin@…>):
In [changeset:"9244447cc4a91c22f8f2668f9667e92a1b2de958"]:
{{{
#!CommitTicketReference repository=""
revision="9244447cc4a91c22f8f2668f9667e92a1b2de958"
[1.6.x] Fixed an encoding issue in the test client.
Refs #20530.
Backport of 7bb62793 and 476b0764 from master.
Conflicts:
django/test/client.py
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/20530#comment:5>
Django
Reporter: mitsuhiko | Owner: aaugustin
Type: Bug | Status: closed
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Aymeric Augustin <aymeric.augustin@…>):
In [changeset:"7fcd6aa6695b39370154d6993cdbb3ba4363de91"]:
{{{
#!CommitTicketReference repository=""
revision="7fcd6aa6695b39370154d6993cdbb3ba4363de91"
[1.6.x] Fixed #20530 -- Properly decoded non-ASCII query strings on Python
3.
Thanks mitsuhiko for the report.
Backport of 65b6eff3 and adaptation of 8aaca65 from master.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/20530#comment:6>
Django
Reporter: mitsuhiko | Owner: aaugustin
Type: Bug | Status: closed
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Aymeric Augustin <aymeric.augustin@…>):
In [changeset:"63b95ca452ea7ef1103e599f8dd733b67278c8dc"]:
{{{
#!CommitTicketReference repository=""
revision="63b95ca452ea7ef1103e599f8dd733b67278c8dc"
[1.6.x] Fixed 9244447c -- incomplete backport.
The test client had been refactored in the mean time. This commit
de-factors the fix. Refs #20530.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/20530#comment:7>
Django
Reporter: Armin Ronacher | Owner: Aymeric
| Augustin
Type: Bug | Status: closed
Component: Core (URLs) | Version: 1.5
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham <timograham@…>):
In [changeset:"4206765a5b157bd6149d9e1aaf459fd4296e9b89" 4206765]:
{{{
#!CommitTicketReference repository=""
revision="4206765a5b157bd6149d9e1aaf459fd4296e9b89"
Refs #20530 -- Added a test for test client query string encoding.
7bb627936034c1b9500a8d250cce75b30f980b23 is only tested in
django-contrib-comments.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/20530#comment:8>