[Django] #33697: Cleanup duplication of HTTP header parsing in utils.http and multipart parser.

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
------------------------------------------------+------------------------
Reporter: Carlton Gibson | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Utilities | Version: 4.0
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 1
UI/UX: 0 |
------------------------------------------------+------------------------
PY311 deprecated the `cgi` module.

We removed usage of that in [https://github.com/django/django/pull/15679
PR 15679].

There's similar logic already in place in `multipartparser.py`, but header
parameters are parsed slightly differently, and the return types of the
helper functions used don't match exactly.
([https://github.com/django/django/pull/15679#issuecomment-1123455222
Comment on the PR linking to the code locations].)

Nonetheless it looks like we should be able to combine the two functions
to have a single parsing routine.

--
Ticket URL: <https://code.djangoproject.com/ticket/33697>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.

--------------------------------------+------------------------------------

Reporter: Carlton Gibson | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Utilities | Version: 4.0

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by Mariusz Felisiak):

* stage: Unreviewed => Accepted

Comment:

It looks that the only difference between `_parseparam()` and
`django.http.multipartparser._parse_header_params()` is that it works on
strings (instead of bytes) and that it contains
https://github.com/python/cpython/commit/1ef0c0349e8fdb5415e21231cb42edbf232b742a
(which is desired in both places).

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:1>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
--------------------------------------+------------------------------------
Reporter: Carlton Gibson | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Utilities | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
--------------------------------------+------------------------------------

Comment (by Carlton Gibson):

+1 Yes. (And there's the list vs scalar thing... but that's just needing
to sit and think it through clearly.) — Ideal would be to unify on a
single version in `utils` and use in both places.

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:2>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
--------------------------------------+------------------------------------
Reporter: Carlton Gibson | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Utilities | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
--------------------------------------+------------------------------------

Comment (by Kapil Bansal):

Hi,
I want to give this issue a try. Although I am not sure how to run and
check what these functions are doing?
Would it be like creating a new django project and tests from where this
function is called? Or something else?

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:3>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
--------------------------------------+------------------------------------
Reporter: Carlton Gibson | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Utilities | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
--------------------------------------+------------------------------------

Comment (by Carlton Gibson):

Hi Kapil.

There's already some coverage, so making the change ensuring the existing
tests pass would be a start.

On implementation, you may find you need to adjust tests, or add new ones.
That's OK.

If we need extra tests after that, they can be suggested on review.

The [https://docs.djangoproject.com/en/dev/internals/contributing/writing-
code/submitting-patches/ Submitting patches] guide should help you on your
next steps.

Thanks!

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:4>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.

-------------------------------------+-------------------------------------
Reporter: Carlton Gibson | Owner: Mehrdad
Type: | Moradizadeh
Cleanup/optimization | Status: assigned

Component: Utilities | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by Mehrdad Moradizadeh):

* owner: nobody => Mehrdad Moradizadeh
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:5>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
-------------------------------------+-------------------------------------
Reporter: Carlton Gibson | Owner: Mehrdad
Type: | Moradizadeh
Cleanup/optimization | Status: assigned
Component: Utilities | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Carlton Gibson):

* has_patch: 0 => 1

Comment:

[https://github.com/django/django/pull/15718 PR]

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:6>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
-------------------------------------+-------------------------------------
Reporter: Carlton Gibson | Owner: Mehrdad
Type: | Moradizadeh
Cleanup/optimization | Status: assigned
Component: Utilities | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mehrdad Moradizadeh):

I have made a [https://github.com/django/django/pull/15718 PR] this issue.

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:7>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
-------------------------------------+-------------------------------------
Reporter: Carlton Gibson | Owner: Mehrdad
Type: | Moradizadeh
Cleanup/optimization | Status: assigned
Component: Utilities | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mariusz Felisiak):

* needs_better_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:8>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
-------------------------------------+-------------------------------------
Reporter: Carlton Gibson | Owner: Mehrdad
Type: | Moradizadeh
Cleanup/optimization | Status: assigned
Component: Utilities | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"2f974e3c1e8c2a74114b1e9f0e8257743c4d472c" 2f974e3]:
{{{
#!CommitTicketReference repository=""
revision="2f974e3c1e8c2a74114b1e9f0e8257743c4d472c"
[4.1.x] Refs #33697 -- Fixed multipart parsing of headers with double
quotes and semicolons.

See
https://github.com/python/cpython/commit/1ef0c0349e8fdb5415e21231cb42edbf232b742a

Backport of 93cedc82f29076c824d476354527af1150888e4f from main
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:10>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
-------------------------------------+-------------------------------------
Reporter: Carlton Gibson | Owner: Mehrdad
Type: | Moradizadeh
Cleanup/optimization | Status: assigned
Component: Utilities | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"93cedc82f29076c824d476354527af1150888e4f" 93cedc8]:
{{{
#!CommitTicketReference repository=""
revision="93cedc82f29076c824d476354527af1150888e4f"

Refs #33697 -- Fixed multipart parsing of headers with double quotes and
semicolons.

See
https://github.com/python/cpython/commit/1ef0c0349e8fdb5415e21231cb42edbf232b742a
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:9>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
-------------------------------------+-------------------------------------
Reporter: Carlton Gibson | Owner: Mehrdad
Type: | Moradizadeh
Cleanup/optimization | Status: assigned
Component: Utilities | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"49b470b9187b6be60e573fed08c8f4a87f133750" 49b470b9]:
{{{
#!CommitTicketReference repository=""
revision="49b470b9187b6be60e573fed08c8f4a87f133750"
Refs #33697 -- Made MultiPartParser use
django.utils.http.parse_header_parameters() for parsing Content-Type
header.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:11>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
-------------------------------------+-------------------------------------
Reporter: Carlton Gibson | Owner: Mehrdad
Type: | Moradizadeh
Cleanup/optimization | Status: assigned
Component: Utilities | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak):

[https://github.com/django/django/pull/15797 New PR]

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:12>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
-------------------------------------+-------------------------------------
Reporter: Carlton Gibson | Owner: Mehrdad
Type: | Moradizadeh
Cleanup/optimization | Status: assigned
Component: Utilities | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"d6e0c7c30c1eb83f49345181dc63723968f297ae" d6e0c7c3]:
{{{
#!CommitTicketReference repository=""
revision="d6e0c7c30c1eb83f49345181dc63723968f297ae"
Refs #33697 -- Made MediaType use
django.utils.http.parse_header_parameters().
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:13>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
-------------------------------------+-------------------------------------
Reporter: Carlton Gibson | Owner: Mehrdad
Type: | Moradizadeh
Cleanup/optimization | Status: assigned
Component: Utilities | Version: 4.0
Severity: Normal | Resolution:

Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mariusz Felisiak):

* needs_better_patch: 1 => 0
* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:14>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
-------------------------------------+-------------------------------------
Reporter: Carlton Gibson | Owner: Mehrdad
Type: | Moradizadeh
Cleanup/optimization | Status: assigned
Component: Utilities | Version: 4.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"d4d5427571b4bf3a21c902276c2a00215c2a37cc" d4d54275]:
{{{
#!CommitTicketReference repository=""
revision="d4d5427571b4bf3a21c902276c2a00215c2a37cc"
Refs #33697 -- Used django.utils.http.parse_header_parameters() for
parsing boundary streams.

This also removes unused parse_header() and _parse_header_params()
helpers in django.http.multipartparser.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:15>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
-------------------------------------+-------------------------------------
Reporter: Carlton Gibson | Owner: Mehrdad
Type: | Moradizadeh

Cleanup/optimization | Status: closed
Component: Utilities | Version: 4.0
Severity: Normal | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mariusz Felisiak):

* status: assigned => closed
* resolution: => fixed

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:16>

[Django]

#33697: Cleanup duplication of HTTP header parsing in utils.http and multipart
parser.
-------------------------------------+-------------------------------------
Reporter: Carlton Gibson | Owner: Mehrdad
Type: | Moradizadeh
Cleanup/optimization | Status: closed
Component: Utilities | Version: 4.0
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by GitHub <noreply@…>):

In [changeset:"154dd1c0edb25f3ab1e951eef8e0c2a1e6274e87" 154dd1c]:
{{{
#!CommitTicketReference repository=""
revision="154dd1c0edb25f3ab1e951eef8e0c2a1e6274e87"
Refs #33697 -- Added backward incompatibility note about removing
multipartparser.parse_header().
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/33697#comment:17>