Re: [Django] #35378: Incorrect folding of long address headers with special characters when using 7bit Content-Transfer-Encoding in EmailMessage

[Django]

#35378: Incorrect folding of long address headers with special characters when
using 7bit Content-Transfer-Encoding in EmailMessage
-----------------------------+-----------------------------------------
Reporter: andres | Owner: Lufafa Joshua
Type: Bug | Status: assigned
Component: Core (Mail) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------+-----------------------------------------
Changes (by Lufafa Joshua):

* owner: nobody => Lufafa Joshua
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/35378#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35378: Incorrect folding of long address headers with special characters when
using 7bit Content-Transfer-Encoding in EmailMessage
-----------------------------+-----------------------------------------
Reporter: andres | Owner: Lufafa Joshua
Type: Bug | Status: assigned
Component: Core (Mail) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------+-----------------------------------------
Changes (by Lufafa Joshua):

* has_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/35378#comment:3>

[Django]

#35378: Incorrect folding of long address headers with special characters when
using 7bit Content-Transfer-Encoding in EmailMessage
-----------------------------+-----------------------------------------
Reporter: andres | Owner: Lufafa Joshua
Type: Bug | Status: assigned
Component: Core (Mail) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------+-----------------------------------------

Comment (by andres):

Thanks, Lufafa Joshua! I manually reproduced your fix here, did a quick
test and it seems to work!
--
Ticket URL: <https://code.djangoproject.com/ticket/35378#comment:4>

[Django]

#35378: Incorrect folding of long address headers with special characters when
using 7bit Content-Transfer-Encoding in EmailMessage
-----------------------------+-----------------------------------------
Reporter: andres | Owner: Lufafa Joshua
Type: Bug | Status: assigned
Component: Core (Mail) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-----------------------------+-----------------------------------------

Changes (by Sarah Boyce):

* needs_better_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/35378#comment:5>

[Django]

#35378: Incorrect folding of long address headers with special characters when
using 7bit Content-Transfer-Encoding in EmailMessage
-----------------------------+-----------------------------------------
Reporter: andres | Owner: Lufafa Joshua
Type: Bug | Status: assigned
Component: Core (Mail) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-----------------------------+-----------------------------------------

Changes (by Lufafa Joshua):

* has_patch: 1 => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/35378#comment:6>

[Django]

#35378: Incorrect folding of long address headers with special characters when
using 7bit Content-Transfer-Encoding in EmailMessage
-----------------------------+------------------------------------

Reporter: andres | Owner: (none)
Type: Bug | Status: new

Component: Core (Mail) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-----------------------------+------------------------------------

Changes (by Lufafa Joshua):

* owner: Lufafa Joshua => (none)
* status: assigned => new

--
Ticket URL: <https://code.djangoproject.com/ticket/35378#comment:7>

[Django]

#35378: Incorrect folding of long address headers with special characters when
using 7bit Content-Transfer-Encoding in EmailMessage
-----------------------------+------------------------------------
Reporter: andres | Owner: (none)
Type: Bug | Status: new
Component: Core (Mail) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-----------------------------+------------------------------------

Comment (by Mike Edmunds):

[I'm responsible for the example cited above. Setting an existing
message's `policy` attribute like that isn't documented anywhere, so
probably shouldn't be considered a valid test case. Apologies for that.]

I've come to the conclusion that any attempt to use modern
email.policy.default to serialize an email.message.Message (which is
constructed with legacy email.policy.compat32) is prone to errors, and
probably just isn't supported by the Python email package. Additional
information in https://github.com/anymail/django-
anymail/issues/369#issuecomment-2184252097

I would suggest ''not'' trying to fix this particular issue within Django.
Long term, there would be a lot of advantages to upgrading
django.core.mail to use modern email.message.EmailMessage (and
email.policy.default). But that's a separate issue, and I think needs to
happen all at once; trying to mix legacy and modern email code seems
problematic.

Here's an updated test, using only documented Python email package
features (generator policy override). It has the same problem, but I think
clarifies this is more of a Python email bug than a Django problem:

{{{#!python
import email.generator
import email.policy
import io
import django.core.mail

to = '"Người nhận a very very long, name" <t...@example.com>'
mime_message = django.core.mail.EmailMessage(to=[to]).message()

fp = io.BytesIO()
g = email.generator.BytesGenerator(
fp,
policy=email.policy.default.clone(cte_type="7bit"),
)
g.flatten(mime_message)
print(fp.getvalue().decode("ascii"))
# [... other headers ...]
# To: =?utf-8?b?TmfGsOG7nWkgbmjhuq1u?= a very very long, name
<t...@example.com>
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35378#comment:8>

[Django]

#35378: Incorrect folding of long address headers with special characters when
using 7bit Content-Transfer-Encoding in EmailMessage
-----------------------------+------------------------------------
Reporter: andres | Owner: (none)
Type: Bug | Status: new
Component: Core (Mail) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-----------------------------+------------------------------------

Changes (by Mike Edmunds):

* cc: Mike Edmunds (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/35378#comment:9>

[Django]

#35378: Incorrect folding of long address headers with special characters when
using 7bit Content-Transfer-Encoding in EmailMessage

---------------------------------+------------------------------------

Reporter: andres | Owner: (none)
Type: Bug | Status: new
Component: Core (Mail) | Version: dev
Severity: Normal | Resolution:

Keywords: email, compat32 | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0

---------------------------------+------------------------------------
Changes (by Mike Edmunds):

* keywords: => email, compat32

--
Ticket URL: <https://code.djangoproject.com/ticket/35378#comment:10>

[Django]

#35378: Incorrect folding of long address headers with special characters when
using 7bit Content-Transfer-Encoding in EmailMessage
---------------------------------+------------------------------------
Reporter: andres | Owner: (none)

Type: Bug | Status: closed

Component: Core (Mail) | Version: dev

Severity: Normal | Resolution: invalid

Keywords: email, compat32 | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------

Changes (by Sarah Boyce):

* resolution: => invalid
* status: new => closed

Comment:

Thank you for the update Mike, updated the ticket accordingly
Also following the discussion of https://groups.google.com/u/2/g/django-
developers/c/2zf9GQtjdIk which is nice to see 👍
--
Ticket URL: <https://code.djangoproject.com/ticket/35378#comment:11>

[Django]

#35378: Incorrect folding of long address headers with special characters when
using 7bit Content-Transfer-Encoding in EmailMessage
---------------------------------+------------------------------------
Reporter: andres | Owner: (none)
Type: Bug | Status: closed
Component: Core (Mail) | Version: dev
Severity: Normal | Resolution: invalid
Keywords: email, compat32 | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
---------------------------------+------------------------------------

Comment (by Mike Edmunds):

(Reported Python email bug as
[https://github.com/python/cpython/issues/121284 python/cpython#121284 ])
--
Ticket URL: <https://code.djangoproject.com/ticket/35378#comment:12>