[Django] #35497: Long email address causes crash when generating a message

[Django]

#35497: Long email address causes crash when generating a message
---------------------------------------------+------------------------
Reporter: Alexandru Chirila | Owner: nobody
Type: Bug | Status: new
Component: Core (Mail) | Version: 5.0
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
---------------------------------------------+------------------------
Trying to generate an email message that has a long recipient address and
non-ASCII characters causes a crash.

How to reproduce:

{{{
EmailMessage(to=["ţēśţ." * 6 + "@example.com"]).message()
}}}

Looking for an existing issue I have found #31784. It seems like the issue
there was only fixed for long names and long addresses.

----

Version info:

- Django: 5.0.6
- Python: 3.12.3

----

Full log:

{{{#!python
In [2]: from django.core.mail import EmailMessage

In [3]: EmailMessage(to=["ţēśţ." * 6 + "@example.com"]).message()
---------------------------------------------------------------------------
UnicodeEncodeError Traceback (most recent call
last)
File .venv/lib/python3.12/site-packages/django/core/mail/message.py:64, in
forbid_multi_line_headers(name, val, encoding)
63 try:
---> 64 val.encode("ascii")
65 except UnicodeEncodeError:

UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-3:
ordinal not in range(128)

During handling of the above exception, another exception occurred:

ValueError Traceback (most recent call
last)
Cell In[3], line 1
----> 1 EmailMessage(to=["ţēśţ." * 6 + "@example.com"]).message()

File .venv/lib/python3.12/site-packages/django/core/mail/message.py:267,
in EmailMessage.message(self)
265 msg["Subject"] = self.subject
266 msg["From"] = self.extra_headers.get("From", self.from_email)
--> 267 self._set_list_header_if_not_empty(msg, "To", self.to)
268 self._set_list_header_if_not_empty(msg, "Cc", self.cc)
269 self._set_list_header_if_not_empty(msg, "Reply-To", self.reply_to)

File .venv/lib/python3.12/site-packages/django/core/mail/message.py:432,
in EmailMessage._set_list_header_if_not_empty(self, msg, header, values)
430 except KeyError:
431 value = ", ".join(str(v) for v in values)
--> 432 msg[header] = value

File .venv/lib/python3.12/site-packages/django/core/mail/message.py:165,
in SafeMIMEText.__setitem__(self, name, val)
164 def __setitem__(self, name, val):
--> 165 name, val = forbid_multi_line_headers(name, val,
self.encoding)
166 MIMEText.__setitem__(self, name, val)

File .venv/lib/python3.12/site-packages/django/core/mail/message.py:67, in
forbid_multi_line_headers(name, val, encoding)
65 except UnicodeEncodeError:
66 if name.lower() in ADDRESS_HEADERS:
---> 67 val = ", ".join(
68 sanitize_address(addr, encoding) for addr in
getaddresses((val,))
69 )
70 else:
71 val = Header(val, encoding).encode()

File .venv/lib/python3.12/site-packages/django/core/mail/message.py:68, in
<genexpr>(.0)
65 except UnicodeEncodeError:
66 if name.lower() in ADDRESS_HEADERS:
67 val = ", ".join(
---> 68 sanitize_address(addr, encoding) for addr in
getaddresses((val,))
69 )
70 else:
71 val = Header(val, encoding).encode()

File .venv/lib/python3.12/site-packages/django/core/mail/message.py:120,
in sanitize_address(addr, encoding)
117 localpart = Header(localpart, encoding).encode()
118 domain = punycode(domain)
--> 120 parsed_address = Address(username=localpart, domain=domain)
121 return formataddr((nm, parsed_address.addr_spec))

File ~/.pyenv/versions/3.12.3/lib/python3.12/email/headerregistry.py:33,
in Address.__init__(self, display_name, username, domain, addr_spec)
31 inputs = ''.join(filter(None, (display_name, username, domain,
addr_spec)))
32 if '\r' in inputs or '\n' in inputs:
---> 33 raise ValueError("invalid arguments; address parts cannot
contain CR or LF")
35 # This clause with its potential 'raise' may only happen when an
36 # application program creates an Address object using an addr_spec
37 # keyword. The email library code itself must always supply
username
38 # and domain.
39 if addr_spec is not None:

ValueError: invalid arguments; address parts cannot contain CR or LF
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35497>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35497: Long email address causes crash when generating a message

-------------------------------------+-------------------------------------
Reporter: Alexandru Chirila | Owner: Clinton
| Christian
Type: Bug | Status: assigned

Component: Core (Mail) | Version: 5.0

Severity: Normal | Resolution:

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by Clinton Christian):

* owner: nobody => Clinton Christian
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/35497#comment:1>

[Django]

#35497: Long email address causes crash when generating a message
-------------------------------------+-------------------------------------
Reporter: Alexandru Chirila | Owner: Clinton
| Christian
Type: Bug | Status: assigned
Component: Core (Mail) | Version: 5.0
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Clinton Christian):

* has_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/35497#comment:2>

[Django]

#35497: Long email address causes crash when generating a message
-------------------------------------+-------------------------------------
Reporter: Alexandru Chirila | Owner: Clinton
| Christian
Type: Bug | Status: assigned
Component: Core (Mail) | Version: 5.0
Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Clinton Christian):

* stage: Unreviewed => Accepted

--
Ticket URL: <https://code.djangoproject.com/ticket/35497#comment:3>

[Django]

#35497: Long email address causes crash when generating a message
-------------------------------------+-------------------------------------
Reporter: Alexandru Chirila | Owner: Clinton
| Christian
Type: Bug | Status: assigned
Component: Core (Mail) | Version: 5.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 1 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Clinton Christian):

* needs_tests: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/35497#comment:4>

[Django]

#35497: Long email address causes crash when generating a message
-------------------------------------+-------------------------------------
Reporter: Alexandru Chirila | Owner: Clinton
| Christian
Type: Bug | Status: assigned
Component: Core (Mail) | Version: 5.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Clinton Christian):

* needs_tests: 1 => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/35497#comment:5>

[Django]

#35497: Long email address causes crash when generating a message
-------------------------------------+-------------------------------------
Reporter: Alexandru Chirila | Owner: Clinton
| Christian
Type: Bug | Status: assigned
Component: Core (Mail) | Version: 5.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* cc: Florian Apolloner (added)
* needs_better_patch: 0 => 1

Comment:

#31784 discussed and went against the idea updating the line length to 998
and so we would need to revisit that discussion if we are to accept the
current PR.
From what I see, the arguments in #31784 still stand and this is not the
approach to resolve this issue.
--
Ticket URL: <https://code.djangoproject.com/ticket/35497#comment:6>

[Django]

#35497: Long email address causes crash when generating a message
-------------------------------------+-------------------------------------
Reporter: Alexandru Chirila | Owner: Clinton
| Christian
Type: Bug | Status: assigned
Component: Core (Mail) | Version: 5.0
Severity: Normal | Resolution:

Keywords: email, compat32 | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mike Edmunds):

* cc: Mike Edmunds (added)
* keywords: => email, compat32

Comment:

[This issue would also be solved—without altering line length—by upgrading
django.core.mail from legacy email.message.Message (policy=compat32) to
modern email.message.EmailMessage (policy=default), and letting the modern
email package handle header sanitization and folding. Though that's a much
larger scope than this individual bug.]

{{{#!python
import email.message
msg = email.message.EmailMessage()
msg["To"] = "ţēśţ." * 6 + "@example.com"
print(msg.as_bytes().decode("ascii"))
# To:
=?utf-8?b?xaPEk8WbxaMuxaPEk8WbxaMuxaPEk8WbxaMuxaPEk8WbxaMuxaPEk8WbxaMu?=
# =?utf-8?b?xaPEk8WbxaM=?=.@example.com
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35497#comment:7>

[Django]

#35497: Long email address causes crash when generating a message
-------------------------------------+-------------------------------------
Reporter: Alexandru Chirila | Owner: Clinton
| Christian
Type: Bug | Status: assigned
Component: Core (Mail) | Version: 5.0
Severity: Normal | Resolution:
Keywords: email, compat32 | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Florian Apolloner):

Replying to [comment:7 Mike Edmunds]:

> [This issue would also be solved—without altering line length—by
upgrading django.core.mail from legacy email.message.Message
(policy=compat32) to modern email.message.EmailMessage (policy=default),
and letting the modern email package handle header sanitization and
folding. Though that's a much larger scope than this individual bug.]

Would be worth to investigate how much effort that would be. I'd be in
favor of staying in line with Python.
--
Ticket URL: <https://code.djangoproject.com/ticket/35497#comment:8>

[Django]

#35497: Long email address causes crash when generating a message
-------------------------------------+-------------------------------------
Reporter: Alexandru Chirila | Owner: Clinton
| Christian
Type: Bug | Status: assigned
Component: Core (Mail) | Version: 5.0
Severity: Normal | Resolution:
Keywords: email, compat32 | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mike Edmunds):

> Would be worth to investigate how much effort [upgrading to Python
modern email API] would be.

Discussion in progress: https://groups.google.com/g/django-
developers/c/2zf9GQtjdIk
--
Ticket URL: <https://code.djangoproject.com/ticket/35497#comment:9>

[Django]

#35497: Long email address causes crash when generating a message
-------------------------------------+-------------------------------------
Reporter: Alexandru Chirila | Owner: Clinton
| Christian
Type: Bug | Status: assigned
Component: Core (Mail) | Version: 5.0
Severity: Normal | Resolution:
Keywords: email, compat32 | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Clinton Christian):

Replying to [comment:9 Mike Edmunds]:

> > Would be worth to investigate how much effort [upgrading to Python
modern email API] would be.
>
> Discussion in progress: https://groups.google.com/g/django-
developers/c/2zf9GQtjdIk

Thanks, I was unfamiliar with this thread, and #31784.

My takeaway:

We are updating the `django.core.mail` to take advantage of changes made
to python's 3.6+ `email` module, while ensuring backwards compatibility
(as much as possible).

Methods in question from `django.core.email`:
`CachedDnsName`
`DNS_NAME`
`EmailMessage`
`EmailMultiAlternatives`
`SafeMIMEText`
`SafeMIMEMultipart`
`DEFAULT_ATTACHMENT_MIME_TYPE`
`make_msgid`
`BadHeaderError`
`forbid_multi_line_headers`
`get_connection`
`send_mail`
`send_mass_mail`
`mail_admins`
`mail_managers`

Deprecations:
`SafeMIMEText`
`SafeMIMEMultipart`
`forbid_multi_line_headers`
`BadHeaderError`

There are several underscore methods in `EmailMessage` and
`EmailMultiAlternatives`. These will need to be removed.
Python's 3.6+ `email.message` API changes do alter default behavior. It
seems like we would need to override these new defaults in order to ensure
backwards compatibility.

Regression tests will need to be introduced to ensure that we aren't
making (unintended) breaking changes.

Are my assumptions accurate? Let me know if I missed anything.

Also, has a ticket already been created for this? If not I can create one,
and reference it here, as the increased scope warrants a new ticket.
--
Ticket URL: <https://code.djangoproject.com/ticket/35497#comment:10>

[Django]

#35497: Long email address causes crash when generating a message
-------------------------------------+-------------------------------------
Reporter: Alexandru Chirila | Owner: Clinton
| Christian
Type: Bug | Status: assigned
Component: Core (Mail) | Version: 5.0
Severity: Normal | Resolution:
Keywords: email, compat32 | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mike Edmunds):

> Also, has a ticket already been created for this?

I just posted to django-developers a few days ago. Waiting to create a
ticket until we get positive/negative votes there.

Agreed that this ticket is not the best place for the larger scope
proposal. I just added another post to the django-developers thread with
more details on the proposed change. I'd prefer to keep the discussion
over on django-developers for now, so that it's all in one place.
--
Ticket URL: <https://code.djangoproject.com/ticket/35497#comment:11>