[Django] #35714: Support Unicode email addresses in SMTP EmailBackend (EAI, SMTPUTF8)

[Django]

#35714: Support Unicode email addresses in SMTP EmailBackend (EAI, SMTPUTF8)
------------------------------+-----------------------------------------
Reporter: Mike Edmunds | Type: Uncategorized
Status: new | Component: Core (Mail)
Version: 5.1 | Severity: Normal
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-----------------------------------------
This ticket proposes allowing use of the SMTPUTF8 extension in Django's
SMTP EmailBackend, as part of fully supporting email address
internationalization (EAI).

Among other things, EAI allows non-ASCII email addresses like
`jlóp...@example.mx`, where the local-part (username) can include Unicode
characters. As of Django 5.1 the current behavior for that address is to
send it using an invalid RFC 2047 encoding, which typically results in a
bounced message. (#35713 would change that to the behavior described below
for `smtputf8=False`.)

For more background on Internationalized Email see
[https://datatracker.ietf.org/doc/html/rfc6530.html RFC 6530] or
Wikipedia's [https://en.wikipedia.org/wiki/International_email
International email]. The SMTPUTF8 extension is defined by
[https://datatracker.ietf.org/doc/html/rfc6531.html RFC 6531].

Much of the popular
[https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol#SMTPUTF8 SMTP
server software supports SMTPUTF8]. However, deployment is not yet
widespread: a July, 2023 analysis found only ~22% of unique MX servers
advertised SMTPUTF8 support: [https://uasg.tech/download/uasg-047-ua-
readiness-report-fy23/ UASG 047 UA-Readiness Report FY23 (PDF)], sections
6.11–6.12.

=== Related tickets

Fully enabling EAI in Django requires three separate changes:
* Supporting SMTPUTF8 in the SMTP EmailBackend: this ticket.
* Allowing EmailMessage.message() to generate messages with
[https://datatracker.ietf.org/doc/html/rfc6532.html RFC 6532] 8-bit, utf-8
encoded headers. Updating to Python's modern email API will enable this:
#35581. (Prerequisite for this ticket.)
* Allowing non-ASCII local-parts to pass Django's EmailValidator: #27029
and some discussion in #26423. (Not required for this ticket.)

To avoid creating a new setting, this ticket leverages:
* Dictionary based EMAIL_PROVIDERS settings, #35514. (Prerequisite for
this ticket.)

(Also, #33969 requested something similar and was closed "needsinfo.")

=== Proposal

Add a new `smptutf8` option to `backends.smtp.EmailBackend.__init__()`.
Valid values would be:
* `True`: unconditionally use SMTPUTF8. Generate messages with 8-bit
headers using email.policy.SMTPUTF8, and request the SMTPUTF8 extension in
smtplib's `sendmail(..., mail_options)`—raising an error if the server
doesn't support it.
* `False`: never use SMTPUTF8. Raise an error when attempting to send a
message with non-ASCII local-parts in any email address.
* `None`: ("auto mode", the default) if an individual message includes any
non-ASCII local-parts in any email address, try to use SMTPUTF8 for that
message—and raise an error if the server doesn't support it. For all other
messages, don't try to use SMTPUTF8.

The `smtputf8` backend option would ''not'' be exposed as a separate
setting. It would only be available through kwargs to `get_connection()`
or the upcoming `EMAIL_PROVIDERS` dict setting.

The `True` setting is intended for use with an SMTP host that is known to
support SMTPUTF8. It offers a slight performance advantage over the `None`
option. Also, it would be the only way to use SMTPUTF8 (rather than RFC
2047 encoded headers) for messages that have non-ASCII characters in
headers like ''Subject,'' but only ASCII local-parts in addresses.

When the SMTP backend uses SMTPUTF8 for a message (either via `True` or
auto mode), it will also pass any non-ASCII domains (IDNs) in email
addresses directly to the SMTP server, rather than performing its own IDNA
encoding.
--
Ticket URL: <https://code.djangoproject.com/ticket/35714>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35714: Support Unicode email addresses in SMTP EmailBackend (EAI, SMTPUTF8)
------------------------------+--------------------------------------

Reporter: Mike Edmunds | Owner: (none)
Type: New feature | Status: new

Component: Core (Mail) | Version: 5.1

Severity: Normal | Resolution:

Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+--------------------------------------

Changes (by Mike Edmunds):

* type: Uncategorized => New feature

--
Ticket URL: <https://code.djangoproject.com/ticket/35714#comment:1>

[Django]

#35714: Support Unicode email addresses in SMTP EmailBackend (EAI, SMTPUTF8)

------------------------------+-----------------------------------------

Reporter: Mike Edmunds | Owner: (none)
Type: New feature | Status: new
Component: Core (Mail) | Version: 5.1
Severity: Normal | Resolution:

Keywords: | Triage Stage: Someday/Maybe

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

------------------------------+-----------------------------------------
Changes (by Claude Paroz):

* stage: Unreviewed => Someday/Maybe

Comment:

Thanks for the thorough proposal. Setting it as Someday/Maybe because it
depends on other tickets. Once all blockers are merged, we can pass the
ticket to Accepted.
--
Ticket URL: <https://code.djangoproject.com/ticket/35714#comment:2>

[Django]

#35714: Support Unicode email addresses in SMTP EmailBackend (EAI, SMTPUTF8)
------------------------------+-----------------------------------------

Reporter: Mike Edmunds | Owner: Abhay Mane
Type: New feature | Status: assigned

Component: Core (Mail) | Version: 5.1
Severity: Normal | Resolution:
Keywords: | Triage Stage: Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-----------------------------------------

Changes (by Abhay Mane):

* owner: (none) => Abhay Mane
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/35714#comment:3>