[Django] #26423: Make EmailValidator use HTML5 validation rather than more complicated regular expressions

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions
------------------------------------------------+------------------------
Reporter: timgraham | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Core (Other) | Version: master
Severity: Normal | Keywords:
Triage Stage: Accepted | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
------------------------------------------------+------------------------
As discussed on the [https://groups.google.com/d/topic/django-
developers/ASBJ0ge2KYo/discussion django-developers mailing list], the
regular expressions for validating email addresses are complicated for
questionable benefit. We should simplify it to use HTML5 type="email"
validation ([http://blog.gerv.net/2011/05/html5_email_address_regexp/
possible candidate]). A deprecation may be needed to give time for
projects to add back more complex validation that they might required.

--
Ticket URL: <https://code.djangoproject.com/ticket/26423>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------

Reporter: timgraham | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Core (Other) | Version: master

Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------

Comment (by sergei-maertens):

We should also assert that non-ascii characters are allowed in the local
part of the e-mail address, see #25986 for background on this issue.

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:1>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: timgraham | Owner: cjbcross
Type: Cleanup/optimization | Status: assigned

Component: Core (Other) | Version: master

Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by cjbcross):

* owner: nobody => cjbcross
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:2>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: timgraham | Owner: cjbcross
Type: Cleanup/optimization | Status: assigned

Component: Core (Other) | Version: master

Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------

Comment (by cjbcross):

Taking ownership on this at PyCon2016 Sprint. Discussed with Markus and
have background on consensus direction a solution should take.

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:3>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: timgraham | Owner: cjbcross
Type: Cleanup/optimization | Status: assigned

Component: Core (Other) | Version: master

Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------

Comment (by claudep):

It would have been nice to know the "consensus direction" mentioned in the
comment above!

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:4>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: timgraham | Owner: cjbcross
Type: Cleanup/optimization | Status: assigned

Component: Core (Other) | Version: master

Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------

Comment (by claudep):

#27029 was closed as a duplicate.

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:5>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: timgraham | Owner: cjbcross
Type: Cleanup/optimization | Status: assigned

Component: Core (Other) | Version: master

Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------

Comment (by claudep):

This [https://github.com/django/django/pull/7039 PR] only marginally
simplifies the regex, but all current tests are still passing, in addition
of the non-ASCII local part.
I'm not sure it solves all concerns of this report, to be
discussed/confirmed.

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:6>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: timgraham | Owner: cjbcross
Type: Cleanup/optimization | Status: assigned

Component: Core (Other) | Version: master

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by claudep):

* has_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:7>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: timgraham | Owner: cjbcross
Type: Cleanup/optimization | Status: assigned

Component: Core (Other) | Version: master

Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by claudep):

* has_patch: 1 => 0

Comment:

My patch was moved to #27029.

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:8>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: timgraham | Owner: cjbcross
Type: Cleanup/optimization | Status: assigned

Component: Core (Other) | Version: master

Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------

Comment (by deltaskelta):

Replying to [ticket:26423 timgraham]:

> As discussed on the [https://groups.google.com/d/topic/django-
developers/ASBJ0ge2KYo/discussion django-developers mailing list], the
regular expressions for validating email addresses are complicated for
questionable benefit. We should simplify it to use HTML5 type="email"
validation ([http://blog.gerv.net/2011/05/html5_email_address_regexp/
possible candidate]). A deprecation may be needed to give time for
projects to add back more complex validation that they might required.

I've found another possible fix here
https://html.spec.whatwg.org/multipage/forms.html#valid-e-mail-address,
but I have tried this regex and it does not match much out of the ASCII
range. What does "HTML5" like browsers match that this regex does not?

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:9>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: timgraham | Owner: cjbcross
Type: Cleanup/optimization | Status: assigned

Component: Core (Other) | Version: master

Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------

Comment (by claudep):

I suggest that Unicode validation be discussed separately in #27029.

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:10>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: Tim Graham | Owner: (none)

Type: Cleanup/optimization | Status: new
Component: Core (Other) | Version: master

Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by Tim Graham):

* status: assigned => new
* owner: Chris Butler => (none)

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:11>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: Tim Graham | Owner: (none)

Type: Cleanup/optimization | Status: new
Component: Core (Other) | Version: master

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by Tim Graham):

* has_patch: 0 => 1

Comment:

[https://github.com/django/django/pull/7742 PR] from Collin Anderson.

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:12>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: Tim Graham | Owner: (none)

Type: Cleanup/optimization | Status: new
Component: Core (Other) | Version: master

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by Tim Graham):

* needs_docs: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:13>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

-------------------------------------+-------------------------------------
Reporter: Tim Graham | Owner: Haris
Type: | Ibrahim K. V.
Cleanup/optimization | Status: assigned

Component: Core (Other) | Version: master

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by Haris Ibrahim K. V.):

* owner: (none) => Haris Ibrahim K. V.

* status: new => assigned

Comment:

I've added some documentation that I think will help keep the user
expectations clear after digging into the conversation / discussion
history of this ticket.

https://github.com/django/django/pull/8081

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:14>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

-------------------------------------+-------------------------------------
Reporter: Tim Graham | Owner: Haris
Type: | Ibrahim K. V.
Cleanup/optimization | Status: assigned

Component: Core (Other) | Version: master

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------

Comment (by MisRob):

#27932 was closed as a duplicate

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:15>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

-------------------------------------+-------------------------------------
Reporter: Tim Graham | Owner: Haris
Type: | Ibrahim K. V.
Cleanup/optimization | Status: assigned

Component: Core (Other) | Version: master

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------

Comment (by Max Nordlund):

I came across this today and realized that the current regexp, in 1.10.5,
contains a type. At least I think it does, since the opening bracket of
the second range in `user_regex` is escaped, but it shouldn't be.

Anyway, I threw together a small PR to fix that, and I hope it can be
merge to stable until this lands. I also added a bunch of new examples,
and at the very least those should get slurped into the PR from Haris.

See https://github.com/django/django/pull/8187

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:16>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: Tim Graham | Owner: (none)

Type: Cleanup/optimization | Status: new
Component: Core (Other) | Version: master

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------

Changes (by Haris Ibrahim K. V.):

* status: assigned => new
* owner: Haris Ibrahim K. V. => (none)

Comment:

Not maintaining the PR any more. Please do feel free to fork and submit a
new PR.

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:17>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: Tim Graham | Owner: (none)

Type: Cleanup/optimization | Status: new
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by Ülgen Sarıkavak):

* cc: Ülgen Sarıkavak (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:18>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: Tim Graham | Owner: Jordan
Type: Cleanup/optimization | Status: assigned

Component: Core (Other) | Version: dev
Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by Jordan):

* owner: (none) => Jordan

* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:19>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: Tim Graham | Owner: Jordan
Type: Cleanup/optimization | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Comment (by Jordan):

I've submitted a PR for this ticket (#19284) which implements HTML5-style
validation for EmailValidator with support for Unicode characters in the
local part. The implementation maintains backwards compatibility with the
domain_allowlist feature while making the validator more permissive for
internationalized email addresses. All CI checks are now passing.

PR: https://github.com/django/django/pull/19284
--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:20>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: Tim Graham | Owner: Jordan
Type: Cleanup/optimization | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by Mike Edmunds):

* cc: Mike Edmunds (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:21>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: Tim Graham | Owner: Jordan
Type: Cleanup/optimization | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by Mike Edmunds):

* has_patch: 1 => 0

Comment:

I've opened a [https://forum.djangoproject.com/t/emailvalidator-
simplification-and-international-email-addresses/39985 forum discussion]
to try to either move this ticket forward or close it wontfix.

Note that the HTML5 valid email address regular expression (alone) is
probably not a valid approach. It would break email features that were
specifically added to Django in past tickets. Details in the forum.

(Also, the most recent PR actually addresses #27029, not this ticket.
Clearing "has patch".)
--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:22>

[Django]

#26423: Make EmailValidator use HTML5 validation rather than more complicated
regular expressions

--------------------------------------+------------------------------------
Reporter: Tim Graham | Owner: Jordan

Type: Cleanup/optimization | Status: closed

Component: Core (Other) | Version: dev

Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 1

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by Mike Edmunds):

* resolution: => wontfix
* status: assigned => closed

Comment:

I am closing this "Make EmailValidator use HTML5 validation" ticket as not
workable per the forum discussion.

Other approaches to avoid "more complicated regular expressions" have come
up in both the original discussion list thread and in #27029. I've tried
to capture a specific proposal for that at
https://forum.djangoproject.com/t/emailvalidator-simplification-and-
international-email-addresses/39985/11.
--
Ticket URL: <https://code.djangoproject.com/ticket/26423#comment:23>