Proposal: Add additional password validators (auth/password_validation.py)

[Brad Solomon]

I recently opened a ticket (30442) proposing to add more password validators to auth/password_validation.py, which was promptly closed.

I'm not here to cry and moan about it being closed as wontfix, but rather, to raise some questions and show why I think more discussion is warranted.

Tickets:

- https://code.djangoproject.com/ticket/30442

- https://code.djangoproject.com/ticket/27568

- https://code.djangoproject.com/ticket/30100

Some thoughts:

Why would it be "easy" to add new password validators?  Why would it be beneficial?

• They don't need to be added to settings.AUTH_PASSWORD_VALIDATORS, which currently uses UserAttributeSimilarityValidator, MinimumLengthValidator, CommonPasswordValidator, NumericPasswordValidator.  In other words, adding additional validators is "ala carte,"; it gives additional options to the developer without changing default behavior, and the developer in turn can simply turn on additional validators rather than installing them from a third-party package.  Let's say we have 3 different tickets with 3 different proposal for password validators.  Does it really make sense to suggest that this should lead to 3 different 3rd party packages?  Is that what is ultimately easiest for the end developer?
• Tests are straightforward and easy to build: like the existing ones in tests/auth_tests/test_validators.py, they really just need to implement test_validate() and test_help_text().
• In this case, more is better.  It puts no additional burden on developers who want to take no action but also lets you "bolt on" additional validator classes as needed.  A consensus in password security is that a single check in isolation is insufficient; it is the combination of multiple checks on a password that help to stem against bad practices.
  

What are some baseline criteria that could be considered when adding new validators?

• Are the new classes lightweight?  Do they use no 3rd party dependencies?  Do they have network dependencies/downloads or perform any file IO at module load time? [both not ideal].  If a user actually decides to add them to their project settings, are they fast and durable?
• Do the classes offer genuinely "different" functionality relative to the existing classes already in auth/password_validation.py?  Or are they in some regard a reformulation of existing checks?

What arguments have been used in closing past tickets?

• "This seems fine as a third-party app to me. Presumably some thought went into what validators should be built-in when the password validation was first added in Django." (27568)
• To this I would ask (not at all facetiously): was there a deliberate thought process that went into what validators should be built in?  What was that thought process?  Where is it documented?
  
• "I think it is a good candidate for a third-party package." (30442)
• Again, I do not ask this rhetorically: what is the argument for placing additional validators in third party packages that (1) are developed and maintained by different developers, (2) require an additional install/bookmark/etc rather than just being built into Django and (3) might even have a different API if the developer so decides?  This would not be blowing up the library with some backwards-compatible alien feature; it is adding close-cousins of existing classes that already have a stable API, all contained within the same module.
  

[Adam Johnson]

Hi Brad,

I appreciate no one has replied here for some time, and your ticket was closed rapidly after your work creating the code. I can imagine it might be a bit frustrating.

Thanks for trying to improve Django. I see you've submitted some other PR's since, thanks for that work.

My thoughts on the password validators:

First, I'd like to say there has probably not been much thought on the issue. The Django fellows, and everyone on this mailing list, have a lot on their plates. The fellows have probably defaulted to "no" on these feature request tickets simply to keep Django at a manageable size. We are conservative in adding new code because it needs maintaining forever (or however long Django will live!). You're right the API surface area is small and stable, but new features all add up. 

Also the suggestion to create a third party package has a precedent. The Django ecosystem is maybe 10,000x more code than Django core - core simply can't contain the long tail. A number of good ideas in Django have actually been third party packages first, for example Andrew Godwin's South which evolved into Django Migrations.

Additionally, releasing a package means that users (yourself included!) can use the features *today*, rather than in 9 months' time when the next Django release comes out. (...or 5 years time when your project actually gets upgraded.)

Second, I would like to consider the password validation more closely. When I think about the topic these days, I point to the NIST guidelines. They went through a big change in 2016. I saw this reported a couple times in security media - it was hailed as a forwards step towards more sane password management across the industry. Here's an article summarizing them in non-governmental language: https://nakedsecurity.sophos.com/2016/08/18/nists-new-password-rules-what-you-need-to-know/ .

in general the guidelines represent a move away from the attitude you spirit you quoted in your PR: "more validation is almost always better"

I think the guideline against composition rules would be against the first three validators you suggested in #30442, to quote the article:

No composition rules. What this means is, no more rules that force you to use particular characters or combinations, like those daunting conditions on some password reset pages that say, “Your password must contain one lowercase letter, one uppercase letter, one number, four symbols but not &%#@_, and the surname of at least one astronaut.”

Additionally, although the article doesn't mention it, I found in the document NIST.SP.800-63b Appendix A a warning against trying to measure entropy. This would argue against adding the entropy validator suggestions in #27568 and #30442:

Complexity of user-chosen passwords has often been characterized using the information theory concept of entropy [Shannon]. While entropy can be readily calculated for data having deterministic distribution functions, estimating the entropy for user-chosen passwords is difficult and past efforts to do so have not been particularly accurate. For this reason, a different and somewhat simpler approach, based primarily on password length, is presented herein.

On the other hand, there is a recommendation to check against databases of known crackable passwords. This would lead us to the pwned passwords validator suggested in #30100 as a more powerful version of Django's built-in CommonPasswordValidator. There are a couple reasons I can see *not* to add it to Django core though:

• The validator relies on a third party web service. If the service changes after a version of Django goes out of support, the validator will stop working and no future release of that version of Django will be issued to fix it.
• There is already a package implementing this validator, by Django core team member James Bennett: https://pypi.org/project/pwned-passwords-django/ . I think packages maintained by James are of a quality only second to Django itself.

That said, whilst it might be "more sane" to move to the NIST guidelines, Django can adopt some pragmatism. We developers can be forced to implement validation to fit in with some regulatory guidelines. I think if there was good evidence that a validator would be required by a large number of Django projects, I believe it wouldn't be out of place in Django core. On the other hand, a great way to prove this utility is to make a third party package that becomes popular :)

Looking at Django Packages, I found only one third party package explicitly offering password validators: https://djangopackages.org/packages/p/django-password-validators/  . And for its name, it only offers one validator at current, UniquePasswordsValidator . It seems there isn't much state of the art for password validators right now - but perhaps they're not a particularly interesting topic for third party package development!

To conclude, I think you asked a number of good questions around what could constitute grounds for adding password validators (or any features) to Django core. I admit I haven't addressed them directly, but I hope I've approached them enough indirectly. If you have any more thoughts, please do respond!

Thanks,

Adam

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/d694dc90-fd26-4a54-a609-547b56b00db9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Adam

[Brad Solomon]

Thanks Adam--the thoroughness of your response makes me more, rather than less, interested in contributing further to Django.  And I must admit you have me beat on just about all points raised.  As a new contributor, hearing your framing of the consideration that goes into feature requests was helpful also.

Brad

You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/GzZ1SB2d_FM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.

To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM3fLAXB2-Bot8Q7C43h7ecXGU7q3-UmJ4tu-uHZLE%3DUGw%40mail.gmail.com.

[Adam Johnson]

No worries, I wouldn't say I beat you, this is just a discussion :) Looking forward to your future contributions

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFqxaX1hS5qxgXeZApkGfitAETNFXss_WKC%3DK4DstQ%3DrPMJBDA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Adam