Ticket 2273: django.contrib.auth.models.User: username is case-sensitive

[Daniel Stanton]

Following the end of https://code.djangoproject.com/ticket/2273 over a year ago, could this be reconsidered please?

By using username__iexact when checking for uniqueness for new usernames, this wouldn't affect any usernames that already exist, and it's then up to the administrators to decide if they want all users to conform to this and take advantage of knowing all usernames are unique when ignoring case, or if it makes no difference to them - either way, it becomes their legacy, not Django's.

As far as adding a feature flag for this, is that really required? I can't imagine any use case where not using iexact would be the preferred option. Would it be helpful to ask the community?

I'm certainly not an experienced developer, so it'd really be great to understand the reasons against. Thanks!

[Tim Graham]

I can't think of a reason not to make the suggested change. It seems like an odd requirement if a site *requires* allowing new usernames to be case-sensitive.

[Florian Apolloner]

On Saturday, August 29, 2015 at 7:08:45 PM UTC+2, Tim Graham wrote:

I can't think of a reason not to make the suggested change. It seems like an odd requirement if a site *requires* allowing new usernames to be case-sensitive.

Which is? Following the ticket I do not see any clear consenus at all. Imo Registration apps should handle that, which is not something Django ships with. I am -0 on changing the admin to disallow such users.

[James Bennett]

On Sat, Aug 29, 2015 at 1:54 PM, Florian Apolloner <f.apo...@gmail.com> wrote:

Which is? Following the ticket I do not see any clear consenus at all. Imo Registration apps should handle that, which is not something Django ships with. I am -0 on changing the admin to disallow such users.

For what it's worth, django-registration uses an 'iexact' lookup to check for duplicate usernames during signup. Treating case variations as the same username seems to be the standard on most sites I've used.

[Tim Graham]

The suggested change is to not allow a new user if there's a username__iexact match.

[James Bennett]

Yeah, that was why I pointed out django-registration already does this. Last time I looked at user-signup apps, it seemed that was the common practice, and also non-Django-powered sites I've used tend to do this. Allowing variations in case to be different users raises a lot of problems of potential impersonation.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/acbbb18a-45f1-4147-bbd4-cb7e4bd1b3ab%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Florian Apolloner]

On Saturday, August 29, 2015 at 9:59:30 PM UTC+2, Tim Graham wrote:

The suggested change is to not allow a new user if there's a username__iexact match.

Yes, but as change to what? To the admin interface? this will only cover admin usage. To a full_clean of the user model [This is often not used anyways]? I am somewhat against that…

[Collin Anderson]

I propose adding a check on UserCreationForm (used by the admin). The current implementation relies on the model fields unique=True check which is case-sensitive (except on mysql).

Whenever I've made a login form, I've always used or made a copy of or at least studied UserCreationForm. I think if we had a nice walk through of how to create your own form which included the iexact hint, I think that would also help solve this problem.

--

You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/650405a3-c860-4d84-be94-5b6a6c779aa2%40googlegroups.com.

[Reid Ransom]

Is it reasonable to consider changing the default for usernames to be case-insensitive in 2.0?

[charettes]

Hi Reid,

2.0 is not planned to be a special release in regard to backward compatibility.

Since this is a backward incompatible change (what should be done with
duplicate usernames, etc.) I doubt it's going to happen.

Simon

[Paul Egges]

Perhaps I'm missing something, but this would not change current users, only the creation of new users. It seems that logins would still be case sensitive. It would just prevent new users from being created that would match in a case insensitive manner with an existing user. For example existing users:

SomeUser and someuser could still use the site in the same manner as the login process would still be case sensitive. But a new user could not signup with a username of someUser.   

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a7c3add7-d13d-4b5c-af73-ac74cbc1c6a0%40googlegroups.com.

[Shai Berger]

Hi,

On Thursday 22 October 2015 00:01:24 Paul Egges wrote:
> Perhaps I'm missing something, but this would not change current users,
> only the creation of new users. It seems that logins would still be case
> sensitive.

Not the way Reid presented it:

> > Le mercredi 21 octobre 2015 15:44:55 UTC-4, Reid Ransom a écrit :
> >> Is it reasonable to consider changing the default for usernames to be
> >> case-insensitive in 2.0?

I think, though, it would be easy enough for us to provide a new, case-
insensitive base class for users, and change the recommendations in our
documentation to tell users to inherit that rather than the current
AbstractBaseUser and AbstractUser. We could also write a simple management
command to validate lower-case uniquness and turn all usernames to lowercase,
as preparation for changing the login and registration logic.

These could all be done outside of core, and perhaps they should be -- the
only point I see for including them in core is the risk that, as a developer,
if core doesn't make you think about it from the get-go, by the time you
decide to make the change you may be stuck with conflicting (lower-case equal)
usernames in your database. But frankly, I would guess that this problem does
not really occur very often; that for most sites, if they decide to switch to
case-insensitive usernames, there would be no problem.

Regretfully, we can't just switch Django to do that, because of the few sites
who will have a problem, and no clear upgrade path.

Shai.

[Reid Ransom]

Yesterday, I noticed a couple users had created multiple accounts with

different capitalization.  One person had 4 accounts.  I've also gotten

a lot of complaints from users that were having trouble logging in.  My

response was generally "Use the forgot password link." but now I'm 

realizing that they probably had difficulty remembering their username

capitalization they used during registration or didn't know the

capitalization our staff used to create their account.

I was already working with a custom user model and all my

conflicting usernames were for the same people so this wasn't too hard

to fix.  Still I would expect Django to want devs to start with case-

insensitive for the reasons already cited in this thread but also since

it's much easier to switch to case-sensitive later if that's what you

really want.

Shai- Could an upgrade path for what you proposed be to change the

AUTH_USER_MODEL setting?

[Paul Egges]

Yeah, I was referring to the post by Daniel at the start of the thread, not Reid's comment.

I would be -1 on changing the check of usernames at login to case-insensitive. But I would be +1 with preventing the creation of new usernames which would match existing usernames in a case-insensitive manner.

[bliy...@rentlytics.com]

> -1 on changing the check of usernames at login to case-insensitive

> +1 with preventing the creation of new usernames

Ditto

[Collin Anderson]

I also agree. And to be clear, we're talking about UserCreationForm, right? Or where specifically would we implement this? (What part of Django do you use that doesn't check this?)

[Tim Graham]

I created https://code.djangoproject.com/ticket/25617 - Disallow usernames that differ only in case in UserCreationForm.

Please voice any concerns on that ticket.

[Info-Screen]

Wouldn't it be possible to implement case-insensitive usernames without loosing backwards compatibility, by checking the username iexact and only if there are multiple possibilities fall back to the old case-sensitive variant?

So something like this:

diff --git a/django/contrib/auth/base_user.py b/django/contrib/auth/base_user.py

index 34dd6ac2f2..748db8bf89 100644

--- a/django/contrib/auth/base_user.py

+++ b/django/contrib/auth/base_user.py

@@ -4,6 +4,7 @@ not in INSTALLED_APPS.

 """

 import unicodedata

 

+from django.core.exceptions import MultipleObjectsReturned

 from django.contrib.auth import password_validation

 from django.contrib.auth.hashers import (

     check_password, is_password_usable, make_password,

@@ -41,7 +42,14 @@ class BaseUserManager(models.Manager):

         return get_random_string(length, allowed_chars)

 

     def get_by_natural_key(self, username):

-        return self.get(**{self.model.USERNAME_FIELD: username})

+        username_field = self.model.USERNAME_FIELD

+

+        # Try case-insensitive match of username.

+        # If there are multiple possiblities fallback to case-sensitive lookup

+        try:

+            return self.get(**{username_field + '__iexac': username})

+        except MultipleObjectsReturned:

+            return self.get(**{username_field: username})

 

 

 class AbstractBaseUser(models.Model):

[Tobias McNulty]

I'm surprised that (as far as I can see) this idea hasn't been discussed in the ~11 year (!) history of this ticket. It was alluded to ~3 years go by mkhalil28, but was missing the backwards-compatible 'except' bit and didn't get any follow-up discussion. To me this seems like an appropriate corresponding change to #25617.

If I'm thinking through this correctly, this would provide, for all intents and purposes:

• case-insensitive usernames for all new sites (and new users on existing sites)
• indefinite backwards compatibility for sites that already have "duplicate" usernames in the database, when case is ignored
• no feature flag that needs to be maintained
• the added benefit of allowing sites that really want case-sensitive usernames to disable this behavior, if needed (by never adding unique=True to the username field)

If a change like this has been considered already, I am curious to know why it was rejected.

I am not suggesting (yet) that we re-open #2273, but I think this would at least be worth exploring further: What would a minimum-impact change for case-insensitive usernames look like, both in terms of code and documentation changes? #25617 + this change seem like a good start, but more work will certainly be required to get this into a state that's ready for formal review. I probably won't have time to do that myself in the near future, but I would be happy to serve as a reviewer.

Tobias

Tobias McNulty
Chief Executive Officer

tob...@caktusgroup.com
www.caktusgroup.com

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/cc07fa69-06b3-4d24-aa2c-e5201ebe936a%40googlegroups.com.

[Info-Screen]

I might be able to try to implement this. I'm new to django, but let's try

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

[Info-Screen]

Code Changes:

Something like I already mentioned could be used when trying to login:

diff --git a/django/contrib/auth/base_user.py b/django/contrib/auth/base_user.py

index 34dd6ac2f2..748db8bf89 100644

--- a/django/contrib/auth/base_user.py

+++ b/django/contrib/auth/base_user.py

@@ -4,6 +4,7 @@ not in INSTALLED_APPS.

 """

 import unicodedata

 

+from django.core.exceptions import MultipleObjectsReturned

 from django.contrib.auth import password_validation

 from django.contrib.auth.hashers import (

     check_password, is_password_usable, make_password,

@@ -41,7 +42,14 @@ class BaseUserManager(models.Manager):

         return get_random_string(length, allowed_chars)

 

     def get_by_natural_key(self, username):

-        return self.get(**{self.model.USERNAME_FIELD: username})

+        username_field = self.model.USERNAME_FIELD

+

+        # Try case-insensitive match of username.

+        # If there are multiple possiblities fallback to case-sensitive lookup

+        try:

+            return self.get(**{username_field + '__iexac': username})

+        except MultipleObjectsReturned:

+            return self.get(**{username_field: username})

 

 

 class AbstractBaseUser(models.Model):

and to check if the username is already taken, something like the following could be used:

    def validate_unique(self, exclude=None):

        errors = {}

        if exclude is None:

            exclude = []

        else:

            exclude = list(exclude)

        try:

            super().validate_unique(exclude)

        except ValidationError as e:

            errors = e.update_error_dict(errors)

        if not (self.USERNAME_FIELD in exclude or self.USERNAME_FIELD in errors):

            # Check if username is unique case-insensitive

            if self.__class__._default_manager.filter(

                **{self.USERNAME_FIELD + '__iexact': getattr(self, self.USERNAME_FIELD)}).exists():

                errors.update(

                    {self.USERNAME_FIELD: self.unique_error_message(self.__class__, (self.USERNAME_FIELD,))}

                )

        if errors:

            raise ValidationError(errors)

Possible Test cases:

- Logging In

-- only 1 case-insensitive

---- Login with different case

---- (Login with same case should already have a test case)

-- multiple case-insensitive

---- Login with casing of one

---- Login with casing of none

- Register

-- Create new different

-- Create new different only in case

For the docs, would it be more than a `versionchanged` block?

---------------------------------------------------------------

Info-Screen

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/a4I7rHuT4Dw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-developers+unsubscribe@googlegroups.com.

To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMGFDKQaa4RX31yJqOapRvehH8GBPObybJmLCJ44JqT0OZpqFA%40mail.gmail.com.

[Shelagh Lewins]

I think case-insensitive usernames is a must-have in the mobile world. Users who register on mobile don't notice that their phone has automatically capitalised the first letter of their username when they enter it, and are then unable to log in on laptops because they are typing the lower-case version.

And because case-sensitive usernames are so unexpected, developers are likely not to notice the problem until some way into the project.

To expect every developer to modify the auth model because it doesn't provide standard functionality seems like the wrong approach to me.

If there are concerns about backwards compatibility, why not make it an optional setting?

[אורי‎]

In Speedy Net all usernames and slugs must be lowercase. If they are entered not lowercase they are converted to lowercase. I don't see any reason why usernames should not be lowercase, especially if there are 2 different users with usernames which differ only in case.

All email addresses in Speedy Net are also converted to lowercase and saved lowercase in the database. I think it would be absurd if u...@speedy.net, U...@speedy.net and U...@SPEEDY.NET would belong to 3 different people with 3 different usernames and email addresses.

From https://github.com/speedy-net/speedy-net/blob/uri_merge_with_master_2018-12-31_a/speedy/core/accounts/tests/test_forms.py:

    def test_slug_gets_converted_to_lowercase(self):

        data = self.data.copy()

        data['slug'] = 'THIS-IS-A-SLUG'

        form = RegistrationForm(language_code=self.language_code, data=data)

        form.full_clean()

        self.assertTrue(expr=form.is_valid())

        self.assertDictEqual(d1=form.errors, d2={})

        user = form.save()

        self.assertEqual(first=user.slug, second='this-is-a-slug')

        self.assertEqual(first=user.username, second='thisisaslug')

    def test_email_gets_converted_to_lowercase(self):

        data = self.data.copy()

        data['email'] = 'EMA...@EXAMPLE.COM'

        form = RegistrationForm(language_code=self.language_code, data=data)

        form.full_clean()

        self.assertTrue(expr=form.is_valid())

        self.assertDictEqual(d1=form.errors, d2={})

        user = form.save()

        email_addresses = UserEmailAddress.objects.filter(user=user)

        email_addresses_set = {e.email for e in email_addresses}

        self.assertSetEqual(set1=email_addresses_set, set2={'ema...@example.com'})

אורי (Uri)

u...@speedy.net

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/3b6d56fd-2664-4b56-8712-e76d51c73172%40googlegroups.com.

[Kye Russell]

Involving email addresses in the discussion complicates things because of the involvement of the email RFC (or RFCs?) which IIRC dictates that the mailbox portion of an email address may be case sensitive. 

Back to OPs point, I feel that you should respect the case of a username for display purposes. I’d merely do a case insensitive / normalisation check for operations like login and conflict checking on signup / username change. As far as email addresses are concerned, this is even more justifiable as you could be dealing with a case sensitive email server. No idea how common that is these days.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CABD5YeFm3A2k7V8zeRPBxyu8zMv7msnmU1ogS_SrCJT76H_Fzw%40mail.gmail.com.

[Nasir Hussain]

What the Standard Says

RFC 5321, the standard that defines how email transport works, lays down the email address case sensitivity issue thus:

The local-part of a mailbox MUST BE treated as case sensitive. Therefore, SMTP implementations MUST take care to preserve the case of mailbox local-parts. In particular, for some hosts, the user "smith" is different from the user "Smith". However, exploiting the case sensitivity of mailbox local-parts impedes interoperability and is discouraged. Mailbox domains follow normal DNS rules and are hence not case sensitive.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CANK-ykkFovkYbB921%2BH5yejZFv6CehLnzmQcqZ4EPKr7WRUjww%40mail.gmail.com.

[אורי‎]

I use Google as my email provider. Is it possible to define with Google case-sensitive email addresses? For example, can I define a mailbox "U...@speedy.net" which is different than my mailbox (u...@speedy.net) and belongs to a different user? I don't think so. And remember, if a user is registered with an email address such as webm...@speedy.net, and then they register again with an email address Webm...@speedy.net, I think in most cases you don't want to treat it as a new email address. Especially since it may happen many times by mistake (for example a user may register and then forget his password, and try to reset his password with another case email address - for example register with Webm...@speedy.net and reset his password with webm...@speedy.net). I think if Google ignores this RFC, then probably Speedy Net can ignore this RFC and it's up to Django to decide what to do with this RFC.

By the way, my name in Hebrew is אורי and there is no such problem in Hebrew - there are no uppercase or lowercase letters. But I'm not sure if I would be able to have an email address with Hebrew letters and receive email from anyone. I don't know anyone who's email address contains a Hebrew letter. I checked now with Chrome and the username part of the email (the local part) can't contain Hebrew letters but the domain part can. So I can register (to Speedy Net, under development) with an email address such as uri@ספידי.נט (but not אורי@speedy.net) and it will be converted to something like "u...@xn--7dblb3ap.xn--debp". If I try to register with אורי@speedy.net I receive an error message from Chrome (and I didn't check other browsers).

"Users who register on mobile don't notice that their phone has automatically capitalised the first letter of their username when they enter it..." - I think a website can disable this feature of capitalizing the first letter in a form, for example in the email address or username field. For example, in Speedy Net we use 'autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"' in the HTML to disable these features.

אורי (Uri) 

u...@speedy.net

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAB5Hk3zUYmYy9BFkFJcb9qWiumEXhY1TiqQQGQn5JS2SPqizJA%40mail.gmail.com.

[Nicolas Le Manchet]

As Kye Russell mentioned, email addresses being case-sensitive for the local part and case-insensitive for the domain part is a whole different problem that has implications way beyond Django. In my opinion, it is best to leave it out of the issue and focus the discussion on usernames.

> > RFC 5321 <https://tools.ietf.org/rfc/rfc5321.txt>, the standard that

> >>>> <https://groups.google.com/d/msgid/django-developers/3b6d56fd-2664-4b56-8712-e76d51c73172%40googlegroups.com?utm_medium=email&utm_source=footer>
> >>>> .

> >>>> For more options, visit https://groups.google.com/d/optout.
> >>>>
> >>> --
> >>> You received this message because you are subscribed to the Google
> >>> Groups "Django developers (Contributions to Django itself)" group.
> >>> To unsubscribe from this group and stop receiving emails from it, send
> >>> an email to django-develop...@googlegroups.com.
> >>> To post to this group, send email to django-d...@googlegroups.com.
> >>> Visit this group at https://groups.google.com/group/django-developers.
> >>> To view this discussion on the web visit
> >>> https://groups.google.com/d/msgid/django-developers/CABD5YeFm3A2k7V8zeRPBxyu8zMv7msnmU1ogS_SrCJT76H_Fzw%40mail.gmail.com

> >>> <https://groups.google.com/d/msgid/django-developers/CABD5YeFm3A2k7V8zeRPBxyu8zMv7msnmU1ogS_SrCJT76H_Fzw%40mail.gmail.com?utm_medium=email&utm_source=footer>
> >>> .

> >>> For more options, visit https://groups.google.com/d/optout.
> >>>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "Django developers (Contributions to Django itself)" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an
> >> email to django-develop...@googlegroups.com.
> >> To post to this group, send email to django-d...@googlegroups.com.
> >> Visit this group at https://groups.google.com/group/django-developers.
> >> To view this discussion on the web visit

> >> https://groups.google.com/d/msgid/django-developers/CANK-ykkFovkYbB921%2BH5yejZFv6CehLnzmQcqZ4EPKr7WRUjww%40mail.gmail.com
> >> <https://groups.google.com/d/msgid/django-developers/CANK-ykkFovkYbB921%2BH5yejZFv6CehLnzmQcqZ4EPKr7WRUjww%40mail.gmail.com?utm_medium=email&utm_source=footer>
> >> .

> >> For more options, visit https://groups.google.com/d/optout.
> >>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django developers (Contributions to Django itself)" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to django-develop...@googlegroups.com.
> > To post to this group, send email to django-d...@googlegroups.com.
> > Visit this group at https://groups.google.com/group/django-developers.
> > To view this discussion on the web visit

> > https://groups.google.com/d/msgid/django-developers/CAB5Hk3zUYmYy9BFkFJcb9qWiumEXhY1TiqQQGQn5JS2SPqizJA%40mail.gmail.com
> > <https://groups.google.com/d/msgid/django-developers/CAB5Hk3zUYmYy9BFkFJcb9qWiumEXhY1TiqQQGQn5JS2SPqizJA%40mail.gmail.com?utm_medium=email&utm_source=footer>
> > .

> > For more options, visit https://groups.google.com/d/optout.
> >
>
> --
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-developers/CABD5YeFwXOFOUgBFBWMyphJDvf45QmFc3K3CER4u7_2Q_i_fsg%40mail.gmail.com.

> For more options, visit https://groups.google.com/d/optout.


--

Nicolas Le Manchet

[Tobias Wiese]

Hello,

On 1/5/19 10:56 PM, Nicolas Le Manchet wrote:
> As Kye Russell mentioned, email addresses being case-sensitive for the local part and case-insensitive for the domain part is a whole different problem that has implications way beyond Django. In my opinion, it is best to leave it out of the issue and focus the discussion on usernames.

There was another Ticket about the email casing #17561 [1].


[1]: https://code.djangoproject.com/ticket/17561

--
Tobias Wiese

[אורי‎]

The "local part" of the email address is the username. But I know not all websites who use Django provide email services. Probably most of them don't.

אורי

u...@speedy.net

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/1546725419.1509232.1626493552.759DCF29%40webmail.messagingengine.com.