auth.user refactor: the profile aproach

[Jacob Kaplan-Moss]

Hi folks --

I've written up a proposal for how *I* would like to address refactoring auth.user: https://gist.github.com/2245327.

In essence, this does two things:

* Vastly "prunes" the required fields on auth.user. The only things left are an "identifier" (which could be username, email, url, uuid, whatever), and a password.

* Introduces a new "profile" system that provides a way to contribute extra related fields. Multiple profiles are supported, along with some syntactic sugar for dealing with multiple profiles in a reasonably reusable way.

And that's about it. I'm deliberately trying to find a middle ground between "do the minimum to allow people to move on" and "throw out and rewrite django.contrib.auth entirely". I'm not expecting everyone to be thrilled by this idea, but I'm hoping that this is "Good Enough" for almost everyone.

For more please see the document. Please do try to read the whole thing: I've had a few rounds of feedback incorporated already, and there's even an FAQ at the end.

I'm not using BDFL fiat here, at least not yet. This is a proposal, and I very much want to hear feedback, objections, and alternatives. I'm particularly interested in hearing from people who've got complicated auth needs and think this absolutely won't work for them. 

I *have* reviewed all the other proposals and I'm between -0 and -1 on all of them. This means that if you don't like my proposal, you'll probably have to come up with a complete *new* idea to have any chance of getting my vote.

Thanks!

Jacob

[Anssi Kääriäinen]

FWIW I am +1 on this proposal. This is assuming that forms, queryset
handling of the data and so on actually work correctly and nicely. I
don't see a blocker in those areas, but before the code is actually
written and tested it is impossible to say if there will be problems.
Devil is in the details and all that.

I hope this will not result in hacks to allow .filter(data__) and so
on. I believe we should aim for as generic solutions as possible. If
we do hacks just for the User model, they _are_ going to cause
problems later on. Most of the features needed would be neat to have
regardless of the User refactor. Custom lookups per model/field for
example would be _really_ neat feature. The downside of aiming for
generic solutions is that we once again get into the situation where
User refactor is blocked by implementing foo, bar and baz before it.

I believe this is the right way forward. Multiple profiles makes sense
in the SQL world, and in the NoSQL world you would just embed the
profiles into the User document. I don't believe performance will be a
problem in practice, at least as long as you don't add too many
different profiles to be loaded by default. The most common use case
will just get better support: you have user profile containing three
fields, and then you have all your local fields defined in one
profile. As now, you have profile + user, now they just model your
data properly.

I don't see a proposal of how __unicode__ is going to be handled (the
Hello, UserName requirement). My proposal: go through the profiles in
the order they are defined, and pick the first one having __unicode__
defined as the default __unicode__. If there is no such profile,
return identifier.

- Anssi

[Donald Stufft]

If i recall on IRC the decider was to just create a display field (e.g. user.data["display"]) that the default profiles can provide (and can be overridden by other profiles of course).

--

You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-d...@googlegroups.com.

To unsubscribe from this group, send email to django-develop...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Anssi Kääriäinen]

On Apr 3, 4:20 am, Donald Stufft <donald.stu...@gmail.com> wrote:
> If i recall on IRC the decider was to just create a display field (e.g. user.data["display"]) that the default profiles can provide (and can be overridden by other profiles of course).

My problem with this is that for example where I work the display
field would contain '%s, %s (%s)' % (self.lastname, self.firstname,
self.empl_no). I would not like to do that data duplication. If the
'display' can be a property then fine. But why not go directly for
__unicode__ in that case?

- Anssi

[Donald Stufft]

If we use __unicode__ (which i'm fine with) then it needs to follow the same resolution path as user.data[] does.

[Alex Ogier]

I realize that arguing with a BDFL might get me nowhere, but I don't think that multi-profile + select_related + proxy attributes on the user model is the proper approach for users going forward. The proposal makes some basic sense as an incremental improvement on the current status quo of a built-in user with fixed 30-character identifier and single one-to-one profile. But in the larger scheme of things it moves further in the wrong direction, in my opinion, towards a model that addresses people's specific 80% questions (email-as-identifier, packageable extra fields) but not the general best-of-all-worlds mechanisms.

Consider the other batteries-included frameworks out there. I think the three biggest ones are probably Ruby on Rails, ASP.NET MVC, and Play Framework. Something all three have in common, something I think would be valuable for Django, is that their User identity is absent, and authentication and authorization modules are added to a developer-created User model ad-hoc. I think it's fair to say, the ecosystem of third-party authentication mechanisms surrounding all three of these competitor frameworks is MUCH healthier than Django's, for the simple reason that it is much more flexible and sane to define your own user and plug in an authentication module then it is to plug authentication into a fixed user model that magically proxies back (even one as simple as a varchar identity + password).

The basic idea is that "authentication" is something that can be provided for any model you like. When a user authenticates, they are providing some sort of authentication token to your project that proves they are who they say they are, and as a developer you are free to attach this authentication token to *whatever you like,* not only contrib.auth.models.User. There is a rich ecosystem of third party authentication apps for our competitor frameworks that all work on this principle. They can provide authentication flows that bounce the user back and forth to twitter.com, their oauth provider, browserid, etc. and it is precisely because they don't demand anything from a central object. Even something as simple as "all user.identifiers that start with 'oauth2$' belong to our auth mechanism" starts to open all sorts of security holes. What if someone wants to authenticate by email+password OR by third party proof of email ownership (my university has a mechanism like this)? Well, if someone can manage to register an account belonging to 'aog...@princeton.edu', maybe by triggering some obscure email change recovery form wizard or something, then suddenly they possess my account on Django.

This is why I think the only sane long-term solution involves distinct and pluggable authentication modules, and a concept of users that doesn't enforce any brand of identity. The second stipulation is very important for social reasons, if only to ensure that the path of least resistance for third-party authentication doesn't involve trying to overload identity mechanisms for disparate and incompatible purposes.

JKM, you seem concerned that the notion of pluggable Users by necessity involves magical settings.

I'm convinced that such an idea is ultimately a bad idea: it allows apps exert action at a distance over other apps. It would allow the idea of a user to completely change without any warning simply by modifying a setting.

I am not convinced that this must be so. Asking a developer to write their own User model is not the same thing as automagically reshaping contrib.auth.models.User based on settings. A developer-defined notion of identity is a thing that should be codified in software by constructing a model. This community seems fixed on the idea that whatever model the developer comes up with to satisfy the bizarre constraints of his particular website, it must eventually be mounted at auth.User lest the world come crashing down as foreign keys break, middleware throws exceptions and California slides into the pacific ocean.

Nearly every other framework out there does this the opposite way: to authenticate with a third-party service, you add fields to your identity model to support whatever credentials are necessary. Not the other way round, where auth modules define models that have OneToOneFields to auth.User that uses an AUTH_PROFILES setting to magically proxy back.

I get that Django's core is very accustomed to the relational database mode of thinking: "If a User might own a Twitter handle, then let's create a table of twitter handles in the twitter-auth app, and foreign key back to the default User model". It's really not that bad to go the other way for a moment, and say "If a User might own a Twitter handle, then let's add a twitter_handle field to Users." The reason being that the latter is *so much more flexible*. You can simulate the first with the second, but not vice versa. Twitter-auth might not need its own table (in fact, it shouldn't need its own table). If you really wanted to, you could make one, and foreign-key *from* the user model which gives you everything the first solution has, with no need to created magical .prof1, .prof2 proxy attributes. You could even let users sign in with multiple handles with a many-to-many. Heck, maybe your blog posts have their own twitter credentials, I don't know.

So here's the short version of my proposal, which I plan on writing up in full tomorrow: Break auth as it stands into reusable pieces. Let people write their own user models, optionally using those pieces. Provide straightforward settings for any contrib apps that absolutely *must* have a specific model to key on (ideally none, but comments and admin probably need shims). Document the new wave of best practices, borrowed straight from the frameworks that got this right from the start: Write your own user, decide what identity means (and hence, what your login forms look like), add in whatever authentication mechanisms you like, add in whatever authorization mechanisms you need (with specific instructions on what contrib.admin demands from your model), and run with that.

Sorry for the rant, hopefully I'm not burning too many bridges,

Alex Ogier

[Carl Meyer]

On 04/02/2012 06:35 PM, Jacob Kaplan-Moss wrote:
> I've written up a proposal for how *I* would like to address refactoring
> auth.user: https://gist.github.com/2245327.

+1 from me.

One minorish nit: I think that "in the face of ambiguity, refuse the
temptation to guess" should apply equally to reading or writing
user.data, and that there shouldn't be an awkward discrepancy between
reading and writing. Thus, if you've got multiple profiles with
overlapping attributes (the less-common case), you need to be explicit
with AUTH_PROFILES in order to either read or write the overlapping keys
via user.data (but writing is then allowed). In the common case of no
overlapping keys, of course, you don't have to worry about AUTH_PROFILES.

Carl

[Russell Keith-Magee]

On 03/04/2012, at 8:35 AM, Jacob Kaplan-Moss wrote:

> Hi folks --
>
> I've written up a proposal for how *I* would like to address refactoring auth.user: https://gist.github.com/2245327.
>
> In essence, this does two things:
>
> * Vastly "prunes" the required fields on auth.user. The only things left are an "identifier" (which could be username, email, url, uuid, whatever), and a password.
> * Introduces a new "profile" system that provides a way to contribute extra related fields. Multiple profiles are supported, along with some syntactic sugar for dealing with multiple profiles in a reasonably reusable way.
>
> And that's about it. I'm deliberately trying to find a middle ground between "do the minimum to allow people to move on" and "throw out and rewrite django.contrib.auth entirely". I'm not expecting everyone to be thrilled by this idea, but I'm hoping that this is "Good Enough" for almost everyone.
>
> For more please see the document. Please do try to read the whole thing: I've had a few rounds of feedback incorporated already, and there's even an FAQ at the end.

I've added a summary of this option to the wiki:

https://code.djangoproject.com/wiki/ContribAuthImprovements#Solution5:Profile-basedsingleusermodel

As always, feel free to correct/update as necessary.

From my reading of the proposal, here are some questions/edge cases. For some of these questions, I fully expect the answer may be "You just can't do that"; however, given that they're plausible edge cases, it's worth being explicit about what we're aiming at.

* Auto-profile creation:

What if my profile is:

class MyProfile(Profile):
first_name = CharField(max_length=100)
last_name = CharField(max_length=100)

i.e., first_name and last_name are both required fields. How does the profile get automatically instantiated in this case? Doesn't the auto-instantiation of profiles essentially mean that there can be no required fields on a profile (or, at least, on an auto-instantiated profile)?

* Regarding AUTH_PROFILE and collisions:

What if I have 2+ profiles, but no clear order of precedence? e.g.,

class MyProfile1(Profile):
name = CharField()
email = EmailField()
color = CharField()

class MyProfile2(Profile):
name = CharField(unique=True)
email = EmailField()
age = IntegerField()

Lets say that for some internal logic reason for profile processing, I need both email and age on MyProfile2, but name from MyProfile1. Under these circumstances, there's no way I can specify a single AUTH_PROFILE ordering that will satisfy my requirements. Is this a case where I have to resort to explicit naming?

Or, alternatively, should we be treating AUTH_PROFILE as a mechanism for purely specifying the resolution to specific lookup problems? i.e., instead of just specifying an order, we specify which model we want ambiguous fields to come from:

AUTH_PROFILE_LOOKUP = {
'name': 'myapp1.MyProfile1',
'email': 'myapp1.MyProfile2',
'color': 'myapp1.MyProfile2',
}

That way, if a field is unambiguous, it's returned from whatever model provides it; if a field is ambiguous, we return the one specified; and if the ambiguity isn't resolved, we return a KeyError (or catch this case as a validation error on startup).

* Required fields and AUTH_PROFILE

Combining the previous two points; what if MyProfile2 has a required field, but it isn't a field selected by data[]? e.g, in the previous example, MyProfile2.name is a required unique field; but if it isn't instantiated with useful data, the profile instances can't exist.

== Commentary ==

For me, the previous three edge cases essentially point to the fact that there can ultimately only be 1 profile for "core" user information. There will almost certainly be uses for multiple profiles (e.g., to store OpenID credentials) -- but then, this is already the case (I've got more than one project in the wild with multiple "UserProfile" objects, without using AUTH_USER_PROFILE). However, as soon as there are overlaps between profiles, you're going to end up with partially populated profiles, and eventually someone will write a 'unified' profile model for each project that doesn't have the overlap ...

... at which point, we've essentially arrived at a swappable User model, just with a required join to get profile data, and a bunch of magic attributes and ORM properties to hide the fact that the join exists.

I understand Django's history has examples where swappable models caused problems that we don't want to revisit. However, as I understand it, these problems were largely caused by the fact that it was *easy* to accidentally swap out a model, and that change wouldn't be effectively communicated to other developers on the same codebase until the magic smoke escaped.

It's also worth pointing out that the problems with swappable models aren't really avoided by the profile-based approach. If I've got a project with N user profile models, grabbing user.data['name'] from one of them, another developer can very easily modify the order of AUTH_PROFILES, or modify the apps in INSTALLED_APPS to add to or change the user profiles that are currently in effect, or alter the precedence in AUTH_PROFILES to override the attribute we need. If the objection to swappable user models is that it's easy to swap out models, then it's just as easy to change the expectations for User models via a profile -- possibly easier, because you aren't modifying a single "AUTH_USER_MODEL" setting; you're changing one of several settings whose consequences *may* include changing the operation of the User model. AUTH_USER_MODEL at least has the consequences written on the box.

(User models are hard. Lets go shopping. :-)

One possible way to avoid this problem would be to make it harder to make these sorts of changes -- i.e., add project/install-level tracking for certain key settings. So, when you do a syncdb, we set a key-value pair in the database for the current value of AUTH_USER_MODEL (or whatever mechanism needs to be persisted). Django checks this value against the current settings.AUTH_USER_MODEL value on startup; if the setting value doesn't match the installation value, we can throw an error on startup.

Would this approach temper your objection to swappable models?

At the end of the day, I'm happy to go with a BDFL judgement on this. However, it's interesting to note that almost every time this problem arises, a "swappable" model of some sort is usually the initially proposed solution. That's because it's an obvious solution -- an obvious solution with problems, but an obvious solution, nonetheless. The alternative being proposed here isn't *clearly* better -- it's really just a matter of making a strategic choice to pick a different set of pain points.

To make matters works, it seems to me that it will require a lot more work to execute -- far from being "minimalist", it's invasive on auth code, ORM code, and (in the long term) the internals of every app that uses User.

For example, any code written against the fields on the existing auth.User will need to be updated to use the new data[] access mechanism. Yes, this will be introduced gradually over releases, but it doesn't change the fact that every app that currently accesses user.is_staff will need to update to user.data['is_staff'] at some point in the next few releases. By comparison, the swappable user approach generally means no changes to app code (or just modifying ForeignKey(User) to LazyForeignKey(User), or ForeignKey(settings.USER_MODEL)), and documenting that is_staff is a required attribute on User for using the app (a property that can be validated as a project integration requirement by a test suite).

There are also some aspects about this proposal that make my teeth itch -- filter(data__...), for example, strikes me as code that will be hard to write, hard to debug, and a PITA to maintain. And if you can filter(data__...), then you need to be able to order_by('data__...') as well. But all this complexity is required if you want to maintain decoupling of profiles. It also requires adding a feature to the ORM to support a specific use case in User, rather than a generic problem (although I suppose we could implement it in such a way as to make "profiles" an option for any other model that needs them).

If we're going to make a choice to avoid the "obvious" solution, I think we need to be very clear (both now, and for posterity) why we made the choices we have made, rather than trying to address or ameliorate the problems with the "obvious" solution.

Yours
Russ Magee %-)

[Donald Stufft]

--

You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Identity doesn't have anything to do with automatically dispatching users. All it is is a unique identifier. That's all this proposal honestly enforces that your users have. Some single piece of identifiable data that can be used to differentiate. This could be a username, or an email address. It could be a random string. Anything you want.

In your example you might have a TwitterProfile that provides 2 fields, "authenticated_by_twitter" and "twitter username". Then if you want to check how a person authenticated to your site, you'd merely check if user.data["authenticated_by_twitter"] was True. The identifier doesn't need to have that data codified in it, (but it could!) and I honestly do not think the statement "all users must have 1 single string of any length that uniquely identifies them" is that big of a burden.

[Alex Ogier]

On Mon, Apr 2, 2012 at 11:04 PM, Donald Stufft <donald...@gmail.com> wrote:

Identity doesn't have anything to do with automatically dispatching users. All it is is a unique identifier. That's all this proposal honestly enforces that your users have. Some single piece of identifiable data that can be used to differentiate. This could be a username, or an email address. It could be a random string. Anything you want.

In your example you might have a TwitterProfile that provides 2 fields, "authenticated_by_twitter" and "twitter username". Then if you want to check how a person authenticated to your site, you'd merely check if user.data["authenticated_by_twitter"] was True. The identifier doesn't need to have that data codified in it, (but it could!) and I honestly do not think the statement "all users must have 1 single string of any length that uniquely identifies them" is that big of a burden.

Perhaps I am more pessimistic than you, but I think it will quickly get out of hand. I mean, what reasonable developer would look at the user model as Jacob proposes, and wouldn't at least consider, "Well, I could make my own profile and ask everyone to add my app to AUTH_PROFILES and be magically joined on all User queries, or I could use the arbitrary length guaranteed-unique varchar to encode a whole mess of twitter credentials. I think I will choose option #2?"

Then as soon as everyone does that and facebook, twitter, browserid, and plain emails all share the same namespace, we open up the same whole can of worms that we get with cache keys, except now failures to manage things properly manifest themselves as security holes in basic authentication instead of cache collisions.

Best,

Alex Ogier

[Tai Lee]

I like this proposal because I am a big fan of a stripped down `User` model which is basically just an ID, whic provides a common hook into groups/permissions and other django and 3rd party profiles.

What I don't like is the magical `data` bag (accessing `User.data` and filter lookups), the new `AUTH_PROFILES` setting, and the idea of Django automagically or lazily creating profiles for me. I would rather see Django rely solely on its `OneToOneField` field to access user profile data.

Any pluggable app (app1) will know what fields it's own profile model has and how to access them, relative to the central `User` object.

If app1 relies on fields defined by another app (app2), that other app should be a requirement of app1 and app1 would know how to access app2's fields.

I am happy to use project-level signals for everything else (syncing common profile data from app1 and app2, or auto-creating user profiles), because project-level signals give me explicit control over what is going to happen and when. I don't need any more magic here.

Cheers.
Tai.

[Donald Stufft]

On Tuesday, April 3, 2012 at 3:37 AM, Tai Lee wrote:

I like this proposal because I am a big fan of a stripped down `User` model which is basically just an ID, whic provides a common hook into groups/permissions and other django and 3rd party profiles.

What I don't like is the magical `data` bag (accessing `User.data` and filter lookups), the new `AUTH_PROFILES` setting, and the idea of Django automagically or lazily creating profiles for me. I would rather see Django rely solely on its `OneToOneField` field to access user profile data.

Any pluggable app (app1) will know what fields it's own profile model has and how to access them, relative to the central `User` object.

If app1 relies on fields defined by another app (app2), that other app should be a requirement of app1 and app1 would know how to access app2's fields.

I am happy to use project-level signals for everything else (syncing common profile data from app1 and app2, or auto-creating user profiles), because project-level signals give me explicit control over what is going to happen and when. I don't need any more magic here.

Cheers.
Tai.

Without the `data` bag we've done nothing except rename the username field and make it longer really. Everything else that you define can already be achieved by ignoring the fields already on User and using OneToOneFields. With the `data` bag we allow a pattern kinda similar to class inheritance.

I think you also mistake the use case, it's not app1 relies on app2, it's app1 relies on _a_ field called email that provides an email address. It doesn't care where it comes from. If we do not provide a common way to access this data with this proposal, than a large number of apps are going to depend on the default profile, and we are stuck again here where everything depends on what Django Core decides for "first_name", "last_name", email length, etc.

On Tuesday, 3 April 2012 10:35:41 UTC+10, Jacob Kaplan-Moss wrote:

Hi folks --

I've written up a proposal for how *I* would like to address refactoring auth.user: https://gist.github.com/2245327.

In essence, this does two things:

* Vastly "prunes" the required fields on auth.user. The only things left are an "identifier" (which could be username, email, url, uuid, whatever), and a password.

* Introduces a new "profile" system that provides a way to contribute extra related fields. Multiple profiles are supported, along with some syntactic sugar for dealing with multiple profiles in a reasonably reusable way.

And that's about it. I'm deliberately trying to find a middle ground between "do the minimum to allow people to move on" and "throw out and rewrite django.contrib.auth entirely". I'm not expecting everyone to be thrilled by this idea, but I'm hoping that this is "Good Enough" for almost everyone.

For more please see the document. Please do try to read the whole thing: I've had a few rounds of feedback incorporated already, and there's even an FAQ at the end.

I'm not using BDFL fiat here, at least not yet. This is a proposal, and I very much want to hear feedback, objections, and alternatives. I'm particularly interested in hearing from people who've got complicated auth needs and think this absolutely won't work for them. 

I *have* reviewed all the other proposals and I'm between -0 and -1 on all of them. This means that if you don't like my proposal, you'll probably have to come up with a complete *new* idea to have any chance of getting my vote.

Thanks!

Jacob

--

You received this message because you are subscribed to the Google Groups "Django developers" group.

To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/PMpcPCKgTuoJ.

[Bruno Renié]

Hi Jacob,

Thanks for taking the time to tackle this issue!

On Tue, Apr 3, 2012 at 2:35 AM, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:
> Hi folks --
>
> I've written up a proposal for how *I* would like to address refactoring
> auth.user: https://gist.github.com/2245327.
>
> In essence, this does two things:
>
> * Vastly "prunes" the required fields on auth.user. The only things left are
> an "identifier" (which could be username, email, url, uuid, whatever), and a
> password.
> * Introduces a new "profile" system that provides a way to contribute extra
> related fields. Multiple profiles are supported, along with some syntactic
> sugar for dealing with multiple profiles in a reasonably reusable way.

I very much like the idea of stripping down the user fields. I am
currently using an approach that's very similar to your proposal in
all my projects, except that instead of creating several profiles, I
directly add fields to the User model using this snippet:

https://gist.github.com/1283969 (I am not the initial author)

With this, adding a field to the user model is as simple as:

class MyUser(models.Model):
newsletter = models.BooleanField()

class Meta:
abstract = True
contribute_to_model(MyUser, User)

I already use this to patch the email field in the User model or make
group names not unique for instance.

The only issue is that the contributed models need to be registered as
early as possible. Currently I import them in my urls.py, but we can
imagine adding hooks for such things that need to be done at startup
time (app-refactor? :) or use your AUTH_PROFILES setting for this (not
enthusiastic about this one).

I am fully aware that this can be abused a lot. However this makes a
couple of things simpler:

* No need to proxy fields on the User model, it stays a normal model
with no special treatment
* Conflicts are simply resolved: if two models contribute the same
field, the last one registered wins
* No need for a special-cased form for editing profile-related fields,
a simple ModelForm with includes = ['fields', 'to', 'include'] works.

This is the technique I will use anyway if your proposal is
implemented, and I don't see anything that would prevent this from
working with the new auth so I'm mainly posting this because I haven't
seen this approach in the different proposals for the auth refactor.

-Bruno

[Alex Ogier]

Hi developers,

I have written up a little bit about the alternate proposal that I made a while ago, Solution 2a from https://code.djangoproject.com/wiki/ContribAuthImprovements

In addition to other arguments, there is a point-by-point breakdown of what I feel are the weaknesses in Jacob's proposal.

You can find it at  https://gist.github.com/2289395 

Best,

Alex Ogier

Jacob

--

You received this message because you are subscribed to the Google Groups "Django developers" group.

[Andre Terra]

DISCLAIMER: I'm just a hobbyist posting to an experienced developers' list.

I for one really like Jacob's approach. 

I may be missing something, but my only concern right now is being able to infer which profiles are attached to the User model at any given point. Moreover, I'd like to be able to catch that KeyError exception (which I would prefer to be wrapped in something more specific like "MultipleDataError") and handle it appropriately.

For example, drawing from the example in the gist, say I want to allow the user to change his 'name' for every associated profile he has. user.data.dict('name').keys() will show me the existing profiles, and I guess I could use these keys to access every profile... but it doesn't seem like the ideal solution. 

I can see the need for something like user.data.update({'name': 'Joe K. Smith'}) to forcefully update the name in every profile. In my humble newbie opinion, there's a lot of syntactic sugar to be added here to make working with multiple profiles as sweet as we want it to be. Things like user.profiles, or even defining a 'default' profile that point how MultipleDataErrors should be resolved.

Cheers,

AT

[Luke Sneeringer]

So, after reading this, I think I really only have a couple questions/concerns:

1. What, if anything, is the primary key on the base User model? Is it the identifier? If so, can it be specified as the primary key, rather than unique=True, db_index=True? If it's not the primary key, is it the usual magical id = models.AutoField()?

2. I'm a little nervous about the magical installation of Profile modules -- specifically, what happens once this system has been around for one to two years and we have a ton of these? I'm imagining a situation where there are lots of apps floating around, each with their own Profile subclasses that install behind the scenes when you install the app (and which you may or may not actually realize up front -- yes, a good developer *should* be aware of that, but that doesn't mean they will be...). What happens if a beginner writes everything against user.data['name'], and then installs an app that adds a new name, which is unordered and magically ends up on top, and therefore introduces new, unexpected, and, most importantly, extremely difficult to debug logic errors?

I agree with some of the criticisms of swappable user models, but at least the developer always knows what he is doing in that case. In this case, the behavior of your user model can change in unexpected ways when you do something that seems totally unrelated; some code that previously worked might either (1) return different data or (2) suddenly raise KeyError.

Perhaps this is the best of bad options. I agree there's a decent bit to be argued in favor of it, but can we figure out a less error-prone way for user.data to work? Honestly, I think it would be less risky to force everyone to write user.admin_profile.name -- it's more typing, but at least it's explicit. The current proposal for user.data doesn't actually pass "refuse the temptation to guess", because it glosses over the fact that developers *will guess* and then their code will stop working after what feels like a completely unrelated change.

3. Might I suggest that the Profile model that the admin requires be called AdminProfile and not DefaultProfile? It's really an admin need more than anything else.

Best Regard,

Luke

[Carl Meyer]

On 04/03/2012 08:28 AM, Alex Ogier wrote:
> I have written up a little bit about the alternate proposal that I made
> a while ago, Solution 2a
> from https://code.djangoproject.com/wiki/ContribAuthImprovements
>
> In addition to other arguments, there is a point-by-point breakdown of
> what I feel are the weaknesses in Jacob's proposal.
>
> You can find it at https://gist.github.com/2289395

Since I weighed in with a +1 on the linked-profiles approach, I should
say that I also find this proposal pretty compelling, and would be happy
to see us go either way. After reflecting on it a bit I think on balance
I prefer this proposal, for the following reasons:

1. Requires zero migration from people with existing projects who are
happy with the existing contrib.auth.User model. I think this is a big
plus: all the other proposals require every single installed Django
codebase to perform a migration.

2. Introduces fewer new-and-different one-off concepts into Django (the
data__ ORM special case, the automatic select_related special case, the
automatic collection of profile fields into user.data...). This also
translates into significantly less implementation work and less new
code, which is another big plus - we want something that'll actually get
done and not introduce a bunch of new code we need to maintain.

By the way, I took the liberty of removing from the wiki page the
references to models/settings circular dependencies, because AFAIK the
statements made about it on the wiki page were simply incorrect.
Importing settings does _not_ immediately loop through INSTALLED_APPS
and load every models file (that only happens if you run a management
command that performs model validation, and even then it happens only
after settings are fully loaded). And "from django.db import models"
itself imports settings, so if there were such a circular dependency
problem, every models.py in existence would already be suffering from
it. I frequently refer to settings in models.py and it does not cause a
problem. If anyone can provide sample code demonstrating that this is
actually a problem, feel free to correct me!

Carl

[Adrian Holovaty]

On Tue, Apr 3, 2012 at 9:28 AM, Alex Ogier <alex....@gmail.com> wrote:
> I have written up a little bit about the alternate proposal that I made a
> while ago, Solution 2a
> from https://code.djangoproject.com/wiki/ContribAuthImprovements

I just now got around to reading Jacob's solution and Alex's solution.
Thanks to everybody for the thoughts and impassioned debate so far.
Here's my take on it.

First, some background: I haven't used the built-in User module in
several years. I always write my own User model from scratch -- it's
so nice and clean. Want a twitter_username field? Just add it. No need
to add a convoluted foreign key or (oh god) one-to-one relationship to
some other table.

To me, this is the Right Way to do things. The framework should bend
to my needs, I shouldn't bend to the framework's needs.

Also, profile modules need to die. They needed to die circa 2006.

So, with that in mind, I've got to say I prefer Alex's solution. I
really think the right way to do it is:

1. Let you create your own User model, with whichever fields you want.

2. Provide a way to tell Django which model you're using for that.

3. In your own code, just deal with that model like you deal with any
other one. No need to jump through hoops.

4. Third-party models should be changed to use something like "user =
UserField()", which would automatically create a foreign key to the
registered User model. If you change your registered User model after
you've created those third-party tables, you're in for trouble. Don't
do that.

5. Django provides some generic APIs for getting at the registered
user. Example: a middleware that sets request.user based on the
current session.

6. Given that some third-party apps will likely want to get access to
common attributes of a User -- notably, email address -- there could
be some sort of standard interface that User models need to adhere to
(duck typing). So, a custom User model would say "for this User model,
the email address is stored in the database field called 'email'" --
or "this User model doesn't have email addresses."

I chatted about this with Jacob on IRC, and we reached consensus on
this approach. I'd like to get moving on this and would be happy to
take it on myself, starting next week.

Adrian

[Daniel Sokolowski]

Alex I have looked over your proposal and I agree on both your concerns with LFK, and Jacob’s approach; however I still find the profile approach is the most flexible solution.

 

Correct me if I’m wrong but both LFK or a swappable user model approach like your fail to address issue of extensibility. If today my project is authorizing with username and password and tomorrow I wish to add OpenAuth then my User model is replaced, whereas with Jacobs approach I add another profile model, yes? What about auth apps, you could only use one, with profiles many could co exist; one for Facebook, Twitter, etc.

 

Your point 4 claiming it’s undiscoverable is not entirely true, AUTH_PROFILE setting can be examined just as apps examine INSTALLED_APPS settings at run time. Your point 2 that being able to change the functionality so quickly is actually superior in my mind, it’s a lot more work to do a schema migration then the creation a of  new profile table.  Points 5 and 3 are good points and remind that there is no perfect solution to new auth.

 

To sum up I am curious to know how your approach handles additional user data and authorization schemes.

Alex Ogier

To unsubscribe from this group, send email to mailto:django-developers%2Bunsu...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Donald Stufft]

Like Carl I was +1 on Profiles and I'm now leaning towards the Swappable User Models.

It's explicit (it only changes when you change the USER_MODEL setting).

It's Duck Typing which is Idiomatic in Python. ("This app depends on a user model that defines ``email`").

If you wish to add OpenID you'd make your UserModel quack like an OpenID Duck, as well as Quack like a Username/Password Duck.

[Carl Meyer]

On 04/03/2012 02:34 PM, Daniel Sokolowski wrote:
> Correct me if I’m wrong but both LFK or a swappable user model approach
> like your fail to address issue of extensibility. If today my project is
> authorizing with username and password and tomorrow I wish to add
> OpenAuth then my User model is replaced, whereas with Jacobs approach I
> add another profile model, yes? What about auth apps, you could only use
> one, with profiles many could co exist; one for Facebook, Twitter, etc.

You would add whatever authorization fields you need to your single user
model. If you add new kinds of authorization that require new fields on
the user model, you'd do a schema migration (just like with any other
modification to any other model).

A third-party app that wants to provide, say, Facebook integration,
could either do its own model with a one-to-one to user (nothing is
going to suddenly prevent that approach, it just won't have any extra
sugar around it), or could provide an abstract base class that your user
model can inherit from that includes the needed fields, or could just
document "to use this app, your user model must have these fields: ..."
and let you do it yourself.

Carl

[Luke Sneeringer]

+1. Also, I'd like to help. :)

L

[Alex Ogier]

Thanks for the response, Adrian.

 

4. Third-party models should be changed to use something like "user =
UserField()", which would automatically create a foreign key to the
registered User model. If you change your registered User model after
you've created those third-party tables, you're in for trouble. Don't
do that.

5. Django provides some generic APIs for getting at the registered
user. Example: a middleware that sets request.user based on the
current session.

 

I think convenience methods are nice and all, but we should be careful of making "The Registered User(tm)" too much of a baked-in concept. Ian Lewis made a great point a few weeks ago: There are domains in which it is frankly inappropriate for various authenticated users to mix in the same database table with the same uniqueness constraints. Maybe you run a medical record management site, and some of your users are administrative staff and doctors, and some are patients. Sure, you could manage this by having a unified model containing everything, and carefully validating with an "is_doctor" field the way contrib.admin handles "is_admin," but it might be more appropriate to have separate models. It becomes much easier to express authorization constraints such as, "As a hospital administrator, Ms. Sally Johnson is authorized to view the patient list of Dr. John Smith, but *absolutely not* his record as a patient himself," or "Patients like Dr. John Smith can be authenticated via Facebook Connect, but doctors need to be authenticated via our internal LDAP." You can have login ModelForms for doctors that log in to doctor records and the Django admin site, and login ModelForms for patients that let them view their own records through the frontend.

 

Obviously supporting multiple user models isn't something that should be a priority for Django, but locking people out of the choice seems unwise, when we can get the option "for free": just let authentication backends return any authenticated object at all to add as request.user. Python's duck typing will get you the rest of the way, except for those persnickety tables that absolutely must have a specific concrete table to foreignkey on. We already do something similar for AnonymousUsers, exposing a common interface at request.user that isn't backed up by the canonical User table. Generalizing that is A Good Idea in my opinion.

 

Anyways I look forward to seeing how this progresses. I'd love to be a part of making this happen. Would you like me to turn my work on breaking up auth.User into reusable chunks into a ticket on trac? I think it makes sense to merge no matter what we decide on for specific settings and access mechanisms to developer-defined users.

 

Best,

Alex Ogier

[Russell Keith-Magee]

On 04/04/2012, at 4:31 AM, Carl Meyer wrote:
>
> By the way, I took the liberty of removing from the wiki page the
> references to models/settings circular dependencies, because AFAIK the
> statements made about it on the wiki page were simply incorrect.
> Importing settings does _not_ immediately loop through INSTALLED_APPS
> and load every models file (that only happens if you run a management
> command that performs model validation, and even then it happens only
> after settings are fully loaded). And "from django.db import models"
> itself imports settings, so if there were such a circular dependency
> problem, every models.py in existence would already be suffering from
> it. I frequently refer to settings in models.py and it does not cause a
> problem. If anyone can provide sample code demonstrating that this is
> actually a problem, feel free to correct me!

Those comments were my doing. I swear that at some point in Django's past, importing settings in models was a source of problems. However, the fact that django.db.models clearly imports settings indicates that regardless of whether this problem existed in the past, it certainly isn't the case any more.

Yours,
Russ Magee %-)

[Russell Keith-Magee]

Extending on this -- User model requirements should be encoded as part of an integration test suite.

As an example of best practice, django.contrib.admin should implement a UserPropertiesTest that checks that the current USER_MODEL has an email field, and a get_permissions method, and so on. That way, if you swap in a user model that doesn't have the properties that the admin requires, it will get flagged as a issue during testing.

This should, of course, be backed up with a clear documentation of exactly what type of User-duck Django's admin requires.

Yours,
Russ Magee %-)

[Russell Keith-Magee]

On 04/04/2012, at 4:34 AM, Adrian Holovaty wrote:

> On Tue, Apr 3, 2012 at 9:28 AM, Alex Ogier <alex....@gmail.com> wrote:
>> I have written up a little bit about the alternate proposal that I made a
>> while ago, Solution 2a
>> from https://code.djangoproject.com/wiki/ContribAuthImprovements
>

> ...

> 4. Third-party models should be changed to use something like "user =
> UserField()", which would automatically create a foreign key to the
> registered User model. If you change your registered User model after
> you've created those third-party tables, you're in for trouble. Don't
> do that.
>

My only concern about this approach is making a special case of swappable Users. There's at least one other example of a swappable model in Django's core -- the Comments model -- and I have no doubt that we could find other examples with a quick survey.

If we're going down the path of swappable models, I'd rather set up the infrastructure to solve the *general* problem of swappable models, rather than just swappable *user* models specifically.

If this required a massive increase in the effort required, I can see how practicality would determine that we just solve the immediate problem. However, in this case, it's really just a matter of avoiding User-specific naming on 4 features:

a) Specifying the model that can be swapped.

This is just the name of the setting. USER_MODEL is fine by itself, and has obvious analogs with other models. However, an alternate way of spelling the same thing would be to have a single setting for all swappable models:

SWAPPABLE_MODELS = {
'user': 'myauth.SuperDuperUser',
'comment': 'comments.Comment',
}

in which you can define any extensible model, and provide a convenient key (e.g., 'user') to identify the value for that model.

b) Preventing the "default" model from being added to the app cache, or synchronized to the database

contrib.auth is going to need ship with a User model, but we really don't want an auth_user table to be created, or get_model(auth,'User') to resolve, if the default User isn't being used. Same goes for any other swappable model.

Rather than make a special case of User inside syncdb, or nesting model definitions in if blocks, lets add a "swappable" attribute to Meta. If swappable is defined, syncdb checks the contents of SWAPPABLE_MODELS to see if this model has been substituted; if it has, then the table isn't created, and the model isn't added to the app cache. The value of pluggable can match the key in SWAPPABLE_MODELS (so auth.User would define swappable='user', following the previous example)

This can also act as a safety mechanism; if a developer has an app that contains a ForeignKey(User), and the User model has been swapped out, this can now raise a validation warning (notifying the developer that the User model could potentially be swapped out) or error (in the case where the model *has* been swapped out).

c) Providing a way to specify foreign keys to that model

Rather than introduce a User-specific UserField(), introduce a generic LazyForeignKey('user') - both of which are effectively just ForeignKey(settings.XXXX) anyway.

d) Provide a way to discover the current user model

Rather than have auth.get_user_model(), leverage the existing app cache. We already have get_model(); it wouldn't be too hard to add a get_swappable_model('user').

> 6. Given that some third-party apps will likely want to get access to
> common attributes of a User -- notably, email address -- there could
> be some sort of standard interface that User models need to adhere to
> (duck typing). So, a custom User model would say "for this User model,
> the email address is stored in the database field called 'email'" --
> or "this User model doesn't have email addresses."
>

As I've noted elsewhere, this can be both a documentation issue, and backed up by an integration test. As a longer term goal, if someone ever takes the bait and does the Validation refactor for the GSoC (hint hint), it could also be extracted as a validation condition for an app.

Yours,
Russ Magee %-)

[Eric Florenzano]

I completely agree with Adrian's proposal, with these amendments suggested by Russell in order to build something slightly more generic, like LazyForeignKey, instead of UserField.  In fact, a UserField coud end up being provided as a subclass of LazyForeignKey with a default name provided (e.g. 'user') and any other user-domain-specific customizations needed.

Thanks,

Eric Florenzano

[bhuztez]

django.contrib.auth.get_user already uses BACKEND_SESSION_KEY to
decide from which backend to get user object.

To support multiple user models, we can just change the
AUTHENTICATION_BACKENDS setting to a dict just as DATABASES.

AUTHENTICATION_BACKENDS = {
'default': {
'BACKEND': 'django.contrib.auth.backends.ModelBackend',
'MODEL': 'django.contrib.auth.models.User',
},
}

Instead of a single global USER_MODEL setting, each app should have
its own USER_MODEL settings, like ADMIN_USER_MODEL,
COMMENTS_USER_MODEL .

I do not like `ForeignKey(settings.ADMIN_USER_MODEL or
settings.USER_MODEL)` . I prefer a combination of solution 2a and 2b
to that.

[Tai Lee]

I'm not so sure that it's necessary or even desirable to solve the "general" problem of swappable models. If anyone can swap any model by changing a setting, that sounds like a recipe for confusion to me.

Seems to me that the main benefit of the swappable models approach is just to avoid backwards incompatibility issues by making the change opt-in, but I think it's also opening Pandora's box. I think that leaving it up to developers to ensure that their swapped-in models simply behave enough like the original is going to cause subtle and difficult to trace bugs.

App developers won't be able to have confidence that models they have defined will actually be used as defined.

This is why I would prefer to simply have an empty `User` model that is basically just a PK, which is used only as the glue that connects all the related bits and pieces (auth models, profile models, group and permissions models, etc.)

The rest of the existing `User` fields can be implemented in an `AdminUser` model, and allow multiple authentication backends, which defaults to just an `AdminUserBackend` authentication backend.

Apps and projects can just use this if they like, or they can define whatever models they like to link to the user (if they want to hook into other data that is linked to the user), along with their own auth backend. Then users can login with either admin credentials, or your project/apps credentials. A few abstract user base classes or auth backend classes could help make this part easier.

I'm not convinced that there should even be this concept (like the "data bag") that an app can rely on "some app" ("any app") defining a value for a user related field (e.g. name, email, twitter name, etc.), without knowing or caring which app has defined it. If you don't know which app defines the data, how can you know that it is the correct data for your app and your purpose? Likewise if you change a value, how do you know it won't inadvertently impact another app that relies on it? You won't know what other apps will be installed along side your own, what fields they will define, or when they will want to overwrite those fields.

There is such a thing as too generic. People can and do already define their own user/profile models and authentication backends. They can and do create their own models to store arbitrary data like name, twitter name, longer email address. The only problem is that there are then required and redundant or insufficient vestigial fields left over in `auth.User`. I have my own user model with username field, but I have to fake an `auth.User.username` to connect my user to groups and permissions. I can't use signals to sync my username/email address to `auth.User.email` for the Django admin because it's only 75 characters long and not RFC compliant.

If `auth.User` was just a glue model, with just a PK, and the other fields were moved to `AdminUser` (and `AdminUser.email increased in length), this would be a non-issue because I would not need to even have an `AdminUser` at all to create users for my app/project. I would only need to create one for users of the admin app, and in that case I should adhere to the AdminUser model's field requirements.

Cheers.
Tai.

[Luke Sneeringer]

Regarding swappable models:
In general, good reusable applications allow developers to substitute in their own class or subclass for the module author's default by either a custom setting or an argument to a method. In general, there's no reason why that mechanism is insufficient, and should be done per application.

Right now, we're trying to solve the swappable model *for User*, which is a bigger problem because it's not confined to a single app; it's defined in auth, but auth functions as a nexus for lots of other things.

Regards,
Luke

[Russell Keith-Magee]

On 04/04/2012, at 8:44 PM, Tai Lee wrote:

> I'm not so sure that it's necessary or even desirable to solve the "general" problem of swappable models. If anyone can swap any model by changing a setting, that sounds like a recipe for confusion to me.

Sure, but that's not what I've proposed. A model would only be swappable if the original app developer declared that model as swappable. An end user wouldn't be able to arbitrarily decide that they wanted to replace a model in an app developed by someone else.

And sure, any feature we add could ultimately end up being used (and overused) in bad ways. However, that's true of any language or library feature. Classes, metaclasses, decorators, or any other Python language feature can be both used and abused, as can Django features like ModelForms or the internals of the Meta class.

My point is that there is nothing about this problem that is unique to User. Django's own codebase contains another example of exactly the same pattern -- Comments. Therefore, we shouldn't pretend that the solution is User specific. At some point, we have to just provide enough documentation and guidance to shepherd people away from bad architectural decisions, and trust that the userbase will take that advice.

Yours,
Russ Magee %-)

[Jacob Kaplan-Moss]

On Wednesday, April 4, 2012 at 9:44 AM, Russell Keith-Magee wrote:

My point is that there is nothing about this problem that is unique to User. Django's own codebase contains another example of exactly the same pattern -- Comments.

As the original author and designer of that pattern, I should probably point out that I now think it's a mistake. Have you actually tried using it? It doesn't really work very well. Every time I've introduced any sort of "swappable model" mechanism I've come to regret it.

I'm -1 on the idea of generalized "swappable model" mechanism. I can stomach it for User because in this context it's not any worse than the alternatives, but as a generalized mechanism it makes me really, really unhappy. Decoupling is a laudable goal, but providing a mechanism to reach in and effect code not under your control isn't good design. We use derisive terms like "monkeypatching" for a reason.

I'm sure there are good reasons for wanting swappable models. Russ, I know you're smarter than me, so the fact that you want LazyForeignKey so much probably indicates that you've got some really solid use cases in mind. But really this is a hard veto from me; I just can't be convinced that this sort of mechanism is anything but bad news. 

However, I don't see why we should actively prevent this sort of thing being done externally, so if there's anything in Django that's precluding the creation of a LazyForeignKey as an external add-on let's consider that limitation a bug and get it fixed.

Jacob

[Adrian Holovaty]

On Wed, Apr 4, 2012 at 9:57 AM, Jacob Kaplan-Moss <ja...@jacobian.org> wrote:
> On Wednesday, April 4, 2012 at 9:44 AM, Russell Keith-Magee wrote:
>
> My point is that there is nothing about this problem that is unique to User.
> Django's own codebase contains another example of exactly the same pattern
> -- Comments.
>
> As the original author and designer of that pattern, I should probably point
> out that I now think it's a mistake. Have you actually tried using it? It
> doesn't really work very well. Every time I've introduced any sort of
> "swappable model" mechanism I've come to regret it.

Totally agree with Jacob here, plus Tai's comment that "There is such
a thing as too generic." We've made the mistake of making things too
generic in the past, and it's kind of infuriating in retrospect, both
philosophically and in terms of code maintenance/understanding.
(django/utils/tree.py, anyone??)

I think our policy should be: make the simplest thing that can
possibly work for a narrowly-tailored use case, then make things more
generic *slowly* if there's a demand. No need to be an Architecture
Astronaut.

Adrian