model inheritance without a new database table
Ryan Kelly
Hi All,
I'm currently working on an authentication backend that adds some
functionality to the standard one, and as part of this I want to modify
the behaviour of the User class. I've been subclassing auth.models.User
using standard model inheritance and it seems to be working fine.
However, I don't need to add any fields to the User class, just
add/modify some methods. It seems like a waste to have an additional
database table created for my model subclass when it won't contain any
useful information. I briefly toyed with the idea of just
monkey-patching the User class, but figured there must be a better
way :-)
So, is there a way to have a model subclass avoid the creation of a
new database table,and just take its data straight out of the table for
its superclass?
Thanks,
Ryan
--
Ryan Kelly
http://www.rfk.id.au | This message is digitally signed. Please visit
ry...@rfk.id.au | http://www.rfk.id.au/ramblings/gpg/ for details
Malcolm Tredinnick
> Hi All,
>
>
> I'm currently working on an authentication backend that adds some
> functionality to the standard one, and as part of this I want to modify
> the behaviour of the User class. I've been subclassing auth.models.User
> using standard model inheritance and it seems to be working fine.
>
> However, I don't need to add any fields to the User class, just
> add/modify some methods. It seems like a waste to have an additional
> database table created for my model subclass when it won't contain any
> useful information. I briefly toyed with the idea of just
> monkey-patching the User class, but figured there must be a better
> way :-)
>
> So, is there a way to have a model subclass avoid the creation of a
> new database table,and just take its data straight out of the table for
> its superclass?
Not at the moment. I'd like to get that into Django, though. It's the
missing "third" type of inheritance: Python-only.
There's a ticket open in Trac (my internet connection is terrible at the
moment, so I'm not going to hunt it out) that is to add an option to the
Meta class saying "this model is not to be managed by Django".
Primarily, that's for database views: you set them up outside of Django
and syncdb won't create tables or anything like that.
Turns out, that same option will work well for this case. If model B
subclasses model A and model B says "Don't manage me", it should all
just work fine. You will be able to add as much Python stuff to the
model as you like and no database table will be created.
Right now, I suspect you would also have to manually set Meta.db_table
and some things like that. However, I strongly suspect each of those are
fairly straightforward to sort out. We can set db_table automatically
for such models, if it's not set. We have to make sure the pk property
talks to the first ancestor model, too. Basically a bunch of fiddly
stuff, but nothing that's brain surgery levels of difficulty.
Regards,
Malcolm
Malcolm Tredinnick
[...]
> There's a ticket open in Trac (my internet connection is terrible at the
> moment, so I'm not going to hunt it out) that is to add an option to the
> Meta class saying "this model is not to be managed by Django".
> Primarily, that's for database views: you set them up outside of Django
> and syncdb won't create tables or anything like that.
In case you want to play with this, the ticket for the initial work is
#3163. The name of the option isn't what it will finally be (I think
we'll end up going for something simple like "managed"), but you can see
the idea there. Similarly, there are a few things that need tidying up
in the patch, but they're mostly minor. A good chance get that checked
in for Django 1.1.
Regards,
Malcolm
Ryan Kelly
> moment, so I'm not going to hunt it out) that is to add an option to the
> Meta class saying "this model is not to be managed by Django".
Thanks Malcolm, I'll definitely take a look and see if I can help out on
the ticket at all.
Cheers,
Ryan Kelly
> > is there a way to have a model subclass avoid the creation of a
> > new database table,and just take its data straight out of the table for
> > its superclass?
>
> and some things like that. However, I strongly suspect each of those are
> fairly straightforward to sort out. We can set db_table automatically
> for such models, if it's not set. We have to make sure the pk property
> talks to the first ancestor model, too. Basically a bunch of fiddly
> stuff, but nothing that's brain surgery levels of difficulty.
Following Malcolm's tips, I have managed to get this working. After
applying the patch from ticket #3163, I've formulated this class
decorator to make the necessary tweaks to the model's meta info:
def virtual_inheritance(cls):
base = cls.__bases__[0]
pk = base._meta.pk
# don't create a new db table
cls._meta.managed = False
# use the superclass table name
cls._meta.db_table = base._meta.db_table
# remove the auto-created parent reference
cls._meta.local_fields[:] = []
# use the superclass primary key
cls._meta.pk = pk
cls._meta.parents[base] = pk
Now I can do the following:
class MyUser(auth.models.User):
def an_extra_method(self):
print "hooray!"
virtual_inheritance(MyUser)
The resulting model behaves exactly as I wanted, adding functionality to
the default User model but taking its data straight out of the auth_user
table.
If I'm feeling inspired tomorrow, I might try to formulate similar logic
as a patch to the ModelBase metaclass, so that subclasses that don't add
any fields will get pure-python inheritance by default. In the
meantime, I hope some other people might find this trick useful.
Cheers,
Malcolm Tredinnick
That (changing the default behaviour) would be a bad idea. The implicit
primary key field shouldn't just magically not appear because something
else isn't there. It would lead to inconsistency with the (valid and
in-use) case of:
class SingleFieldModel(models.Model):
pass
(which has one field -- the auto-generated primary key one).
The good news, though, is that I suspect a patch to ModelBase will be a
bit simpler than your decorator, since we can just conditionally check
_meta.managed to determine whether to add things like the auto-fields
and the links back to the parent.
Regards,
Malcolm
Ryan Kelly
> > If I'm feeling inspired tomorrow, I might try to formulate similar logic
> > as a patch to the ModelBase metaclass, so that subclasses that don't add
> > any fields will get pure-python inheritance by default.
>
> That (changing the default behaviour) would be a bad idea. The
> implicit primary key field shouldn't just magically not appear because
> something else isn't there.
Indeed - and on further reflection it seems that what I want is a little
different from normal subclassing anyway. Standard subclasses require
only that all instances of the subclass are also instances of the base
class. In this case, I'm trying to make all instances of the base class
automatically be available as instances of the subclass.
I wonder if there's a standard terminology for this - I'm calling them
"virtual subclasses" but maybe they should be called "facade subclasses"
or something like that?
> The good news, though, is that I suspect a patch to ModelBase will be
> a bit simpler than your decorator, since we can just conditionally
> check _meta.managed to determine whether to add things like the
> auto-fields and the links back to the parent.
Yep, getting a patch working for this was pretty straightforward.
However, I think it would be better to use a distinct option instead of
overloading "managed", since it would be quite legitimate for someone to
want an unmanaged subclass that still lived in its own (manually
created) database table.
My current patch uses "virtual", so I can do the following:
def MyUser(auth.models.User):
class Meta:
virtual = True
def an_extra_method(self):
print "hooray!"
What do you think? I intend to pretty this up and submit a patch
sometime this evening.
Malcolm Tredinnick
> > > If I'm feeling inspired tomorrow, I might try to formulate similar logic
> > > as a patch to the ModelBase metaclass, so that subclasses that don't add
> > > any fields will get pure-python inheritance by default.
> >
> > That (changing the default behaviour) would be a bad idea. The
> > implicit primary key field shouldn't just magically not appear because
> > something else isn't there.
>
>
> Indeed - and on further reflection it seems that what I want is a little
> different from normal subclassing anyway. Standard subclasses require
> only that all instances of the subclass are also instances of the base
> class. In this case, I'm trying to make all instances of the base class
> automatically be available as instances of the subclass.
Oh, okay. Yeah, that's quite different. I'm not convinced that should go
into Django. Again, there's a consistency thing: we present the object /
class type that is asked for, not some transparently descended version.
If we did it one way for one type of inheritance and another way for
another type, things get weird pretty quickly.
The use-case for the pure-Python inheritance that has been floating
around is the case when you want to, say, use a different User manager
in your own views. You don't want to change the auth app's behaviour in,
say, admin, but you'd prefer a different default when you're using it.
So you subclass User, using Python-only inheritance, add a new default
manager and then actually use the subclass, rather than User in your
code.
> I wonder if there's a standard terminology for this - I'm calling them
> "virtual subclasses" but maybe they should be called "facade subclasses"
> or something like that?
There's quite probably 14 different standard terminologies. This is the
object-oriented-paradigm vocabulary space, after all. :-)
>
> > The good news, though, is that I suspect a patch to ModelBase will be
> > a bit simpler than your decorator, since we can just conditionally
> > check _meta.managed to determine whether to add things like the
> > auto-fields and the links back to the parent.
>
> Yep, getting a patch working for this was pretty straightforward.
> However, I think it would be better to use a distinct option instead of
> overloading "managed", since it would be quite legitimate for someone to
> want an unmanaged subclass that still lived in its own (manually
> created) database table.
Excellent point. Good catch.
>
> My current patch uses "virtual", so I can do the following:
>
> def MyUser(auth.models.User):
> class Meta:
> virtual = True
> def an_extra_method(self):
> print "hooray!"
>
>
> What do you think? I intend to pretty this up and submit a patch
> sometime this evening.
Not in love with the word "virtual" here. Words like "abstract" and
"virtual" tend to describe base classes (ancestors), rather than
derived / descendent classes.
Still, I wouldn't let that hold things up. I'll think about it and come
up with a name later on, unless a better option appears beforehand.
Regarding the patch, please remember that tests and documentation are
your friend. I suspect the documentation might be a little fiddly to
write clearly, but give it your best shot (and don't forget to change
the bit where it says there are two types of inheritance -- it's going
to look a bit like a Spanish Inquisition sketch otherwise).
Thanks for doing this. It's been on my list for quite a few months and I
just haven't the time. Will be good to get it out of the way (that was
really the main thing holding up the "managed" ticket, and now you go
and point out that we can't entirely piggy-back off that anyway).
Regards,
Malcolm
Ryan Kelly
> Oh, okay. Yeah, that's quite different. I'm not convinced that should go
> into Django. Again, there's a consistency thing: we present the object /
> class type that is asked for, not some transparently descended version.
I think my explanation may have been a bit off...
> The use-case for the pure-Python inheritance that has been floating
> around is the case when you want to, say, use a different User manager
> in your own views. You don't want to change the auth app's behaviour in,
> say, admin, but you'd prefer a different default when you're using it.
> So you subclass User, using Python-only inheritance, add a new default
> manager and then actually use the subclass, rather than User in your
> code.
....because this sounds like exactly how I want to use this feature. I
subclass auth.User and add my own managers/methods/etc, and use this
subclass in my own code. But if someone creates a new instance of
auth.User via the standard class, it should also show up in queries
posed through my subclass.
Likewise, if I add a new user through my special subclass, it will show
up as an ordinary user to e.g. the admin interface.
So I'm not trying to add anything magical - the "virtual subclass" is
simply a "facade" or "wrapper" around the data in the base class table
that presents a modified interface. Yikes, there's that naming problem
again...
> Still, I wouldn't let that hold things up. I'll think about it and come
> up with a name later on, unless a better option appears beforehand.
Originally I named it "use_superclass_table" but decided that was just
too ugly to deal with. "interface_only"? Glad not to be the one stuck
making that decision :-)
Thanks for all your feedback, I'll get some code into trac tonight.
Malcolm Tredinnick
[...]
> > The use-case for the pure-Python inheritance that has been floating
> > around is the case when you want to, say, use a different User manager
> > in your own views. You don't want to change the auth app's behaviour in,
> > say, admin, but you'd prefer a different default when you're using it.
> > So you subclass User, using Python-only inheritance, add a new default
> > manager and then actually use the subclass, rather than User in your
> > code.
>
> ....because this sounds like exactly how I want to use this feature. I
> subclass auth.User and add my own managers/methods/etc, and use this
> subclass in my own code. But if someone creates a new instance of
> auth.User via the standard class, it should also show up in queries
> posed through my subclass.
Okay, so that's the situation I would expect. It doesn't automatically
replace all usages of User with your subclass (which is what I thought
you were going for). It simply means the subclass is a proxy view of the
same persistent data as the main class. That should be just a matter of
pointing both models to the same database table, etc, as you've done.
>
> Likewise, if I add a new user through my special subclass, it will show
> up as an ordinary user to e.g. the admin interface.
>
> So I'm not trying to add anything magical - the "virtual subclass" is
> simply a "facade" or "wrapper" around the data in the base class table
> that presents a modified interface. Yikes, there's that naming problem
> again...
If you what you're seeking a word for is to describe the style of
subclassing, call it a proxy class, since it stands in for the original
model.
Certainly within the Django documentation, I'd like to avoid words like
"facade" or "virtual". The latter has the normal direction reversed (in
other languages, it's applied to ancestor classes, so will cause at
least a slight brain hesitation when people read it), the former is not
a common enough word to be clear to the thousands of people reading the
docs who don't have English as a first language.
"Proxy" meets the "common enough" test, for my tastes, at least. You run
across proxies for everything in computer science, particularly
web-exposed products. if somebody doesn't already know the word, they
need to learn it pretty quickly in any case.
>
> > Still, I wouldn't let that hold things up. I'll think about it and come
> > up with a name later on, unless a better option appears beforehand.
>
> Originally I named it "use_superclass_table" but decided that was just
> too ugly to deal with. "interface_only"? Glad not to be the one stuck
> making that decision :-)
Again, "interface" has that not particularly common touch to it. We
should try hard to stick to really obvious words to describe things (and
"interface class" is already used in OO for something that definitely
isn't the child class, so the confusion problem, again). Yeah, naming
stuff is hard. We knew that already.
Regards,
Malcolm
Ryan Kelly
Thanks again for your feedback on this Malcolm, I've created the
following ticket in Trac:
Proxy models: subclass a model without creating a new table
http://code.djangoproject.com/ticket/10356
Cheers,