Single Table Inheritance
Jordan MacDonald
answer to a simple scenario and I'm wondering why this hasn't been
addressed before.
I have three models, structured like so:
Document
-Presentation
-Spreadsheet
Document is never instantiated on its own; a prime candidate for an
abstract base class. However, there are times where I want to list/
search across all documents, and I'd like to be able to write
Document.objects.all(). I'd then like to be able to iterate over this
enumerable and have each object cast to its proper class.
This is something accomplished with single table inheritance in Rails;
why don't we have the equivalent in Django?
I know I could just use the Document class and have a type field, but
then I have to do all of the type checking legwork manually. I was
hoping Django would handle normalizing/denormalizing as part of the
ORM. In essence, creating its own type field automatically in the back-
end and casting each object to the appropriate class based on the
string value in this field.
Does anyone know why this isn't available? Is there an equally
efficient method of modeling this approach of which I am unaware?
Carl Meyer
On 03/29/2011 11:40 AM, Jordan MacDonald wrote:
> I have three models, structured like so:
>
> Document
> -Presentation
> -Spreadsheet
>
> Document is never instantiated on its own; a prime candidate for an
> abstract base class. However, there are times where I want to list/
> search across all documents, and I'd like to be able to write
> Document.objects.all(). I'd then like to be able to iterate over this
> enumerable and have each object cast to its proper class.
>
> This is something accomplished with single table inheritance in Rails;
> why don't we have the equivalent in Django?
>
> I know I could just use the Document class and have a type field, but
> then I have to do all of the type checking legwork manually. I was
> hoping Django would handle normalizing/denormalizing as part of the
> ORM. In essence, creating its own type field automatically in the back-
> end and casting each object to the appropriate class based on the
> string value in this field.
>
> Does anyone know why this isn't available? Is there an equally
> efficient method of modeling this approach of which I am unaware?
I agree with you that there are cases where single-table inheritance is
the nicest model. We don't have it in Django because multi-table
inheritance covers a wider array of use cases with a cleaner database
schema, and allows you to e.g. have required fields on subclasses
enforced by your database schema, which is not possible with STI.
I'm not sure I'd want to have yet another variety of inheritance as a
first-class feature of the Django ORM; the array of existing options
with MTI, abstract inheritance, and proxy models is quite confusing
enough to new users!
I do, however, think an STI implementation for the Django ORM would be a
great candidate for an external utility app (if someone hasn't done it
already). I _think_ it'd be possible to do it without any modifications
to Django itself (although it would probably require poking at some
internal undocumented APIs), and might see significant uptake.
Carl
Jeremy Dunck
> Hi Jordan,
>
> On 03/29/2011 11:40 AM, Jordan MacDonald wrote:
>> I have three models, structured like so:
>>
>> Document
>> -Presentation
>> -Spreadsheet
>> I'd like to be able to write
>> Document.objects.all(). I'd then like to be able to iterate over this
>> enumerable and have each object cast to its proper class.
...
> I'm not sure I'd want to have yet another variety of inheritance as a
> first-class feature of the Django ORM; the array of existing options
> with MTI, abstract inheritance, and proxy models is quite confusing
> enough to new users!
What about keeping abstract inheritance in this case, but allowing
Document.objects.* to work by returning instances of the subclasses.
Filtering, etc. would only work based on the Document base class.
It would mean doing some unions, but would still fit the use case
pretty well, I think.
Carl Meyer
> What about keeping abstract inheritance in this case, but allowing
> Document.objects.* to work by returning instances of the subclasses.
> Filtering, etc. would only work based on the Document base class.
>
> It would mean doing some unions, but would still fit the use case
> pretty well, I think.
I'd certainly be intrigued to look at a patch that implemented that.
Carl
Jordan MacDonald
Well, maybe I can look into how abstract base classes are currently
implemented and see if there's a way to generate query sets for all
derived classes from the parent.
Thanks for the insight!
Shawn Milochik
possible in Django. If so, it will solve this problem.
I am not familiar with custom metaclass stuff done within models.Model.
1. Create a custom metaclass as described in "Pro Python," page 124.
2. Add this metaclass to the abstract model.
This would allow the base-class to be aware of its subclasses and have
a class method that returns a queryset of them.
Further detail:
This simply amounts to creating a list object in the abstract base
class. Each time a subclass is instantiated, that class is added to
the parent class's list. That's all. That list could be used in a
custom manager to get the desired queryset.
What I don't know is how nicely this will play with the existing
metaclass work in Django. It seems that a metaclass can be easily made
by subclassing the one used for models.Model instead of the default
'type,' but this is beyond my experience level. Is this a reasonable
approach?
Shawn
Jeremy Dunck
I made a ticket; hopefully I'll get around to it some day:
http://code.djangoproject.com/ticket/15711
Carl Meyer
What you've outlined here is certainly possible (and yes, you'd need to
subclass the ModelBase metaclass). I haven't looked at the abstract
inheritance stuff recently, but I think there would be some alternative
ways for the abstract base to know about its children that wouldn't
require the metaclass assignment. However: getting the list of
subclasses is (less than) half the battle; the trickier part is giving
the ORM the capability to do UNION queries on similar tables, so you can
get results from multiple tables in a single QuerySet.
Carl
Shawn Milochik
> Hi Shawn,
>
> What you've outlined here is certainly possible (and yes, you'd need to
> subclass the ModelBase metaclass). I haven't looked at the abstract
> inheritance stuff recently, but I think there would be some alternative
> ways for the abstract base to know about its children that wouldn't
> require the metaclass assignment. However: getting the list of
> subclasses is (less than) half the battle; the trickier part is giving
> the ORM the capability to do UNION queries on similar tables, so you can
> get results from multiple tables in a single QuerySet.
Carl,
Thanks for the explanation. That does make sense and I see where it
gets really tricky when you delve into the ORM.
I'm not proposing a change to Django itself or suggesting that this
should be a standard practice. I do think that this is a fairly clean
solution for an individual to use to solve this problem if they have
it.
They can create a custom manager on the abstract class that would
return an iterable, perhaps using itertools.chain() of the querysets.
It depends on what they expect to do with the output of this custom
manager, and they'd obviously lose the ability to treat this output as
a queryset by using additional sorts & filters and such. But if the
goal is to be able to get instances from all subclasses at once then
this is a viable solution, FWIW.
Shawn
Carl Meyer
On 03/29/2011 02:46 PM, Shawn Milochik wrote:
> I'm not proposing a change to Django itself or suggesting that this
> should be a standard practice. I do think that this is a fairly clean
> solution for an individual to use to solve this problem if they have
> it.
>
> They can create a custom manager on the abstract class that would
> return an iterable, perhaps using itertools.chain() of the querysets.
Ah, I didn't realize that's the direction you were headed. Yeah, you can
do this, and I've done it; it starts to hurt as soon as you want, say,
sorting + pagination without pulling all of both tables into memory.
> It depends on what they expect to do with the output of this custom
> manager, and they'd obviously lose the ability to treat this output as
> a queryset by using additional sorts & filters and such. But if the
> goal is to be able to get instances from all subclasses at once then
> this is a viable solution, FWIW.
Yup.
Carl
Johannes Dollinger
Am 29.03.2011 um 20:46 schrieb Shawn Milochik:
> They can create a custom manager on the abstract class that would
> return an iterable, perhaps using itertools.chain() of the querysets.
>
> It depends on what they expect to do with the output of this custom
> manager, and they'd obviously lose the ability to treat this output as
> a queryset by using additional sorts & filters and such. But if the
> goal is to be able to get instances from all subclasses at once then
> this is a viable solution, FWIW.
FWIW, here is an implementation that does just that: https://github.com/emulbreh/shrubbery/blob/master/shrubbery/db/union.py
__
Johannes
Forest Bond
I haven't used it, but maybe you should look into django_polymorphic:
http://bserve.webhop.org/django_polymorphic/
Thanks,
Forest
--
Forest Bond
http://www.alittletooquiet.net
http://www.pytagsfs.org
Alex Robbins
try haystack.
http://docs.haystacksearch.org/dev/searchqueryset_api.html#filter
I've setup a site with a real base class, done queries on that and
then returned the child classes. It worked, but it felt pretty hacky
all the way through. (You solution has a much nicer interface than
mine, so your's wouldn't feel as bad I'm guessing.)
If you go ahead and make the switch to haystack you'll be able to
search across any and all of your models, without worrying about
inheritance. I've been very happy with haystack since switching to it.
Alex
On Mar 29, 3:20 pm, Johannes Dollinger <emulb...@googlemail.com>
wrote: