magic-removal: "Change subclassing syntax"

26 views
Skip to first unread message

Joseph Kocherhans

unread,
Jan 25, 2006, 3:19:47 PM1/25/06
to django-d...@googlegroups.com
Is this a good summary of the current thinking on subclassing?

http://groups.google.com/group/django-developers/browse_frm/thread/ea5e0bf903058fac/9a68ac0d99cb6d7d?q=semantics&rnum=1#9a68ac0d99cb6d7d

The wiki doesn't say a whole lot about it, although it's probably in
someone's brain ;)

Joseph

Jason Davies

unread,
Jan 25, 2006, 5:43:15 PM1/25/06
to Django developers

I think that's the best we've got. Is it worth creating a wiki page?

Jason

Adrian Holovaty

unread,
Jan 25, 2006, 5:55:49 PM1/25/06
to django-d...@googlegroups.com
On 1/25/06, Jason Davies <jason....@gmail.com> wrote:
> I think that's the best we've got. Is it worth creating a wiki page?

Yeah, let's collect the design decisions and code examples on a wiki page.

Adrian

--
Adrian Holovaty
holovaty.com | djangoproject.com | chicagocrime.org

Jeremy Dunck

unread,
Jan 25, 2006, 6:02:54 PM1/25/06
to django-d...@googlegroups.com
On 1/25/06, Adrian Holovaty <holo...@gmail.com> wrote:
>
> On 1/25/06, Jason Davies <jason....@gmail.com> wrote:
> > I think that's the best we've got. Is it worth creating a wiki page?
>
> Yeah, let's collect the design decisions and code examples on a wiki page.

This makes me think the world needs a wiki whose wikitext syntax is
restructured text.

Anybody know of one?

Daniel Poelzleithner

unread,
Jan 25, 2006, 6:15:47 PM1/25/06
to django-d...@googlegroups.com
Jeremy Dunck wrote:

> This makes me think the world needs a wiki whose wikitext syntax is
> restructured text.

In general ?
At least trac has a macro for restructured text I think.

kindly regards
daniel

Max Battcher

unread,
Jan 25, 2006, 10:56:03 PM1/25/06
to django-d...@googlegroups.com
Jeremy Dunck wrote:
> This makes me think the world needs a wiki whose wikitext syntax is
> restructured text.
>
> Anybody know of one?

I've seen a home-brew Zope-based one. It would be real quick to build
one as a django app... In fact its own my personal project todo list.
How soon do you want it? :-)

--
--Max Battcher--
http://www.worldmaker.net/

Max Battcher

unread,
Jan 25, 2006, 11:04:53 PM1/25/06
to django-d...@googlegroups.com
Joseph Kocherhans wrote:
> The wiki doesn't say a whole lot about it, although it's probably in
> someone's brain ;)

I'm wondering about the ETA of this as well. I could use a good
subclassing system for my project *today*, particularly because as far
as I can tell the One-To-One relationship is currently broken (broken
default ordering; no way to order a class in the admin based upon a
join; broken generic object_detail view).

(Like many, I need a subclass of ``django.contrib.auth.models.User`` for
site-specific data.)

Jeremy Dunck

unread,
Jan 26, 2006, 7:04:33 AM1/26/06
to django-d...@googlegroups.com
On 1/25/06, Max Battcher <m...@worldmaker.net> wrote:
> I've seen a home-brew Zope-based one. It would be real quick to build
> one as a django app... In fact its own my personal project todo list.
> How soon do you want it? :-)

Mine, too, but you may get to it first. As Soon As Our Copious Free
Time Makes Possible. ;-)

Antonio Cavedoni

unread,
Jan 26, 2006, 9:27:16 AM1/26/06
to django-d...@googlegroups.com
On 26 Jan 2006, at 0:02, Jeremy Dunck wrote:
> This makes me think the world needs a wiki whose wikitext syntax is
> restructured text.
>
> Anybody know of one?

Apparently you can use ReST with MoinMoin:

http://moinmoin.wikiwikiweb.de/HelpOnParsers/ReStructuredText

It’s not the native wikitext, but support can be added in.

Cheers.
--
Antonio


Joseph Kocherhans

unread,
Jan 26, 2006, 12:11:26 PM1/26/06
to django-d...@googlegroups.com
On 1/25/06, Adrian Holovaty <holo...@gmail.com> wrote:
>
> On 1/25/06, Jason Davies <jason....@gmail.com> wrote:
> > I think that's the best we've got. Is it worth creating a wiki page?
>
> Yeah, let's collect the design decisions and code examples on a wiki page.

I'm writing up a draft of this right now. I'll post it sometime today.

Joseph

Joseph Kocherhans

unread,
Jan 26, 2006, 1:26:32 PM1/26/06
to django-d...@googlegroups.com

http://code.djangoproject.com/wiki/ModelInheritance

This is still really rough. I hope I didn't leave too much out. Let
the arguing commence ;-)

Joseph

Jacob Kaplan-Moss

unread,
Jan 26, 2006, 2:27:12 PM1/26/06
to django-d...@googlegroups.com
On Jan 26, 2006, at 12:26 PM, Joseph Kocherhans wrote:
> http://code.djangoproject.com/wiki/ModelInheritance

Thanks for doing this, Joseph! It looks really good, too, but here
are the (inevitable) nitpicks:

1. Modeling parent relations in SQL: #3 seems to make the most sense
for me -- I can't think of a case in a one2one where the child id
would need to differ from the parent id. Also::

CREATE TABLE "myapp_restaurant" (
"id" integer NOT NULL PRIMARY KEY REFERENCES
"myapp_places" ("id"),
"description" text NOT NULL
);

works just fine (at least in Postgres); I'd suggest that SQL
relations be used explicitly here.

2. Modeling joins in SQL / lookup API: my general take on this is
"make it explicit". If I ask for places, give me a list of places
and don't bother looking up restaurants. So::
``place_instance.description`` should raise an ``AttributeError``
regardless if weather or not it is actually a restaurant.

However, there means there should be a way, given a Place, to get a
Restaurant; I would suggest::

>>> p = Place.objects.get(pk=1)
>>> r = Restaurant(p)
>>> i = ItalianRestaurant(p) # or ...(r)

(I suggest this syntax since it looks like exactly like casting --
``foo = int(bar)``).

Other than that, this looks GREAT.

Jacob

Maniac

unread,
Jan 26, 2006, 2:51:19 PM1/26/06
to django-d...@googlegroups.com
Jacob Kaplan-Moss wrote:

> If I ask for places, give me a list of places and don't bother
> looking up restaurants. So:: ``place_instance.description`` should
> raise an ``AttributeError`` regardless if weather or not it is
> actually a restaurant.

Why? This is not how duck typing works all over the Python: if the
object does have an attribute you shouldn't do nothing special (like
casting) to get it.

Maniac

unread,
Jan 26, 2006, 2:58:01 PM1/26/06
to django-d...@googlegroups.com
Joseph Kocherhans wrote:

>http://code.djangoproject.com/wiki/ModelInheritance
>
>This is still really rough. I hope I didn't leave too much out. Let
>the arguing commence ;-)
>
>

If the last bit ("Change the current usage of subclassing") is only
about removing inherited attributes why not do it this way:

class MyArticle(Article):
del Article.field1
del Article.field2
...

Robert Wittams

unread,
Jan 26, 2006, 3:01:41 PM1/26/06
to django-d...@googlegroups.com
Jacob Kaplan-Moss wrote:

> 1. Modeling parent relations in SQL: #3 seems to make the most sense
> for me -- I can't think of a case in a one2one where the child id would
> need to differ from the parent id. Also::
>
> CREATE TABLE "myapp_restaurant" (
> "id" integer NOT NULL PRIMARY KEY REFERENCES "myapp_places"
> ("id"),
> "description" text NOT NULL
> );
>
> works just fine (at least in Postgres); I'd suggest that SQL relations
> be used explicitly here.

Agree here. There is absolutely no meaning in a subtyping relationship
in which the members of the supertype do not share an identity domain.

> 2. Modeling joins in SQL / lookup API: my general take on this is "make
> it explicit". If I ask for places, give me a list of places and don't
> bother looking up restaurants. So:: ``place_instance.description``
> should raise an ``AttributeError`` regardless if weather or not it is
> actually a restaurant.
>
> However, there means there should be a way, given a Place, to get a
> Restaurant; I would suggest::
>
> >>> p = Place.objects.get(pk=1)
> >>> r = Restaurant(p)
> >>> i = ItalianRestaurant(p) # or ...(r)
>
> (I suggest this syntax since it looks like exactly like casting --
> ``foo = int(bar)``).

This is horrible. If something is a restaurant, it should always act
like a restaurant. It should not depend on interrogating the object "Are
you a restaurant?" - this would become as tedious as dynamic_cast in
C++. Think about a method that is overridden in Restaurant. Does it make
any sense at all for its behaviour to depend on whether you have
explicitly cast it to the correct type? This completely breaks OO - very
similar to object slicing in C++. Not something to duplicate.

If a base class has subclasses, all queries on the base class need to be
joined to the tables representing all subclasses. Then the results can
be instantiated into the correct type. Maybe the .values() query can
just return the base class values if the joins are scaring people.

Please don't rely on phrases like "explicit is better than implicit" to
make your decisions for you - if we did that, everyone would be using ASM.

Max Battcher

unread,
Jan 26, 2006, 3:14:53 PM1/26/06
to django-d...@googlegroups.com

Agreed. Instead, I think the explicitness Jacob wants should actually
be a factor of the Manager class in question, particularly because the
biggest impact is on the amount of database work involved (to join or
not to join). Maybe the default manager doesn't join, it just returns
everything in terms of the class that it is attached to. But you could
explicitly create a "PolymorphicManager" that can (and does) join
subclassed models. Something like:

``
app_objects = managers.PolymorphicManager(Place, Restaurant,
ItalianRestaurant)
``

This also makes it explicit which classes are participating in a join,
rather than potentially joining a large number of installed model classes.

Robert Wittams

unread,
Jan 26, 2006, 3:27:21 PM1/26/06
to django-d...@googlegroups.com

> app_objects = managers.PolymorphicManager(Place, Restaurant,
> ItalianRestaurant)
> ``
>
> This also makes it explicit which classes are participating in a join,
> rather than potentially joining a large number of installed model classes.
>

Thus making it completely unnatural to inherit from a class that you
didn't write your self - ie you must monkey patch in the manager, and
then what kind of trouble do you get in when two apps by different users
want to extend the same core class.

Horrific. Coupling supertypes to subtypes is madness....

I think all that is needed here is education - it needs to be pointed
out what the costs of subtyping are.

One option would be to provide a method on managers to provide a "faster
but broken" manager, ie joinless. It needs to have a name to suggest
that it is not really recommended though, and should only be used if it
actually has a beneficial performance impact...

Robert Wittams

unread,
Jan 26, 2006, 3:17:32 PM1/26/06
to django-d...@googlegroups.com

Removing inherited attributes (IE removing database fields) breaks the
ability to share an identity domain for the inheritance tree. This is a
requirement for any vaguely sane subtyping relationship. The subtyping
in trunk just isn't subtyping in any meaningful sense, it is definition
copying, which is something far less useful. All the sane use cases so
far also seem to rely on replaces_module, which no longer means anything
in magic-removal.

That aside, a plain reading of your suggestion would suggest that you
are deleting the attributes from the parent class. You might be able to
cobble this together to work in the entirely unintuitive way you suggest
with a __del__ in the Field class and a module global list which is
checked in the metaclass. I wouldn't recommend it though - it sounds
immensely confusing.

Joseph Kocherhans

unread,
Jan 26, 2006, 7:24:59 PM1/26/06
to django-d...@googlegroups.com
On 1/26/06, Robert Wittams <rob...@wittams.com> wrote:
>
> I think all that is needed here is education - it needs to be pointed
> out what the costs of subtyping are.

I agree. I think most of the time the join is going to be simple
anyhow... take comments for instance. Say I have a Comment class and 3
different subtypes. If I just want the MusicComments, the join is a
simple one between two tables (and would only scary with a deep
inheritance tree) Now if I want to look at ALL the comments, the join
is on 4 tables (barring select_related). The equation is simple: # of
joins = # of subclasses, but only when you access a superclass. Poor
java programmers ;-)


> One option would be to provide a method on managers to provide a "faster
> but broken" manager, ie joinless. It needs to have a name to suggest
> that it is not really recommended though, and should only be used if it
> actually has a beneficial performance impact...

My gut reaction is to punt on this and wait to see if it has some real
life use. (i.e. if the default way is intolerably slow.) It's not a
huge api change to add it later.

Joseph

Robert Wittams

unread,
Jan 26, 2006, 7:44:33 PM1/26/06
to django-d...@googlegroups.com

Just a few vague notes about how I planned to implement this:

I wanted to possibly in future support different inheritance models.
The other sane ones (AFAIK) are:

* Concrete table model (only works with databases with sequences). A
table is made with all attributes for each concrete class. Base class
queries are done with unions rather than joins. All the concrete tables
in one inheritance hierarchy share a sequence for id generation so that
no two instances of the supertype share an id.

* Inheritance-hierarchy in one table. Every attribute gets stuck into
one table. One column is given over to the type of each row. Attributes
are null for all rows that they are not valid for. Extra check
constraints generated for attributes that are not logically null. Good
for the join-shy.

So the idea was to have a class InheritanceModel, that eg the manager
and the sql creation stuff would refer to. There would be a default
implementation using one-to-one relationships. This could be overridden
in class Meta.

So signals can be used to control this. A signal should be emitted by
the metaclass for each base class that is a subclass of Model. The
inheritance model should listen for these signals and build up the
inheritance tree.

I think it is worth seperating out the inheritance policy so different
database mappings can be tried out in future.

Anyway,
Good luck,

Rob

John Szakmeister

unread,
Jan 27, 2006, 5:36:27 AM1/27/06
to django-d...@googlegroups.com
On Wednesday 25 January 2006 18:15, Daniel Poelzleithner wrote:
> Jeremy Dunck wrote:
> > This makes me think the world needs a wiki whose wikitext syntax is
> > restructured text.
>
> In general ?
> At least trac has a macro for restructured text I think.

Yes, it does:
http://projects.edgewall.com/trac/wiki/WikiRestructuredText

-John

Carlo C8E Miron

unread,
Feb 1, 2006, 2:33:55 PM2/1/06
to django-d...@googlegroups.com
2006/1/26, Joseph Kocherhans <jkoch...@gmail.com>:

> > I'm writing up a draft of this right now. I'll post it sometime today.
> http://code.djangoproject.com/wiki/ModelInheritance

referring to 2. Modeling joins in SQL, why did you use LEFT JOIN instead
of INNER JOIN? as subtypes inherits from supertypes, if you ask for an
ItalianRestaurant record you surely have also one row in Restaurant and
in Place table; so, why the need of a LEFT JOIN?
This of course implies that if you ask for a Restaurant record you will
not have any ItalianRestaurant row (and obviously none
"has_decent_gnocchi" attribute. Yes, there is a typo on the wiki page,
"gnocchi" is the correct spell for that first course ;-)

HAND
(c)
--
Carlo C8E Miron, ICQ #26429731
--
Disclaimer:
If I receive a message from you, you are agreeing that:
1. I am by definition, "the intended recipient".
2. All information in the email is mine to do with as I see fit and
make such financial profit, political mileage, or good joke as it
lends itself to. In particular, I may quote it on USENET or the WWW.
3. I may take the contents as representing the views of your company.
4. This overrides any disclaimer or statement of confidentiality that
may be included on your message.

Joseph Kocherhans

unread,
Feb 1, 2006, 2:57:16 PM2/1/06
to django-d...@googlegroups.com
On 2/1/06, Carlo C8E Miron <carlo...@gmail.com> wrote:
> 2006/1/26, Joseph Kocherhans <jkoch...@gmail.com>:
> > > I'm writing up a draft of this right now. I'll post it sometime today.
> > http://code.djangoproject.com/wiki/ModelInheritance
>
> referring to 2. Modeling joins in SQL, why did you use LEFT JOIN instead
> of INNER JOIN? as subtypes inherits from supertypes, if you ask for an
> ItalianRestaurant record you surely have also one row in Restaurant and
> in Place table; so, why the need of a LEFT JOIN?

Beacuse I don't use INNER JOIN often enough for it to be readily
accessible in my brain ;-) You're right though, INNER JOIN makes more
sense. (I would hope that it performs better accross every supported
db engine, but who knows)


> This of course implies that if you ask for a Restaurant record you will
> not have any ItalianRestaurant row (and obviously none
> "has_decent_gnocchi" attribute. Yes, there is a typo on the wiki page,
> "gnocchi" is the correct spell for that first course ;-)

Feel free to fix the wiki. That's what it's for ;-)

Joseph

Carlo C8E Miron

unread,
Feb 2, 2006, 4:54:30 AM2/2/06
to django-d...@googlegroups.com
Ciao Joseph!

2006/2/1, Joseph Kocherhans <jkoch...@gmail.com>:


> Beacuse I don't use INNER JOIN often enough for it to be readily
> accessible in my brain ;-) You're right though, INNER JOIN makes more
> sense. (I would hope that it performs better accross every supported
> db engine, but who knows)

In my experience (mainly PostgreSQL, Oracle and <sigh>SQLServer</sigh>),
this is absolutly true.

> > This of course implies that if you ask for a Restaurant record you will
> > not have any ItalianRestaurant row (and obviously none
> > "has_decent_gnocchi" attribute. Yes, there is a typo on the wiki page,
> > "gnocchi" is the correct spell for that first course ;-)
> Feel free to fix the wiki. That's what it's for ;-)

Done. And added a little innocent self-advertise, too ;)

Cheers,

Reply all
Reply to author
Forward
0 new messages