Many-to-Many with Intermediate Table - How to use built in admin?
Vortexmind
I am currently developing a rather large data model, and I need to use
Many-to-Many with Intermediate Table (as I need to stick some
attributes within the association). I try to explain with the Django
example about this issue:
from django.db import models
class Reporter(models.Model):
first_name = models.CharField(maxlength=30)
last_name = models.CharField(maxlength=30)
class Article(models.Model):
headline = models.CharField(maxlength=100)
pub_date = models.DateField()
class Writer(models.Model):
reporter = models.ForeignKey(Reporter)
article = models.ForeignKey(Article)
position = models.CharField(maxlength=100)
So far, if i add
class Admin:
pass
to all the above I can edit them. But what if I want to have the inline
admin to make the Writer relation when adding an Article or a Reporter?
If I do as above, I must add Reporter, then Article, then I can finally
go in the Writer admin interface and do the join. I hope you understand
me :)
Thank to you all, so far Django is the framework that gave me less
problems ... I just started to look at it yesterday ^^
Russell Keith-Magee
The admin application doesn't currently have any special handling for
m2m relations with intermediate tables, so the approach you describe
is the only way to do it.
The only way to do this would be to write your own view, and customize
the manipulator to represent m2m with intermediate tables.
That said, 'm2m with intermediate' is a relatively common use case, so
if you have any neat ideas on how to represent such a structure, feel
free to suggest them. The idea has been discussed before, but no
obvious solution has emerged (the real sticking point is querying the
intermediate table - check the archives for previous discussions).
Yours,
Russ Magee %-)
yary
...
> That said, 'm2m with intermediate' is a relatively common use case, so
> if you have any neat ideas on how to represent such a structure, feel
> free to suggest them. The idea has been discussed before, but no
> obvious solution has emerged (the real sticking point is querying the
> intermediate table - check the archives for previous discussions).
Here's some brainstorming. This post is long, and it's all daydreaming.
I suspect someone else may have already come up with the same answer
and it doesn't work for some reason... Checked the archives, found some
relevant discussions, but not sure I found the right ones :-/
The problem is, how to add some attributes to a ManyToMany relation-
how to define them and use them. Django currently hides the
intermediate table, and that's good. It can stay hidden for the usual
use case, where all you want is a many-to-many relationship. I think,
when someone wants to add "fields" to the relationship, then it should
become visible, and also synthesize models with both relationship
attributes and table behavior.
How would it look?
Starting with defining half, the inner Meta class seems like a good
place to go:
class Reporter(models.Model):
first_name = models.CharField(maxlength=30)
last_name = models.CharField(maxlength=30)
class WritingRole(models.Model):
responsibility = models.CharField(maxlength=30)
description = models.CharField(maxlength=30)
pay_scale = models.ForeignKey(PayScale)
class Article(models.Model):
headline = models.CharField(maxlength=100)
pub_date = models.DateField()
writers = models.ManyToManyField(Reporter)
# Here's a stab at 'm2m with intermediate' defining syntax
class Meta:
writers.actual_pay=models.SmallIntegerField() # We don't pay
much!
writers.writing_role=models.ForeignKey(WritingRole)
That's it. Looks pretty clean, is easy to understand.
Maybe the "writers.<fields>' don't even go in the Meta subclass, maybe
they stay in the main class.
Recursion requires a little thought.
# 'self' should always mean 'this model',Article in this case
writers.similar_articles=models.ForiegnKey('self')
# so spell out when you really want to self-relate
writers.collusion_group=models.ForeignKey('Article.writers')
# And someone will find a need for ManyToMany recursion!
How to use?
The old API doesn't change, things work as always
art = Article.objects.get(pk=3)
writers = art.writers.all() # as usual, gives all reporters
writer1 = reps[1] # get a single reporter from the list
print writer1.last_name # "Smith"
written_things = writer1.article_set.all() # find all articles for
the reporter
print written_things[0].headline # "Django Gets Useful Model
Enhancements"
Maybe you're wondering why I didn't call the variables "reporter" and
"article". I'm thinking that when you define "writers"- when you define
a relationship with attributes- you're saying that the model on the
other side has something extra, when viewed from this side. So, you get
a subclass of the foreign model.* It does all the things that the
related class does, plus lets you retreive the attributes for that
particular relation:
writer1.actual_pay # 0 if working for free!
writer1.writing_role.responsibility # "Muse"...
# describe the role writer1 had when writing this article
written_things[0].writing_role.description
Given a plain old Reporter object "reporter1":
reporter1.writers_set.all() # gets all Article-plus objects for
writer
reporter1.writers_set.get(headline='Foo') # may have the syntax a bit
off here- find Article-plus object with headline 'Foo'
reporter1.writers_set.filter(writingrole__responsibility='Author') #
find Article-pluses for which reporter1 was the author.
* I realize this abstraction my be contentious- if you ask for a
Reporter you should get a Reporter and not some subclass Django creates
for you. That's my reaction too. It does seem cleaner to make a
'Writers' object of its own that behaves like a model with two foreign
key fields, and the extra attributes. Indeed Django should, so one can
make requests like:
Writers.objects.filter(reporter__last_name='Smith',actual_pay=0).count()
# how often do people named Smith work for free?
But then how would that abstraction work for other common cases?
reps = art.writers.all() # really gives Reporters this time
rep1 = reps[1] # get a single reporter from the list
# We've lost the information we got traversing the relation!
rep1.actual_pay # gives an attribute error
art.writers.actual_pay # what writer are we talking about?
art.writers[1].actual pay # huh? Even if that syntax works to select
a single writer, 'art.writers' returns Reporters, which don't have
"actual_pay"
Writers.objects.get(article=art,reporter=rep1).actual_pay # This
works. I think it hits the database again.
So, you can get something working with an auto-created class named
after the M2M relation (with an override to let user give it a
different name, perhaps). I do think that traversing such a
"relationship-with-extras" should give a subclass of the related
object: something that has behaviors and data from the related class,
and also the relationsip-with-attribute-class. It makes the app writers
job easier, and it doesn't break existing code, since existing code
doesn't have "relationship-with-extras" yet.
("relationship-with-extras" is fun!)
yary
previous post with a grain of salt while I rethink it! sorry...
Vortexmind
I'll try to think about something ...
yary
auto-synthesized composite classes!? Django went through a magic
removal, adding magic won't fly.
An antidote to magic is explicitness. Let's try the previous example
from a different angle, spelling everything out.
class Reporter(models.Model):
first_name = models.CharField(maxlength=30)
last_name = models.CharField(maxlength=30)
class Article(models.Model):
headline = models.CharField(maxlength=100)
pub_date = models.DateField()
# Here's the new option 'reation_model' - am open to tweaking the
name!
writers = models.ManyToManyField(Reporter,
relation_model='WritingRole')
class WritingRole(models.Model):
responsibility = models.CharField(maxlength=30)
description = models.CharField(maxlength=30)
pay_scale = models.ForeignKey(PayScale)
The idea here is that adding a 'realation_model' option to ManyToMany
field tells it to use the named model, instead of a hidden one. Syncdb
won't create a table when it sees that. Django should check if
'WritingRole' has the needed foreign keys, if not, it'll add
article = ForeignKey(Article)
reporter = ForeignKey(Reporter)
# lowercase the model name, underscore CamelCase- precedant for
that
All queries seem pretty straightforward.
# how often do people named Smith work for free?
WritingRole.objects.filter(reporter__last_name='Smith',actual_pay=0).count()
# Who were the muses?
WritingRole.objects.filter(responsibility=MUSE).reporter.all()
How about getting to the WritingRole model?
# Another way of finding overly generous Smiths
Reporter.objects.filter(last_name='Smith',writing_role__actual_pay=0).count()
# writing_role_set is a reverse relationship from both sides
# find unpaid writing roles, assuming only one Smith
Reporter.objects.get(last_name='Smith').writing_role_set.filter(actual_pay=0)
# find muses for an article
Article.objects.get(pk=1).writing_role_set.filter(
responsibility=MUSE).reporter.all()
How did writing_role become a filter on the Article & Reporter
managers? How did the models get a writing_role_set? That's easy- it
came from the WritingRole model, same as it does now!
Changing the related names should be as easy as:
class WritingRole(models.Model):
responsibility = models.CharField(maxlength=30)
description = models.CharField(maxlength=30)
pay_scale = models.ForeignKey(PayScale)
# explicitly name the required ForeignKeys for this M2M model
published_item = ForeignKey(Article,
related_name='roles_and_payments')
helper = ForeignKey(Reporter, related_name='chores_and_income')
phew.
Now, back to the post that handled it all- how to handle in the admin?
When the admin sees a M2M field with a 'relation_model' option, it
should treat it as a ForeignKey to the relation model... so if the
example Article model has a 'class Admin', it will allow edits to
WritingRoles. Which means 'edit_inline' should also be an available
option for ManyToMany if relation_model is specified.
I'm happy with this proposal. Low magic, and it mostly uses code
already in Django. In fact I think I could write up patches for it,
despite being a realitve newbie to Django.
Discussion?
Vortexmind
Russell Keith-Magee
>
> The idea here is that adding a 'realation_model' option to ManyToMany
This idea has been suggested previously, and has been rejected
previously - it has some problems when you hit queries.
> All queries seem pretty straightforward.
In your example, yes. However, as always, the devil is in the detail. Consider:
class ModelA(Model):
name = CharField(...)
class Relation(Model):
name = CharField(...)
class ModelB(Model):
name = CharField(...)
relation = ManyToManyField(ModelA, relation_model=Relation)
What is the meaning of:
ModelA.objects.filter(relation__name='foo')
Does this query reference the name of the Relation model, or of ModelB?
This is an artificial example, but it represents the crux of the
problem - there is no easy way to integrate queries over the
intermediate table without introducing ambiguities in queries over the
related table.
Come up with a clean syntax that differentiates the two, and we have a
winner. So far, I haven't seen a viable option.
Yours,
Russ Magee %-)
yary
> On 11/3/06, yary <not...@gmail.com> wrote:
> >
> > The idea here is that adding a 'realation_model' option to ManyToMany
>
> This idea has been suggested previously, and has been rejected
> previously - it has some problems when you hit queries.
>
> > All queries seem pretty straightforward.
>
> In your example, yes. However, as always, the devil is in the detail. Consider:
>
> class ModelA(Model):
> name = CharField(...)
>
> class Relation(Model):
> name = CharField(...)
>
> class ModelB(Model):
> name = CharField(...)
> relation = ManyToManyField(ModelA, relation_model=Relation)
>
> What is the meaning of:
>
> ModelA.objects.filter(relation__name='foo')
>
> Does this query reference the name of the Relation model, or of ModelB?
>
> This is an artificial example, but it represents the crux of the
> problem - there is no easy way to integrate queries over the
> intermediate table without introducing ambiguities in queries over the
> related table.
I see your ambiguity, and raise you one:
class ModelA(Model):
name = CharField(maxlength=40)
class ModelB(Model):
name = CharField(maxlength=40)
relation = ForeignKey(ModelA, related_name='relation')
class ModelC(Model):
name = CharField(maxlength=40)
relation = ForeignKey(ModelA, related_name='relation')
What is the meaning of:
ModelA.objects.filter(relation__name='foo') ?
Django's model syntax already allows ambiguities. I just tried the
above, and Django doesn't complain when you create the model, it does
when you try to use the filter. Which leads me to think of five
possible solutions:
0. Remind people that the convention for both ForeignKey and
ManyToManyFields is to use the name of the foreign model when building
the field name. The contrived example is ambiguous because it defies
that practice and uses the relation name instead. Problem solved by
cultural norm.
1. Current solution, as "the way it is in 0.95": allow ambiguities, if
the app writer contrives a painful model, they live with the pain and
can't use the masked attribute. Problem solved by loud complaining late
in the game.
2. Open a ticket to have Django warn about ambiguous models at syncdb
time, so model creator will know to rename attributes/set up
distinctive "related_name". Problem solved by gentle prodding early in
game.
The last two aren't serious, but they'd fix it too:
3. Always require programmer to specify which model they want when
traversing a join. Problem solved by excessive verbosity.
4. Drop relations from the Django DB model altogether. Problem solved
by excessive pedanticness.
Summing it up: ambiguity exists now, and it hasn't stopped Django from
providing very useful features. Fixing ambiguity is a related but
separate issue. To fix that, I'd vote for #2 (and only warn, because
there could be ambiguity in existing apps that work now, because they
never tickle the overloaded filters).
Russell Keith-Magee
>
> Russell Keith-Magee wrote:
> Django's model syntax already allows ambiguities. I just tried the
> above, and Django doesn't complain when you create the model, it does
Erm... I'd be interested to know how you tested this, because I get:
Error: Couldn't install apps, because there were errors in one or more models:
clashtest.modelc: Accessor for field 'relation' clashes with related
field 'ModelA.relation'. Add a related_name argument to the definition
for 'relation'.
clashtest.modelc: Reverse query name for field 'relation' clashes with
related field 'ModelA.relation'. Add a related_name argument to the
definition for 'relation'.
clashtest.modelb: Accessor for field 'relation' clashes with related
field 'ModelA.relation'. Add a related_name argument to the definition
for 'relation'.
clashtest.modelb: Reverse query name for field 'relation' clashes with
related field 'ModelA.relation'. Add a related_name argument to the
definition for 'relation'.
both on syncdb and validate. I've done a lot of work in this area,
including writing a fairly complex unit test to check all sorts of
obscure model validation conditions - if you've found a bug, I want to
kill it.
Yours,
Russ Magee %-).
yary
so maybe that's a dev feature. Or I may have had a typo in a model the
first time through- in fact I recall that I had a space in 'class Model
C', so if the 'name clash' detection requires all models to be created
at once, that's how it got bypassed.
So Django does check for name clashes already (under ideal
circumstances). Having a 'join_model' will be helpful for a feature my
client wants, I'll try making the name clash detection work for that
too, and post the results.