Proposal: Composite Foreign Keys
David Cramer
keys except more useful):
class MyModel2:
pass
class MyModel3:
pk_part1 = models.AutoField()
pk_part2 = models.ForeignKey(MyModel2)
name = models.TextField()
class MyModel:
pk_part1 = models.IntegerField()
pk_part2 = models.ForeignKey(MyModel2)
mymodel3 = models.CompositeForeignKey(MyModel3, pk_part1=pk_part1,
pk_part2=pk_part2)
MyModel.objects.filter(mymodel3__name="Hi")
Anyways, I think you get the gist. I think this would cover all use
cases, and is a very simple, similar, and clean API.
David Elias
definition API:
class Entity:
account = models.CharField()
number = models.IntegerField()
class Meta:
primary_key=('account', 'number')
class Invoice:
number = models.IntegerField()
group = models.CharField()
year = models.CharField()
entity = models.ForeignKey(Entity)
# will create entity_account and entity_number attributes
class Meta:
primary_key=('number', 'group', 'year')
class InvoiceItem:
invoice = models.ForeignKey(Invoice)
number = models.IntegerField()
product = models.ForeignKey('...')
class Meta:
primary_key=('invoice', 'number')
# OR
primary_key=('invoice__number', 'invoice__group',
'invoice__year', 'number')
Malcolm Tredinnick
On Thu, 2008-11-13 at 10:53 -0800, David Cramer wrote:
> Here's my proposal for the composite keys (which act much like generic
> keys except more useful):
>
> class MyModel2:
> pass
>
> class MyModel3:
> pk_part1 = models.AutoField()
> pk_part2 = models.ForeignKey(MyModel2)
> name = models.TextField()
>
> class MyModel:
> pk_part1 = models.IntegerField()
> pk_part2 = models.ForeignKey(MyModel2)
>
> mymodel3 = models.CompositeForeignKey(MyModel3, pk_part1=pk_part1,
> pk_part2=pk_part2)
Why is the primary key specified on the relation field, rather than just
being part of MyModel3? A model has a primary key, and the foreign
relation should use that in the natural case. It's not a property of the
relation, it's a property of the target model and should be declared
there.
There might be an argument for a multi-column version of to-field, but a
tuple or list would seem the easiest way of specifying that, rather than
a variable number of arguments.
At that point, I think normal ForeignKey fields could be adjusted to
Just Work(tm), since they'll know they're related to a model with a
multi-column primary key.
Regards,
Malcolm
David Cramer
this up. It would allow for a more generic version of a generic
foreign key. Although the more I think about it we'd still want to
magically handle ForeignKey's so the same problem exists.
I guess I could write a composite foreign key manager, which could be
used in both instances. The problem is the API is very confusing at
the moment, and I haven't been able to work out a good way to handle
(in the code) mycomposite__exact=1 (single to multiple relation
lookup). Maybe you could help with this :)
Really, this issue is all thats stopping me from completing the
composite PKs patch.
On Nov 13, 7:19 pm, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:
Malcolm Tredinnick
On Thu, 2008-11-13 at 19:50 -0800, David Cramer wrote:
> I was thinking beyond just handling Foreign Keys actually when I wrote
> this up. It would allow for a more generic version of a generic
> foreign key.
Generic relations are quite different to foreign keys and the two
shouldn't be merged.
Generic Relations say how to get from an object id and a content type to
a remote object. They aren't a database-layer construct, although a
database constraint to, e.g., the content-type table could be
enforced(but that's not relevant here). ForeignKeys are a representation
of a database constraint, always referring to same remote table. They
are necessarily different concepts and I dont' think attempting to merge
them is going to gain much in clarity. At some point when multi-column
primary keys are supported, we might be able to work out a way for
generic relations to handle them, but I'm not holding my breath about
that and it's certainly a phase 2 or phase 3 thing. The reason it's
almost impossible to support generic relations with arbitrary
multi-column fields because the number of columns is, well, arbitrary
(attempting to jam multiple remote values into one local database column
is a bad idea, too. At some point the hacks mount up beyond the point of
usefulness). That is, one content type requires one value, a different
one requires two and a third content type might require four values to
identify the object. I strongly suspect we'll end up just saying
GenericRelation fields only work for single valued objects ids. Which
isn't at all unreasonable, since Django's generic relation classes
aren't really compulsory for data modelling. Any given situation has a
solution that doesn't involve them.
> Although the more I think about it we'd still want to
> magically handle ForeignKey's so the same problem exists.
There's no magic in programming. :-)
Maybe be you mean transparently. If so, I think this is a requirement,
too, but it's not a problem. We know when the ForeignKey field is
specified which model it refers to, so we can include the correct number
of database columns at table creation time. We have perfect information
when we create the table, which is where this differs from any generic
relations.
> I guess I could write a composite foreign key manager, which could be
> used in both instances. The problem is the API is very confusing at
> the moment, and I haven't been able to work out a good way to handle
> (in the code) mycomposite__exact=1 (single to multiple relation
> lookup). Maybe you could help with this :)
The value on the right hand side can be any sequence or tuple (or even
an object that knows how to return a sequence or tuple). The ORM will
know that the field involved is a multicolumn field and know to expect
multiple values on the RHS to associate with the multiple LHS columns.
It doesn't make sense to say mycomposite__exact=1, since mycomposite
expects multiple values. It's clear what
mycomposite__exact = (1,2)
means, however.
I will finish my work on this when I get a chance and as soon as
possible. I realise you've been waiting for a bit. Haven't had a lot of
time for Django the past few weeks, but I'm getting back on the horse
again now, although there'll be a bit of focus on 1.1 feature triage and
1.0.X for a little bit first.
It doesn't mean that you need to specify the primary key columns on the
ForeignKey field, though. As I pointed out in the original message, you
know which are the primary key columns because they're declared on the
model referred to.
>
> Really, this issue is all thats stopping me from completing the
> composite PKs patch.
You should still have something that's in multiple portions, though.
Relations are kind of a second phase to the rest. So you could open a
ticket with the first part of the patch if you want some early review on
that (although you've missed the 1.1 window now, so it will be
deprioritised a little bit officially, but I'm still interested in
coming to some conclusion on this).
Regards,
Malcolm
David Cramer
be able to specify relations (e.g. the filter example), like it would
with a generic foreign key, but not restricted to content_type/
object_id. We also will run into the issue with that, where object_id
is a composite as well (but this can sit on the backburner IMHO).
Anyways, adding the fields "transparently" (thats what I meant) is
easy. I've done it quite a bit in my own code. I spent several hours
however, trying to find a way to make ".filter(pk=(1,2,3))", ".filter
(pk=dict(blah=1, blah2=2, blah3=3))" work, however. That is the
specific area I've been waiting on. I have however had lots of time to
find various random admin bugs which I've been slowly fixing up.
Filters/etc are all working in my local version.
On Nov 13, 10:14 pm, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:
Steve Holden
Does that apply even to recursive (reflexive) relationships, when the
related model must be specified by name rather than by reference? I
don't know enough about the internals to be clear on that, but it seems
like it could be problematic.
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Malcolm Tredinnick
On Thu, 2008-11-13 at 23:47 -0500, Steve Holden wrote:
[...]
> > Maybe be you mean transparently. If so, I think this is a requirement,
> > too, but it's not a problem. We know when the ForeignKey field is
> > specified which model it refers to, so we can include the correct number
> > of database columns at table creation time. We have perfect information
> > when we create the table, which is where this differs from any generic
> > relations.
> > [...]
> Does that apply even to recursive (reflexive) relationships, when the
> related model must be specified by name rather than by reference? I
> don't know enough about the internals to be clear on that, but it seems
> like it could be problematic.
It needs to be taken into account, but it's certainly solvable and not
an obstacle to success. There are two solutions available:
(1) Postpone creating the first table until we have read the second
table so that we know the number of columns. This isn't too hard,
although it means you end up making two passes over some classes'
internals in some cases. But table creation is such a rare event and the
Python code is so fast that it's easily affordable. We're not talking
about something changing from 15 minutes to 20 minutes or anything.
(2) The poor-man's approach is "ALTER TABLE", which is a small-ish
change, but butt ugly. I prefer the former solution, but the this is the
silly drop-in I've got at the moment just to make things possible to
develop with.
Realise that the reason we have string-ified forwards references is to
avoid the need for circular imports. But at some point we'll end up
importing everything anyway and then we can go and look at the number of
required columns. This is why forwards references work in the first
place -- at a later time we go back and fix up the internal references
to point to the Python objects in question. We can change table creation
to operate similarly. It's not that hard.
Regards,
Malcolm
Steve Holden
> On Thu, 2008-11-13 at 23:47 -0500, Steve Holden wrote:
> [...]
>
>>> Maybe be you mean transparently. If so, I think this is a requirement,
>>> too, but it's not a problem. We know when the ForeignKey field is
>>> specified which model it refers to, so we can include the correct number
>>> of database columns at table creation time. We have perfect information
>>> when we create the table, which is where this differs from any generic
>>> relations.
>>> [...]
>>>
>> Does that apply even to recursive (reflexive) relationships, when the
>> related model must be specified by name rather than by reference? I
>> don't know enough about the internals to be clear on that, but it seems
>> like it could be problematic.
>>
>
> It needs to be taken into account, but it's certainly solvable and not
> an obstacle to success. There are two solutions available:
>
> (1) Postpone creating the first table until we have read the second
> table so that we know the number of columns. This isn't too hard,
> although it means you end up making two passes over some classes'
> internals in some cases. But table creation is such a rare event and the
> Python code is so fast that it's easily affordable. We're not talking
> about something changing from 15 minutes to 20 minutes or anything.
>
self-referential relationship, though now you mention it I see that the
same issue can occur if the order of model definitions is sub-optimal.
> (2) The poor-man's approach is "ALTER TABLE", which is a small-ish
> change, but butt ugly. I prefer the former solution, but the this is the
> silly drop-in I've got at the moment just to make things possible to
> develop with.
>
Doesn't mean it couldn't be useful as a first implementation, though.
> Realise that the reason we have string-ified forwards references is to
> avoid the need for circular imports. But at some point we'll end up
> importing everything anyway and then we can go and look at the number of
> required columns. This is why forwards references work in the first
> place -- at a later time we go back and fix up the internal references
> to point to the Python objects in question. We can change table creation
> to operate similarly. It's not that hard.
>
>
now, but thanks for the explanations.
regards
Steve
Malcolm Tredinnick
On Fri, 2008-11-14 at 00:17 -0500, Steve Holden wrote:
[...]
> Sure. Perhaps it's not common terminology: by "recursive" I meant a
> self-referential relationship, though now you mention it I see that the
> same issue can occur if the order of model definitions is sub-optimal.
The recursive case is just a collapsed form of forwards references, so I
didn't address that specifically. You don't even need sub-optimal
ordering to require forwards references. For example, circular reference
loops (particularly with NULL relations) aren't unheard of. Even in
Django, you need forwards reference to implement the "through" option
for ManyToMany fields, since you need to define the intermediate model
with a ForeignKey to model A and then define model A with a many-to-many
through the intermediate model. So one of them has to be defined second.
All I'm saying is that forwards references crop up naturally; they're
not a wart (and references to the same model are the other obvious case
they crop up naturally).
Regards,
Malcolm
Frank Liu
Django 2.0? Or will this be ever included? Let me explain my
frustration a bit.
My company only allows Oracle db and the legacy tables span hundreds
of schemas and hundreds of tables in each schema, in multiple
databases. This limitation on single primary key and single databse
connection single schema is really making me feel that Django is only
catering to the simple bloggers and social networkers.
However, to django's credit, Django is very well designed. But without
having the aforementioned resolved, i will never be able to get my
team to deploy django formally. It only leaves me with a sigh of not
being able to use all of this great work but having to suffer through
php (yes currently our legacy reporting tool is written in php, in
>1000 lines of code (5 different lanuagues) per file). I feel the only
reason i check the user group now is just to see the status on this
proposal...and lo and behold, it didn't even make it into 1.1 compiled
feature list for rejection.
What a bummer. How much longer do we have to wait. I mean can someone
just take a look at sqlalchemy and emulate it? I mean come on, you
people are smart, why hasn't anyone done anything. In addition,
opinionated as you are, it is not hard to just opinionate your way
into a decision about how the admin portion should be handled and get
on with the rest. Finally, dont the great JKM and Malcolms use
composite primary keys...?!!
(#U%)QU
Alright, i feel better now. But still, come on, surprise our community
a bit. (plus i mean the only reason anyone ever goes for Sqlalchemy,
isn't it just for the composite key, and multi db support? (at least
for me))
Frank
On Nov 13, 9:25 pm, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:
Karen Tracey
@#$%!.....will this composite primary key feature be included in
Django 2.0? Or will this be ever included? Let me explain my
frustration a bit.
[snipped]
Did I miss something? Your note makes it sound like the core team actively prevented this proposal from being considered for 1.1. I don't believe that is the case.
Jacob posted this note:
http://groups.google.com/group/django-developers/msg/0b517e283421034b
to this list on September 15. The last paragraph of that note stated:
Proposing features for 1.1
==========================
To propose features for 1.1, make a post on django-dev, and add a link to your message at
http://code.djangoproject.com/wiki/Version1.1Features.
That's not an onerous procedure; anyone could propose anything for 1.1 consideration, and that wiki page was available for editing for something like two months (actually a bit longer than the published schedule had said it would be). If you wanted this feature so much, why did you not do something to make sure it got posted as a proposal on the wiki page?
Karen
Frank Liu
the list (judging from the title of this thread).
Regardless, even if this had been added, it would've been given a -1
anyway, judging from the readiness of this feature.
More importantly though, my gripe about this is that if you go through
a search for this feature, you would see entries from 2006. Over the
last 2 years, there were multiple threads on this and yet nothing was
really done to it. How many time have we seen attempts at integrating
sqlalchemy and django? And really what features did people intend to
add? It's really obvious (at least to me) that it's for the multi db
support and composite primary key (I mean exactly what else does
sqlalchemy do better, ok more backends, what else?). Why don't people
get this?
And let me correct myself a bit. It's not that no one has done
anything, it's just that people have been only writing bits and pieces
of hacks here and there to patch the flaws, but this problem has
proven to be more difficult than just patches. For example, there are
numerous suggestions in this thread in the API of the model. However,
SO WHAT? does it matter? I like both. Both have merits. THEN WHAT?
what about the admin layer, what this and that relationship, what
about these recursive thing. Almost every one of these composite key
threads had been stalled somewhere either in admin or the model api
that nothing substantial was done. (usually a thread like this goes on
a few 10s of exchanges and finally left cold...just like this one).
And I think this is not the developers' fault. I think fundamentally
there are 2 reasons for this: 1. the original design of the orm is not
up to the task so that extending it proved to be too difficult. As
have been mentioned in numerous places, the orm is one place where
loose coupling should be taken with a grain of salt.
2 lack of leadership. To deal with this difficult situation, it is
important to have the big guys like JKM (ok i don't know any other
obvious names, but JKM here represent the core team) to say, "ok now
let's address the orm issues and this has to be done before release of
1.1" and so the smart minds of everyone else will start churning AND
making those opinionated decisions that django is so famous for (which
most of time is really brilliant). But no, JKM never has any interest
in this because either he doesn't work in an enterprise environment
anymore (because he's too successful unlike the rest of us still
working in an Office Space envrionement) or django had seen tremendous
growth even without it, why bother. (what i mean to say here is JKM,
please post a note on the main site and mobilize your goons to address
this issue. I am pretty sure it would be done before i go to work
tomorrow).
Finally, don't get me wrong, i love django and python. It is simply
the best language and best web framework out there for me (where
enterprise reporting fit beautifully into a micro-app type of
architecture instead of pylons or turbogears full blown application
architecture for each report). But the orm really drives me crazy. Of
course we all wrote custom managers, and overriding save to deal with
the composite key short comings, but come on, there's gotta be some
more pythonic (battery-included, there's only 1 obvious way) way of
dealing with this.
Frank
On Nov 16, 12:12 pm, "Karen Tracey" <kmtra...@gmail.com> wrote:
>
> > @#$%!.....will this composite primary key feature be included in
> > Django 2.0? Or will this be ever included? Let me explain my
> > frustration a bit.
>
> [snipped]
>
> Did I miss something? Your note makes it sound like the core team actively
> prevented this proposal from being considered for 1.1. I don't believe that
> is the case.
>
> Jacob posted this note:
>
> http://groups.google.com/group/django-developers/msg/0b517e283421034b
>
> to this list on September 15. The last paragraph of that note stated:
>
> Proposing features for 1.1
> ==========================
>
> To propose features for 1.1, make a post on django-dev, and add a link to
Steve Holden
> I guess i thought the core developers would've added this proposal to
> the list (judging from the title of this thread).
>
> Regardless, even if this had been added, it would've been given a -1
> anyway, judging from the readiness of this feature.
>
> More importantly though, my gripe about this is that if you go through
> a search for this feature, you would see entries from 2006. Over the
> last 2 years, there were multiple threads on this and yet nothing was
> really done to it. How many time have we seen attempts at integrating
> sqlalchemy and django? And really what features did people intend to
> add? It's really obvious (at least to me) that it's for the multi db
> support and composite primary key (I mean exactly what else does
> sqlalchemy do better, ok more backends, what else?). Why don't people
> get this?
>
different database front-end (it would be unfair to characterize
SQLAlchemy as an ORM: it's actually quite a lot more than that) is
almost certainly way too radical a step for a point release. It may well
be way too radical a step for release 2. OK, so it's an important use
case for you, but surely you can see that puts you in the minority, and
the development team have to think about the majority first.
> And let me correct myself a bit. It's not that no one has done
> anything, it's just that people have been only writing bits and pieces
> of hacks here and there to patch the flaws, but this problem has
> proven to be more difficult than just patches. For example, there are
> numerous suggestions in this thread in the API of the model. However,
> SO WHAT? does it matter? I like both. Both have merits. THEN WHAT?
> what about the admin layer, what this and that relationship, what
> about these recursive thing. Almost every one of these composite key
> threads had been stalled somewhere either in admin or the model api
> that nothing substantial was done. (usually a thread like this goes on
> a few 10s of exchanges and finally left cold...just like this one).
>
> And I think this is not the developers' fault. I think fundamentally
> there are 2 reasons for this: 1. the original design of the orm is not
> up to the task so that extending it proved to be too difficult. As
> have been mentioned in numerous places, the orm is one place where
> loose coupling should be taken with a grain of salt.
>
>
radical change would be needed for you to have your pony. Or would you
disagree?
> 2 lack of leadership. To deal with this difficult situation, it is
> important to have the big guys like JKM (ok i don't know any other
> obvious names, but JKM here represent the core team) to say, "ok now
> let's address the orm issues and this has to be done before release of
> 1.1" and so the smart minds of everyone else will start churning AND
> making those opinionated decisions that django is so famous for (which
> most of time is really brilliant). But no, JKM never has any interest
> in this because either he doesn't work in an enterprise environment
> anymore (because he's too successful unlike the rest of us still
> working in an Office Space envrionement) or django had seen tremendous
> growth even without it, why bother. (what i mean to say here is JKM,
> please post a note on the main site and mobilize your goons to address
> this issue. I am pretty sure it would be done before i go to work
> tomorrow).
>
>
project, it also mischaracterizes the relationship between Jacob and the
other core developers. Further, it does so in a somewhat unpleasant way
which I hope you will retract, as none of the core developer I've come
across could be described as "Jacob's goons". We don't need this
unpleasantness on a list which is intended to discuss the future of a
large software framework, and it can serve no useful purpose.
> Finally, don't get me wrong, i love django and python. It is simply
> the best language and best web framework out there for me (where
> enterprise reporting fit beautifully into a micro-app type of
> architecture instead of pylons or turbogears full blown application
> architecture for each report). But the orm really drives me crazy. Of
> course we all wrote custom managers, and overriding save to deal with
> the composite key short comings, but come on, there's gotta be some
> more pythonic (battery-included, there's only 1 obvious way) way of
> dealing with this.
>
>
required changes could be made in less than 24 hours is a breathtaking
oversimplification, however. If you believe it, then roll your own
sleeves up and you could surely have it done by the end of the week.
Open source, remember? This means that leadership is a bit like herding
cats. I think you just took a good shot at pissing the cats off.
regards
Steve
Jacob Kaplan-Moss
It's hard for me to figure out how to answer this: if you've got a
problem with my leadership skills, I don't really see how anything I
say makes much of a difference. Frankly, your tone is completely
inappropriate and I feel I'm enforcing absurdly out-of-line behavior
simply by responding.
However, I'm just going to ignore those parts of your post and focus
on the real question.
Support for composite keys has indeed been requested before. In fact,
it's ticket #373; opened about three years ago! On July 20th, 2006, I
commented:
"""
[He]re are the issues [...] that would need to be solved to make this work:
There's three basic problems in dealing with composite primary keys in Django.
The first is that a number of APIs use "obj._meta.pk" to access the
primary key field (for example, to do "pk=whatever" lookups). A
composite PK implementation would need to emulate this in some way to
avoid breaking everything.
Second, a number of things use (content_type_id, object_pk) tuples to
refer to some object -- look at the comment framework, or the admin
log API. Again, a composite PK system would need to somehow not break
this.
Finally, there's the issue of admin URLs; they're of the form
"/app_label/module_name/pk/"; there would need to be a way to map URLs
to objects in the absence of a primary key.
"""
(http://code.djangoproject.com/ticket/373#comment:3)
That's a pretty clear list, and it's been sitting out there for over
two years. I've pointed folks at that comment any number of times
since then, and at some point someone expanded somewhat into a wiki
page (http://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys)
that could serve as a simple spec.
And yet three years on I've yet to see a patch appear on #373. Yes,
this gets asked for time and time again, but nobody seems to want it
enough to write even a *partial* fix. Why should this be?
I think the main reason is that the lack of the feature is quite easy
to work around in most cases. So most people who could fix it just
don't feel like it's worth the time and move on. Somehow, despite the
strum und drang there isn't really enough energy here to prompt anyone
to work up a patch.
Patches are the unit of currency in this community. With very few
exceptions, every one of Django's thousands of commits began life as a
patch posted by someone in the community. We committers can be a lot
more effective when we review and integrate other peoples' patches — I
can review a dozen patches from a dozen different people in the time
it takes me to fix one bug on my own — so by necessity we have to rely
on our community.
If there's a feature you need, implement it. If you can't figure out
where to start, ask — I'm on #django-dev during most of the work week,
and I'd be happy to help anyone who wants to hack on this feature. If
you don't want to or can't implement it yourself, there's a legion of
options available ranging from asking around for help to organizing a
team to contracting someone qualified.
Finally, please keep in mind that the feature list we're drafting
right now isn't set it stone. Anything that gets finished between now
and the feature freeze date for 1.1 (2/15/09) is a candidate for
inclusion. We develop these feature lists to help people figure out
what to work on; nobody's gonna tell anyone not to scratch their own
itch — that's what open source is all about.
Jacob
David Cramer
David Cramer
Director of Technology
iBegin
http://www.ibegin.com/
Jacob Kaplan-Moss
> Also keep in mind that work has been done, but as I haven't had a lot of
> spare time to continue it (attempt #3?). It's a very complex problem as
> well, like Jacob mentioned. You have to deal w/ the legacy use of single
> primary keys, as well as the new composites. While I almost have a fully
> functioning version (barring ForeignKey support) it's still going to take a
> little bit.
If you'll share your code — #373 is begging for a patch... — others
can use it as a starting place.
Jacob
David Cramer
On Nov 16, 5:25 pm, "Jacob Kaplan-Moss" <jacob.kaplanm...@gmail.com>
wrote:
Frank Liu
Let me apologize first for using JKM as an alias to the whole core
team. I guess the problem here is that adding this feature really
entail changes in too many areas. It would be nice if you and the
other core team members can set a direction for changes on this issue.
The various patches suggested in those previous threads all seem to
fix one problem while giving rise to another somewhere else. So having
a direction that fundamentally address this issue i think would be a
better approach.
On the other hand, i have to say that I am not too familiar with this
open source style of working on a project. At my company, issues like
this that span across multiple sub sections usually can only be
resolved by management stepping in. From what I've read here in the
user/dev mailing list, it seems that the only reason that this can't
be resolved is due to lack of leadership from above. (Totalitarian
states usually are pretty efficient in execution...military, another
example...perhaps sqlalchemy could be a third example with MB being
the strong leader). Anyway, i guess by seeing your email I've already
achieved quite a bit. Simply having you recognizing such a problem
again after 2 years i think it's already going somewhere. Perhaps in a
few days we will see some patches coming in.
I am glad to see that you are willing to help. I will try to
contribute if i can although i don't do this by trade.
Oh yeah, as for the tone...I was pretty disappointed not to see this
proposal being included in your list earlier today (it's like Santa
had forgotten you the day of christmas) and it somehow seemed to me
that this list is a final list for 1.1 release.
Plus, if you had a chance to look at the type of databases that I have
to deal with by day, you will see why this is so important to me.
Composite primary is the only non-issue issue at work; you gotta see
the kind of lack of constraints, primary key, proper privilege, tests/
verification/process, bizarre division of schemas and all the other
database craziness at my work, you would understand my frustration.
The django orm + admin/admindocs, with all these beautiful scaffolding/
self-documenting/constraints generating/best practice imposing
goodness, would be a great way to rectify some of that craziness. But
no, it's not too usable because it doesn't handle composite pk (ok,
hacking it would make some parts available--it is working for me right
now, but imagine what a Lumbergh-from Office Space- type of manager
would say...and imagine what all the Java IT guys would say). And
finally, i don't think my frustration is particular to my company/
group, everyone i know to who work in a similar enterprise environment
in the silicone valley seems to have the exact same problems (unless
your company is a hip facebook/web app development company). Leaving
people like us behind I think is a huge missed opportunity for
django.
Frank
On Nov 16, 2:19 pm, "Jacob Kaplan-Moss" <jacob.kaplanm...@gmail.com>
wrote:
David Cramer
declare them in class Meta: primary_key = ('field', 'field', 'field').
There is no ForeignKey/Relational? handlers as of right now, but the
admin is mostly working.
Alex Gaynor pointed out there is one unrelated change I accidently
committed. The list_select_related changes I proposed.
There is also an issue with URL resolving in the admin (its a 1.1
proposal) so it's using a , vs a / for the delimiter, but it's a
simply setting change.
Jacob Kaplan-Moss
> http://github.com/dcramer/django-compositepks/tree/master#
Cool; I'll take a look tomorrow. Do you want me to make changes in my
own tree, or would you rather patches back to you?
Jacob
David Cramer
There's still quite a bit that will need done, mostly with code cleanup, and tests, but I think it will give everyone a pretty good start.
If you'd like I can give you just direct commit access to the branch as well.
Malcolm Tredinnick
On Sun, 2008-11-16 at 15:55 -0800, Frank Liu wrote:
> Hi Jacob,
>
> Let me apologize first for using JKM as an alias to the whole core
> team. I guess the problem here is that adding this feature really
> entail changes in too many areas. It would be nice if you and the
> other core team members can set a direction for changes on this issue.
> The various patches suggested in those previous threads all seem to
> fix one problem while giving rise to another somewhere else. So having
> a direction that fundamentally address this issue i think would be a
> better approach.
>
I'm sorry, but you're essentially 100% incorrect about the
maintainership techniques and a large number of your other claims about
the ORM.
The changes required for composite primary keys are relatively
localised. There's more than one place, but there's a limited number of
them. For example, since a lot of the logic is already encapsulated by
having a "pk" property already, it's mostly those places that need
attention.
Similarly, despite your beliefs, all the maintainers have have been
setting directions for this work in the sense of encouraging, guiding
and providing regular detailed feedback, as the most casual regular
observer on this mailing list could see. But it's not code production by
fiat in the open source world. We aren't going to tell people what to
work on, but we do keep an eye on things and also have our own interests
that we push along. For a couple of years, there's been fairly
coordinated and steady activity towards improvement in various areas,
including the ORM. They remain ongoing.
Again, as with Jacob, I've been reluctant to respond to this thread
because you're just so out of line with your criticisms and the ad
hominem attacks are fairly insulting (and since you mentioned me by name
and by implication in your attacks on Jacob, I am definitely going to
take offence at your mischaracterisations). You happen to want composite
primary keys for your particular situation. But in the whole broad
scheme of every possible web application and database usage, they just
aren't the most common use-case in the world, so will happen in due
course when they're done. Yes, they're on the table; have been for quite
a while. Which is why they're always part of the equation in ORM design
work. The latest round of large ORM changes last year used this case as
one influence over quite a lot of the design, contrary to your
observations (and I can speak with some authority to that point).
If this was/is so blindingly important to you, where's your patch? You
are the one with the apparently show stopping need for this and you get
paid at least 10 times as much as me to work on Django (even if you get
paid $0), so if it's so important to your job, put up or shut up. You
are welcome to use Django. We are glad you do. But that doesn't mean you
get to post crap like this when your favorite feature isn't yet
implemented.
> On the other hand, i have to say that I am not too familiar with this
> open source style of working on a project.
Clearly.
> At my company, issues like
> this that span across multiple sub sections usually can only be
> resolved by management stepping in.
One of the reasons why most corporate software is of much lower quality
than Open Source software.
> From what I've read here in the
> user/dev mailing list, it seems that the only reason that this can't
> be resolved is due to lack of leadership from above.
Nonsense. You're mischaracterising the entire situation. It isn't "not
resolved" (whatever definition of "resolved" you happen to want to
choose). It's ongoing work that hasn't been finished yet. The bar for
quality is high and there are other, simultaneous priorities and limited
resources.
> (Totalitarian
> states usually are pretty efficient in execution...military, another
> example...perhaps sqlalchemy could be a third example with MB being
> the strong leader). Anyway, i guess by seeing your email I've already
> achieved quite a bit. Simply having you recognizing such a problem
> again after 2 years i think it's already going somewhere. Perhaps in a
> few days we will see some patches coming in.
Seriously, put down the weed and step away from the keyboard. Have you
followed any of the discussion about this at all?
David, myself and a number of others have been having ongoing
discussions for a few months in the latest round. It's progressing
fairly nicely, given the time we have available and all the other pieces
we're all working on as well. As mentioned above, it's also been one of
the many, many considerations when designing other pieces of the
infrastructure and has been duly waiting in the queue for the
appropriate moment to start making the changes (which is around now).
The reason the ticket has been open is to indicate it's ongoing work and
we haven't been unaware of it. It's just not done yet (see, again,
priorities and lack of a complete patch that meets the quality
standards).
This whole sub-thread has been quite disappointing.
Regards,
Malcolm
Rock
with Oracle. To support model partitioning for selected models, we
munged the generated SQL for model creation to add composite keys and
the partitioning logic. It was a little bit hairy, but once the model
creation was correct, everything else "just worked". No server hacks
or avoidance of the Django ORM was required to run the apps.
Sadly the code was special cased for the specific form of partitioning
used and therefore not suitable for release as open source, but the
code was not exceptionally hard to write. So get to it. If it really
is too tough for you, there are some hotshot Django consultants around
who may have some free cycles on rare occasions and might be willing
to help you.
Hanne Moa
> Plus, if you had a chance to look at the type of databases that I have
> to deal with by day, you will see why this is so important to me.
Yo. Not that big, but there's a DBA involved for some of 'em. Let's
just say I can't use the auth-system or anything depending on that :)
Funny thing though, got a manytomany-table without a primary key: I
can add a new row easily enough, the only problem is in delete()-ing
specific rows, since that seems to use the primary key. So I clear()
and re-add instead of delete()-ing, since that doesn't use the primary
key. Fragile, but it beats doing the puppy-eyes to the DBA.
HM
Joey Wilhelm
I ran into that exact problem myself and filed the following ticket related to the issue: http://code.djangoproject.com/ticket/9519
I also found that you could run your own delete operation through the ORM, however, to work around this. It's more work and certainly not in line with DRY or any of the normal sensibilities... but it does at least work. Here's an example of that, built off the example in the ticket: http://dpaste.com/92116/
David Cramer
Hanne Moa
> You guys are really bad about keeping on topic. :)
Good way to prevent a discussion from entering Godwin-territory :)
I should perhaps have mentioned something along the lines of "this is
an example of where there is (perhaps) an unnecessary need for a
primary key". After all, the fewer places that *must* have x, the
easier it is to change the underlying structure of x.
To go back off track (wouldn't do to stay on it for long), maybe we
need an official place to discuss the use of legacy databases in
Django.
HM
Alex Gaynor
So here we are 4 years later, version 1.5 is about to be released and we still don't have a solution for composite primary keys in the code base? Or am I missing it in the docs somewhere? --
You received this message because you are subscribed to the Google Groups "Django developers" group.
To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/UnrMyhTujxIJ.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
Jacob Kaplan-Moss
> So here we are 4 years later, version 1.5 is about to be released and we
> still don't have a solution for composite primary keys in the code base? Or
> am I missing it in the docs somewhere?
pretty hostile. That's not a great way to deal with a community of
volunteers who work on stuff in their free time; it makes it sound
like you think you're entitled to our free work.
If that's not your intention -- and I hope it's not! -- please check
your tone next time. You'll find you'll be a lot more productive
around here if you're polite and offer to help.
Jacob
Ron Smith
--
You received this message because you are subscribed to the Google Groups "Django developers" group.