Composite Primary Keys

[Cristiano Coelho]

Hello there,

What's the status for this? This (https://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys) is 3 years old (last edit) and the links on it are even older. Googling around only gave me some very old projects so it wasn't good neither.

[Roger Gammans]

Hi,


We've got some patches that we are using in production, against
1.8 . Out long term aim would be to get them ready for a PR , but I they
are missing all sorts of things, such unit tests and migrations (we
don't use migrations at the moment - long story, so these haven't been
tested). The other thing which I think is maybe is some of the
prefetch_related API, I've had difficulties with it and just worked
around it in my app.

And it really needs another user at least to kick the tires which we
aren't exercising - we mainly use because we have a lot of intersection
tables of natural / 'foreign' keys and a legacy model which didn't add a
serialized Pk for these.

I'd also like to know how interested the core developers would be in
merging some support for this.

> --
> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-develop...@googlegroups.com.
> To post to this group, send email to
> django-d...@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/b2873ece-39bb-4536-b23d-d988c7122204%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[bliy...@rentlytics.com]

Sounds cool, I think we would totally use that.

[Michal Petrucha]

Hi Cristiano,

I personally have spent in total perhaps about a year (or even more)
working on this (which includes two Summer of Code programs).
Unfortunately, my work was spread over more than three years, which
means a significant chunk of effort went into keeping up with changes
in Django and, ultimately, that's the part at which I failed. Twice.
(With a patch of this magnitude, all you need is to fall behind by
three weeks, and it gets very difficult and/or time-consuming to merge
new changes into it.)

The latest effort was launched by Thomas Stephenson, and resulted in
the following DEP:
https://github.com/django/deps/blob/master/draft/0191-composite-fields.rst,

There was even a pull request, but eventually, Thomas withdrew his
patch. You can find the corresponding thread on this mailing list
here:
https://groups.google.com/d/msgid/django-developers/ed326cce-6784-429b-869b-f6f66d3c77fd%40googlegroups.com

I wasn't around at that particular time, and I haven't yet found the
time to carefully read that thread and let it sink in, but I can give
a summary of my observations from 2013, when I last worked on this.

If you're interested, you can take a look at the wall of text I wrote
as my GSoC proposal in 2013: https://gist.github.com/koniiiik/5408673.
You might also want to search this mailing list for previous threads
about this topic.


I seem to recall that just the most basic functionality of
CompositeField was the easy part. At that time (before migrations
landed), it meant mostly just making some parts of Django play nicer
with virtual fields (which was an almost entirely unexplored concept
in Django back then), some hackery for query lookups (since we didn't
have the nice API for custom lookups and transforms), and a fancy
class to represent composite values in model instances. Whipping up
something like this takes a few days, at most weeks, of focused work,
and you can have something sort of usable, with many sharp edges that
you have to keep in mind.

Things start to get ugly when you try to make CompositeField play nice
with the rest of the ORM, and the most obvious troublemaker is
ForeignKey. Composite ForeignKey is something that requires a big
refactor of the way related fields work, but even with a simple
ForeignKey to a unique column on a model with a composite primary key,
you'd start stumbling on errors with Python-level delete cascades,
prefetching and such -- those require an overhaul of how __in lookups
are handled. Then there are things like generic foreign keys, database
migrations, introspection, and potentially other features that would
need to be taken into account. Most of them could be postponed to a
later time, but some of them would most likely have to be part of the
initial patch.

Anyway, with the recent _meta refactor, reimplementing ForeignKey as a
virtual field should hopefully be easier now, although no one will
know for sure unless someone actually steps up and tries to do it. I
certainly hope that I'll be able to tackle this once again, but I
cannot make any promises.


Cheers,

Michal