Composite Fields

36 views
Skip to first unread message

David Cramer

unread,
Sep 16, 2008, 2:51:22 AM9/16/08
to Django developers
So one of the requirements for composite primary keys (and in fact the
last thing I need to handle) is composite fields. Right now there's a
mess for handling the .pk alias (and the new .pks alias), which will
be able to be moved into a composite field. This is also required in
order to do ForeignKey(ModelWIthCompositePrimaryKey).

I want to open this up to people who are more familiar with the
underlying API, and who care much more about this than myself, and see
what you all think is reasonable for an API to handle multiple
database fields, in one field instance.

Malcolm Tredinnick

unread,
Sep 16, 2008, 7:36:12 AM9/16/08
to django-d...@googlegroups.com
Hi David,

On Mon, 2008-09-15 at 23:51 -0700, David Cramer wrote:
> So one of the requirements for composite primary keys (and in fact the
> last thing I need to handle) is composite fields. Right now there's a
> mess for handling the .pk alias (and the new .pks alias),

I had some recollection that you'd posted a reference to a patch in an
earlier thread on this, but now I can't find it. Is the current work
you've done online somewhere. Tomorrow or Thursday (my time), I'm going
to update my composite column stuff to trunk so that I can refresh my
brain about what the remaining steps were and I'd like to make sure it
fits in as smoothyl as possible with the work you're doing.

Hopefully there won't be a need to introduce any second attribute here
(and it can't be called "pks" in any case, since that's the same problem
as before: by definition alone, there is only one pk on any table/model.
Calling it by a plural name is confusing). It's probably fairly natural
to just use "pk" always, since it can return either a single value or a
tuple in the appropriate circumstances.

> which will
> be able to be moved into a composite field. This is also required in
> order to do ForeignKey(ModelWIthCompositePrimaryKey).
>
> I want to open this up to people who are more familiar with the
> underlying API, and who care much more about this than myself, and see
> what you all think is reasonable for an API to handle multiple
> database fields, in one field instance.

The approach I've been taking is basically a generalised version of
GenericForeignKeys: We already have all the infrastructure in place for
declaring individual fields that correspond to a column. Trying to
combine multiple values of that nature into a single class rapidly looks
ugly and hard to read. So, instead, we declare the individual column
types and then declare the composite field as an ordered combination of
those individual types. Haven't quite worried about the final form of
the API yet, but it was converging on:

latitude = DecimalField(max_digits=6, decimal_places=3)
longitude = DecimalField(max_digits=6, decimal_places=3)
lat_long = CompositeField(latitude, longitude)

After the declaration, one can simply work directly with the "lat_long"
attribute in filters and as a class attribute and ignore "latitude" and
"longitude", or talk to the individual attributes if that is more
appropriate (analogously to working with either the pair in all database
interactions or with the columns individually). Filtering in querysets
on one or other of the constituent pieces of a composite field will
usually make sense.

Foreign keys are then straightforward to use: you just refer to the
remote model, using to_field if required, and the ForeignKey class will
always know the number of target columns, since it's specified up front.

When I was thinking about this a bit before DjangoCon, I realised it
possibly means we don't have to worry about declaring a Meta.primary_key
attribute. We could just support "primary_key=True" on the
CompositeField class and that would make ForeignKey references the same
as before (ForeignKey(OtherModel) will just refer to the primary key
field, be it a single column one or a composite beast).

Thoughts?

Malcolm

sector119

unread,
Sep 16, 2008, 8:09:34 AM9/16/08
to Django developers
Malcolm, may be it is possible to make db api not depend on primary
key field(s) at all? There are to many tasks where PK, specially
serial field, not required and useful..

Malcolm Tredinnick

unread,
Sep 16, 2008, 8:44:43 AM9/16/08
to django-d...@googlegroups.com

That's an entirely separate issue and not part of this thread. Please
don't jumble up unrelated things.

I think, though, that if you tried to remove this dependency, it would
be very difficult. There are lots of subtle consequences.

Regards,
Malcolm


Rock

unread,
Sep 16, 2008, 10:36:59 AM9/16/08
to Django developers
Malcolm,

I strongly concur with the proposed API. It looks like it can be used
to implement future RangeHash partitioning support. Consider:

I have a model to be partitioned that has an IntegerField that will
become the range partitioning value:

week = models.IntegerField() # 1 to 53 depending on the week of
creation

The hash partitioning value is a field that works precisely like "id":

hash_id = models.IntegerField( auto_everything=True ) # (I made
"auto_everything" up as I am not sure precisely what parameters I need
to set in order to make this happen.)

Now I can create my RangeHash partitioning primary key with a
compositeField:

range_hash = models.CompositeField( hash_id, week, primary_key=True )


What I am doing today for my partitioned models is to use the
automatically generated "id" field, but that looks dicey with the
CompositeField since you can't do this:

range_hash = models.CompositeField( id, week, primaryKey=True )

since "id" doesn't exist yet and, by designating this CompositeField
as the primary key, id won't be generated. Rather than try to special
case this situation, ensuring that I can create a field that works
precisely like "id" by hand and then using that seems to be the way to
go.

Rock

David Cramer

unread,
Sep 16, 2008, 4:37:50 PM9/16/08
to Django developers
I think that will work Malcom. One thing we'll need to deal with,
is .name, .attname, etc on these. It should act the same way on the pk
as it does on the multi-column field (because after all, this will be
just a wrapper). Right now .pks is just an alias for .pk.as_tuple or
something. I had it throwing errors if you tried accessing .pk.name if
there were multiple fields in it, as name is the field name, and the
field name becomes multiple.

I will post up what I have done later tonight.

Malcolm Tredinnick

unread,
Sep 16, 2008, 7:41:26 PM9/16/08
to django-d...@googlegroups.com

On Tue, 2008-09-16 at 13:37 -0700, David Cramer wrote:
> I think that will work Malcom. One thing we'll need to deal with,
> is .name, .attname, etc on these. It should act the same way on the pk
> as it does on the multi-column field (because after all, this will be
> just a wrapper). Right now .pks is just an alias for .pk.as_tuple or
> something. I had it throwing errors if you tried accessing .pk.name if
> there were multiple fields in it, as name is the field name, and the
> field name becomes multiple.

The name attribute can return a tuple for CompositeFields. I've actually
added a multicolumn attribute to the Field class so that one isn't stuck
have to do isinstance() calls all the time; we can key off
field.multicolumn (True or False), although I've got a fixme to think
about calling that something less fiddly to type and less
database-specific. Once we start treating them like proper fields that
can have sequence values, I think it all falls intro place fairly
easily.

Regards,
Malcolm

Michael Radziej

unread,
Sep 18, 2008, 4:09:18 AM9/18/08
to django-d...@googlegroups.com
On Tue, Sep 16, Malcolm Tredinnick wrote:

> Hopefully there won't be a need to introduce any second attribute here
> (and it can't be called "pks" in any case, since that's the same problem
> as before: by definition alone, there is only one pk on any table/model.
> Calling it by a plural name is confusing). It's probably fairly natural
> to just use "pk" always, since it can return either a single value or a
> tuple in the appropriate circumstances.

> [...]

I really like this approach! It could also replace unique_together.

Cheers,

Michael

--
noris network AG - Deutschherrnstraße 15-19 - D-90429 Nürnberg -
Tel +49-911-9352-0 - Fax +49-911-9352-100
http://www.noris.de - The IT-Outsourcing Company

Vorstand: Ingo Kraupa (Vorsitzender), Joachim Astel, Hansjochen Klenk -
Vorsitzender des Aufsichtsrats: Stefan Schnabel - AG Nürnberg HRB 17689

Reply all
Reply to author
Forward
0 new messages