Proposal for new field type: CompositeField

[Michael P. Jung]

Quite often I find myself in the need of fields composed of other
fields. e.g.

class AddressField(CompositeField):
addressee = models.CharField(max_length=50)
street = models.CharField(max_lenght=50)
zipcode = models.CharField(max_lenght=10)
# ... more fields here

class Foo(models.Model):
billing_address = AddressField()
shipping_address = AddressField()

I implemented [1] this idea by providing a custom meta class and
contribute_to_class method. The given example would creates the fields
'billing_address_addresee', 'billing_address_street', etc. as well as
'shipping_address_addressee', 'shipping_address_addressee', etc.

The fields 'billing_address' and 'shipping_address' in the Foo class are
properties, which provide a set and get method. The setter is capable of
copying all contained fields while the getter returns a Proxy object
which implements __getattr__, __setattr__ and __cmp__.

This makes it possible to use these fields quite naturally:

foo = Foo()
foo.shipping_address.addressee = 'Max Mustermann'
foo.shipping_address.street = 'Musterstr. 1'
foo.shipping_address.zipcode = '12345'
# ...
if foo.billing_address != foo.shipping_address:
foo.billing_address = foo.shipping_address

One should be aware that the proxy works by assigning the fields to the
underlying model. This causes the following two snippets to do something
different:

1.
foo = Foo()
foo.shipping_address = ...
# copy all shipping_address sub fields to billing_address
foo.billing_address = foo.shipping_address

2.
foo = Foo()
foo.billing_address = foo.shipping_address
# billing_address sub fields aren't affected
foo.shipping_address = ...

This is quite counter intuitive, when thinking of those composite fields
as 'references'. It was the only way to implement this feature without
having to fiddle with the ORM internals. ForeignKey fields work simmilar
- you have to make sure the referenced object has an id set or you're
screwed - so I don't consider this too bad.

Nesting of CompositeFields is not possible right now, but shouldn't be a
big deal to implement. I just didn't ran across a use case, yet, so I
didn't want to spend any time on this.

Feel free to grab the code from [1] and tell me what you think. I tried
to provide some meaningful test cases which also act as example code.

I guess this could also provide a nice foundation for #373 [2].

[1] https://hg.labs.terreon.de/common/chimes/
[2] http://code.djangoproject.com/ticket/373

--mp

[Stephen Crosby]

Michael,

In your address example, I'm not sure what advantage this CompositeField has over just creating an address model/table and using a regular foreign key to reference it. You mentioned the need to create a field composed of other fields which to me sounds a lot like a regular one-to-many relationship between two normalized tables. Can you explain the reasons you decided to do it this way and the advantages you see?

--Stephen

--

You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Michael P. Jung]

On 2009-12-24 19:45, Stephen Crosby wrote:
> In your address example, I'm not sure what advantage this
> CompositeField has over just creating an address model/table and
> using a regular foreign key to reference it. You mentioned the need
> to create a field composed of other fields which to me sounds a lot
> like a regular one-to-many relationship between two normalized
> tables. Can you explain the reasons you decided to do it this way and
> the advantages you see?

Actually the whole idea of this is to store the data denormalized. I
have to agree that the given example makes little to no sense, as it is
lacking a sane context. In one of our applications the AddressField is
used in several models, but the stored addresses are never shared, thus
normalizing them brings no benefit. Contrary, a separate table would
cause more complex and slower queries.

I agree that a 1-to-N relationship, with N being some rather low
constant number, shouldn't be realized that way, but I find it quite
practical for true 1-to-1 relationships.

Better and less synthetic examples for composed fields would be:

class Length(CompositeField):
value = models.DecimalField(...)
unit = models.CharField(choices=UNIT_CHOICES)

class Location(CompositeField):
longitude = models.FloatField()
latitude = models.FloatField()
altitude = models.FloatField()

class FullNameField(CompositeField):
prefix = models.CharField(max_lenght=20, blank=True)
first = models.CharField(max_lenght=20)
middle = models.CharField(max_lenght=20, blank=True)
last = models.CharField(max_lenght=20)
suffix = models.CharField(max_lenght=20, blank=True)

class PhoneNumber(CompositeField):
country = models.CharField(max_lenght=4, blank=True)
area = models.CharField(max_lenght=8, blank=True)
subscriber = models.CharField(max_lenght=16)

I hope all that makes a bit more sense now?

--mp

[Jerome Leclanche]

I like it, always wondered if there was a way to do this in the core. Hope the patch gets through.

J. Leclanche / Adys

--mp

[VernonCole]

Let me give a hearty YES to this proposal. I have been told that a
composite field, like this, is a violation of good database design,
and that a 'true' relational database cannot have this feature. That
is why the guy who designed the database I helped implement in 1982
did not call his design 'relational'. He wanted the capability and
added it. I still use that ancient database language (RDM -- you've
never heard of it -- worked on PDP-11s) today and still love the
feature. It's just darn handy.
To the 'relational database' purists, my reply is this:
You are already using SQL, which is NOT a relational database
language. Codd himself said so. Look it up.
--
Vernon Cole

> > django-develop...@googlegroups.com<django-developers%2Bunsu...@googlegroups.com>

[Andy Mikhailenko]

Maybe I'm missing something, but I don't understand how is this
different from having a bunch of separate fields. The CompositeField
adds a namespace, but foo.bar_x=1 seems to be no harder to read than
foo.bar.x=1. I must admit that this field solves another problem well:
it makes easier to copy sets of fields. However, if this is done
frequently, the chances are high that denormalization is what we
actually need. So I think this field may be great in some use cases,
but in most cases it would rather overcomplicate things. Feel free to
ignore this comment if I'm missing the point :)

Andy

[Michael P. Jung]

On 2009-12-26 05:17, Andy Mikhailenko wrote:
> Maybe I'm missing something, but I don't understand how is this
> different from having a bunch of separate fields. The CompositeField
> adds a namespace, but foo.bar_x=1 seems to be no harder to read than
> foo.bar.x=1. I must admit that this field solves another problem
> well: it makes easier to copy sets of fields.

It's not just about copying sets of fields, but rather implement custom
types, which are separated into more than one database column. It's all
about not having to repeat yourself and ensuring consistency when using
those field types.

Another great example for a CompositeField is the LocalizedField, which
I just completed and pushed [1] to the repository.

The implementation is pretty simple and makes defining columns, that
need translation a breeze. django-multilingual [2] solves the same thing
in a somewhat different way with extra tables.

On 2009-12-26 05:17, Andy Mikhailenko wrote:
> However, if this is done frequently, the chances are high that
> denormalization is what we actually need. So I think this field may
> be great in some use cases, but in most cases it would rather
> overcomplicate things. Feel free to ignore this comment if I'm
> missing the point :)

Denormalization is surely about to become a killer feature. Though it
has a somewhat different scope than composite fields.

Composite fields are about combining data that is never ment to be
separated, while denormalization is about speeding up database access.
Just think of complex numbers. I guess most people wouldn't implement it
using an intermediate table. And those who do are probably going to find
their code sooner or later at The Daily WTF [3]. ;-)

The more I think about this I feel that this could be the best example
for a CompositeField so far:

class ComplexNumberField(CompositeField):
real = models.IntegerField()
imag = models.IntegerField()
def get_proxy(self, model):
proxy = super(ComplexNumberField, self).get_proxy(model)
return complex(proxy.real, proxy.imag)
def set_all(self, model, value):
proxy = super(ComplexNumberField, self).get_proxy(model)
proxy.real = value.real
proxy.imag = value.imag

Accessing a field of the type ComplexNumberField will not return a
proxy, but rather a complex number. This is a special case since complex
is an immutable type. This also made me think that get_proxy shouldn't
be directly called from the property.

[1] https://hg.labs.terreon.de/common/chimes/rev/7d7059656f5d
[2] http://code.google.com/p/django-multilingual/
[3] http://thedailywtf.com/

--mp

[Michael P. Jung]

I just implemented [1] the ComplexNumberField using the CompositeField
class. It also shows that it is possible to change subfield attributes
inside the __init__ method since the subfields are deep copied for every
instance.

I hope that the test cases are mostly complete. I plan on writing down
the documentation as soon as I've gathered some more feedback.

[1] https://hg.labs.terreon.de/common/chimes/rev/bfce45dd4367

--mp