How do I make Django evaluate a ForeignKey for json serialization?

[adambossy]

I have a model that refers to a ForeignKey, e.g.:

class Template(models.Model):
type = models.ForeignKey(Type)
...

class Type(models.Model)
name = models.CharField(max_length=32)
...

I can fetch these objects fine using the model API.

t = Template.objects.get(id=1)
print t.type
>> <Type: Type object>
print t.type.id
>> 3
print t.type.name
>> TemplateType3

If I serialize these, I cannot get 'name.' I know Django lazily
retrieves these, but even by iterating or calling t.type.name, they
are unavailable during serialization. Is there any way to have Django
evaluate the ForeignKey object so that I can retrieve the field,
'name?' I need fields to contain "name": "TemplateType3" (or whatever
it may be)! :)

from django.core import serializers
print serializers.serialize('json', Template.objects.get(id=1))

>> {
"pk": 1,
"model": "template",
"fields": {
"type": 3
}
},

Thanks,
Adam

[Russell Keith-Magee]

It isn't entirely clear what you're trying to do. Do you want to
change the Django serializers so that they output:

"type": "TemplateType3"

If this is the case, then you will need to write your own serializer.
Django serializes objects in a specific way, optimized for
deserialization later on as fixtures. This output format won't suit
many applications. If you need a different format, you will need to
write a serializer that outputs in the format you require. Django
ships with SimpleJSON, so you can build serialized output from
primitives if need be. You could also use Django's serialization
library to write your own serializer that outputs in a format of your
choice. If your desired output format is similar to Django's default
format, you may find that you can use django/core/serializers/json.py
as a starting point for this implementation.

Alternatively, are you trying to manipulate a deserialized object in
some way? If this is the case, you will need to provide more specific
details on what it is you are trying to achieve.

Yours
Russ Magee %-)

[adambossy]

Russ,

Thanks for the reply. Specifically, I am wondering if there is some
feature of the Models API that allows me to retrieve the foreign key
object so that it is included in the serialized string. That is,
without writing my own serializer (which I suspect I may have to do).
Alternatively, I was hoping somebody may be able to offer a technique
for passing messages that I may have overlooked. With queries such as
those in my example, I may be retrieving m Type objects, each of which
may have n unique Class objects with foreign key pointers, leading to
a whole mess of data to be passed in the json, making it a nightmare
to deserialize it on the other end.

Throughout my application, I am using SimpleJSON for some queries, and
serializers.serialize for others. SimpleJSON is useful for python
dictionaries and serializers.serialize is good for QuerySets.
Ultimately, though, all the information I am passing is retrieved from
QuerySets; my choice of serializer is based on whether I am iterating
through the QuerySets and writing the data to python dictionaries
(simplejson) or simply passing them directly to a response object
(serializers.serialize). The reason for this is that the object vary
in complexity and sizel for simple queries, I'll just pass the whole
QuerySet to serialize() and for complex ones I'll break it down and
create a dictionary to pass through simplejson.

I suppose that my overall intent is a cry for help for a simpler, one-
size-fits-all solution that can allow me to easily serialize objects,
yet pass small messages. Perhaps a middle layer between Django and my
front end (the Google Web Toolkit) is necessary. The fact that GWT
only has basic JSON support doesn't help, as every message I pass
requires a custom handler to decode the JSON.

I'll keep tinkering... Let me know if you have any ideas.

Adam


On Dec 27, 4:45 am, "Russell Keith-Magee" <freakboy3...@gmail.com>
wrote:

[Russell Keith-Magee]

On Sun, Dec 28, 2008 at 5:32 AM, adambossy <adam...@gmail.com> wrote:
>
> Russ,
>
> Thanks for the reply. Specifically, I am wondering if there is some
> feature of the Models API that allows me to retrieve the foreign key
> object so that it is included in the serialized string. That is,
> without writing my own serializer (which I suspect I may have to do).

The serializer is decoupled from the Model API, so your question is
unrelated to the model API. Serialization exploits the model metatdata
embedded in models; the issue in this case is that the data extracted
from the model by the serializer doesn't meet your requirements. You
want to have a different serialization format (i.e., you want to roll
out serialized objects in a different format to that provided by
default). This means you will need to write your own serializer.

Looking for deeper solutions to this problem -

Ticket #4656 possibly describes part of your problem - this ticket
describes an enhancement that would allow the serializer to follow.
This would be much like the select_related() operator on querysets,
but for serialization. This ticket hasn't seen any activity for a
while, but it has been accepted for inclusion as soon as a working
implementation is available.

There is also a lingering ticket idea floating around - it has been
suggested many times that Django's serializers are not flexible enough
- that your use case should be a matter of configuring Django's
serializer, rather than writing a new serializer from scratch. In
principle, I have no objection to this idea - but I am yet to see a
good proposal for what such configuration would look like.

> I'll keep tinkering... Let me know if you have any ideas.

Let me know if you have any :-)

Yours
Russ Magee %-)

[mattim...@gmail.com]

On Dec 28, 10:01 am, "Russell Keith-Magee" <freakboy3...@gmail.com>
wrote:

> On Sun, Dec 28, 2008 at 5:32 AM, adambossy <adambo...@gmail.com> wrote:
>
> > Russ,
>
> > Thanks for the reply. Specifically, I am wondering if there is some
> > feature of the Models API that allows me to retrieve the foreign key
> > object so that it is included in the serialized string. That is,
> > without writing my own serializer (which I suspect I may have to do).
>
> The serializer is decoupled from the Model API, so your question is
> unrelated to the model API. Serialization exploits the model metatdata
> embedded in models; the issue in this case is that the data extracted
> from the model by the serializer doesn't meet your requirements. You
> want to have a different serialization format (i.e., you want to roll
> out serialized objects in a different format to that provided by
> default). This means you will need to write your own serializer.
>
> Looking for deeper solutions to this problem -
>
> Ticket #4656 possibly describes part of your problem - this ticket
> describes an enhancement that would allow the serializer to follow.
> This would be much like the select_related() operator on querysets,
> but for serialization. This ticket hasn't seen any activity for a
> while, but it has been accepted for inclusion as soon as a working
> implementation is available.

The code attacted #4656 can be used as is for achieving the kind of
serialization that Adam requires. I'm am also using it quite
extensively in a large RESTful application. You can register my
serializer classes to replace the default Django json serializer. In
your settings.py add:

SERIALIZATION_MODULES = { 'json' : 'wadofstuff.serializers.json' }

Then in your own code from your example above

from django.core import serializers
print serializers.serialize('json', Template.objects.get(id=1),
relations=('type',))

{
"pk": 1,
"model": "template",
"fields": {

"type": {
"pk": 3,
"model": "type",
"fields": {"name": "TemplateType3"}
}
},


cheers

Matthew

--
Matthew Flanagan
http://wadofstuff.blogspot.com