more info when serializing

[Bryan L. Fordham]

So, say I have a model something like this:

class Bar(models.Model):
user = models.ForeignKey(User)
name = models.CharField(maxlength=50)
description = models.TextField()

where user is tied to a django.contrib.auth.models.User entity. When I
serialize this to json, I get:
[{"pk": "1", "model": "foo.bar", "fields": {"description": "", "user":
1, "name": "Phone"}}]

Which is fine, but I need both the username and user id on the front
end. Is there a simple way to get this info all in one shot that I'm
just missing?

Ideally, it would return something like: ..."user":
{"username":"bfordham", "pk":1}...

I'm using the django-rest-interface, if that makes a difference one way
or the other.

Thanks
--B

[Malcolm Tredinnick]

On Sat, 2007-11-17 at 16:21 -0500, Bryan L. Fordham wrote:
> So, say I have a model something like this:
>
> class Bar(models.Model):
> user = models.ForeignKey(User)
> name = models.CharField(maxlength=50)
> description = models.TextField()
>
> where user is tied to a django.contrib.auth.models.User entity. When I
> serialize this to json, I get:
> [{"pk": "1", "model": "foo.bar", "fields": {"description": "", "user":
> 1, "name": "Phone"}}]
>
> Which is fine, but I need both the username and user id on the front
> end. Is there a simple way to get this info all in one shot that I'm
> just missing?
>
> Ideally, it would return something like: ..."user":
> {"username":"bfordham", "pk":1}...

The serializer doesn't support this. It would take a bit of design work
to figure out how to specify such an extension easily. You might like
put some thought into that, though.

Things that immediately spring to mind as requiring addressing:
- what should the format look like for many-to-many objects (and
making sure we can deserialise as well)
- once you add one level of indirection, it will take
approximately 8 seconds for somebody to want to go two levels
and more. So it should be extensible beyond just "the immediate
foreign key relative".
- reverse relations
- deserialisation, if possible.
- is it easy to say "all the normal fields plus these extra
ones".
- could passing a ValuesQuerySet to the serializer, together
with #5768 be a solution in some fashion (as a way of specifying
these extra fields)?

I don't know how easy this would all be or how genuinely useful. Could
be worth a bit of thinking, though.

Malcolm

--
I just got lost in thought. It was unfamiliar territory.
http://www.pointy-stick.com/blog/

[Ben Ford]

You could use a objects.values(...) which return a dictionary if I remember correctly, and then simplejson.dumps...
Ben

--
Regards,
Ben Ford
ben.f...@gmail.com
+6281317958862

[bfor...@socialistsushi.com]

> The serializer doesn't support this. It would take a bit of design work
> to figure out how to specify such an extension easily. You might like
> put some thought into that, though.

For my specific case, I wrote a custom serializer, extending the current
json one, that added the additional info I want from User. It won't
deserialize properly, but I don't care for this application.

I put a bit of thought into it this afternoon, and I may play around with
seeing if any of them would actually work. What I have is good enough for
me in this particular case, but again it's little more than a hack, and
certainly not something that should be merged in anywhere 8)

Thanks
--B

[bfor...@socialistsushi.com]

> You could use a objects.values(...) which return a dictionary if I
> remember
> correctly, and then simplejson.dumps...

I'll play with this and see how it works out tomorrow

Thanks
--B

[mattim...@gmail.com]

On 18 Nov, 11:25, Malcolm Tredinnick <malc...@pointy-stick.com> wrote:
> On Sat, 2007-11-17 at 16:21 -0500, Bryan L. Fordham wrote:
> > So, say I have a model something like this:
>
> > class Bar(models.Model):
> > user = models.ForeignKey(User)
> > name = models.CharField(maxlength=50)
> > description = models.TextField()
>
> > where user is tied to a django.contrib.auth.models.User entity. When I
> > serialize this to json, I get:
> > [{"pk": "1", "model": "foo.bar", "fields": {"description": "", "user":
> > 1, "name": "Phone"}}]
>
> > Which is fine, but I need both the username and user id on the front
> > end. Is there a simple way to get this info all in one shot that I'm
> > just missing?
>
> > Ideally, it would return something like: ..."user":
> > {"username":"bfordham", "pk":1}...

Hi,

I've come up with a solution to this that is very similar to the way
Rails does it [1]. I'm also CC-ing to django-developers as it covers
ticket #4656 and feel it is wandering more into the developer realm
where a design decision is needed.

I've written a new python & json serializer for Django. An example
invocation and output for the Bar model above would look like:

>>> from django.core import serializers
>>> serializers.serialize('json', Bar.objects.all(), relations={'user':{'fields':('username',)}})
[{

"pk": "1",
"model": "foo.bar",
"fields": {
"description": "",

"user": {"pk":"1", "model":"auth.user", "fields":
{"username":"bfordham"}},
"name": "Phone"
}
}]

The new serializer takes the following keyword arguments:

fields - list of fields to be serialized only.
excludes - list of fields to be excluded.
relations - list of related fields to be serialized or a
dictionary with the keys being the fields to serialize and the values
being a dictionary of arguments to pass to sub-serializer. ie. fields,
excludes, relations, methods.
methods - list of methods to include. Methods cannot take
arguments. Not yet implemented.
attributes - list of other class attributes to serialize that
aren't model fields. Not yet implemented.

At the moment with no arguments it only serializes regular model
fields that aren't ForeignKeys or ManyToManys. You have to explicitly
include related fields using the 'relations' argument. I could make
this backwards-compatible with the way the serializers work now if
needed.

>
> The serializer doesn't support this. It would take a bit of design work
> to figure out how to specify such an extension easily. You might like
> put some thought into that, though.
>
> Things that immediately spring to mind as requiring addressing:
> - what should the format look like for many-to-many objects (and
> making sure we can deserialise as well)

Here is how I do it with User model as an example:

>>> serializers.serialize('python', User.objects.filter(username='jdoe'), relations=('groups',))
[{
'fields': {'date_joined': datetime.datetime(2006, 8, 14, 10, 34,
56),
'email': u'john...@example.com',
'first_name': u'',
'groups': [{'fields': {'name': u'group1'},
'model': u'auth.group',
'pk': 2},
{'fields': {'name': u'group2'},
'model': u'auth.group',
'pk': 1}],
'is_active': True,
'is_staff': True,
'is_superuser': False,
'last_login': datetime.datetime(2007, 9, 19, 12, 29, 2,
649167),
'last_name': u'',
'password': u'sha1$xxxxxxxxxxxxxxxxxxxxx',
'username': u'jdoe'},
'model': u'auth.user',
'pk': 2}]

> - once you add one level of indirection, it will take
> approximately 8 seconds for somebody to want to go two levels
> and more. So it should be extensible beyond just "the immediate
> foreign key relative".

My serialiser allows for as many levels of indirection as you want as
well as specifying which fields you want included, excluded etc at
each level.

> - reverse relations

Not yet implemented.

> - deserialisation, if possible.

Not yet implemented.

> - is it easy to say "all the normal fields plus these extra
> ones".

The attributes and methods arguments are for this, though I haven't
implemented that part either and I'm thinking of merging the two
arguments into one but haven't thought of a good name. Perhaps
"extras"?

> - could passing a ValuesQuerySet to the serializer, together
> with #5768 be a solution in some fashion (as a way of specifying
> these extra fields)?
>
> I don't know how easy this would all be or how genuinely useful. Could
> be worth a bit of thinking, though.
>
> Malcolm

I wrote this because I have a need for it. I'd also like to use it
with the django_rest_interface project but that would need some
largish changes for it to work and I'm not certain if it fits in to
the REST way of doing things.

What do you think?

[1] http://dev.rubyonrails.org/browser/trunk/activerecord/lib/active_record/serializers/json_serializer.rb

regards

matthew
--
Matthew Flanagan
http://wadofstuff.blogspot.com

[mattim...@gmail.com]

Hi,

With a bit of prompting from another Djangoista I've posted the python
and json "full" serializer implementations to http://code.djangoproject.com/ticket/4656
.

Please have a try and let me know if you find and bugs or have any
other feedback.

cheers

Matthew

On Dec 14 2007, 5:02 pm, "mattimust...@gmail.com"

> 'email': u'john....@example.com',

> [1]http://dev.rubyonrails.org/browser/trunk/activerecord/lib/active_reco...
>
> regards
>
> matthew
> --
> Matthew Flanaganhttp://wadofstuff.blogspot.com