Re: use as a couchdb python backend?

[ricopan]

I can query couchdb with nested and composite key values if I do this:

import couchdb
import jsonpickle as jp
jp.set_preferred_backend('simplejson')
jp.set_encoder_options('simplejson',sort_keys=True)
couchdb.json.use(encode=jp.encode,decode=jp.decode)

then to query with a composite, nested key supplied by my_key_dict:

key_for_couch = [my_key_dict[k] for k in sorted(my_key_dict)] #
maybeuse an ordered dict
results = db.view('my_designs/my_view',key=key_for_couch)

With this setup, couchdb handles all the encoding/decoding, but
sorting of key values needed.

This was a product of trial and error more than any real understanding
of jsonpickle and couchdb, and though its working for my test cases, I
have yet to use in production.
I find it surprising I needed to set simplejson sort_keys True, as
thought it would default to this, but otherwise with nested keys
containing non-json objects (eg the adict key below), the query fails.

Maybe those in the know could shed some light on this....

tested with:

my_key_dict= {
'a': 'a',
'alist': [1, 2, 3],
'aset': set(['rice', 'beans']),
'adict': {'aset': set(['rice', 'beans']), 'alist': [1, 2, 3], 'one':
1},
'one': 1,
'simple_dict': {'foo': 'bar'}

[David Aguilar]

On Sat, Oct 02, 2010 at 06:42:48AM -0700, ricopan wrote:
>
> I can query couchdb with nested and composite key values if I do this:
>
> import couchdb
> import jsonpickle as jp
> jp.set_preferred_backend('simplejson')
> jp.set_encoder_options('simplejson',sort_keys=True)
> couchdb.json.use(encode=jp.encode,decode=jp.decode)
>
> then to query with a composite, nested key supplied by my_key_dict:
>
> key_for_couch = [my_key_dict[k] for k in sorted(my_key_dict)] #
> maybeuse an ordered dict
> results = db.view('my_designs/my_view',key=key_for_couch)
>
> With this setup, couchdb handles all the encoding/decoding, but
> sorting of key values needed.
>
> This was a product of trial and error more than any real understanding
> of jsonpickle and couchdb, and though its working for my test cases, I
> have yet to use in production.
> I find it surprising I needed to set simplejson sort_keys True, as
> thought it would default to this, but otherwise with nested keys
> containing non-json objects (eg the adict key below), the query fails.

You bring up some good points.

From the POV of "give me something that works"
it's probably in our best interest to make these
things the default.

Nested object hierarchies necessitate sorted keys so that
decode() has a chance to reconstruct the object hieararchy
correctly. I think it would be good idea if the
global backend-specific options comes pre-seeded with
sort_keys=True.

We have already made decode() assume sorted keys
(and that's the only way we can really do it)

$ git show 8a0a07a713ed11a924d6e34cdb6b2fc9b6238ec4
commit 8a0a07a713ed11a924d6e34cdb6b2fc9b6238ec4
Author: David K. Hess <nos...@nospam.com>
Date: Tue Feb 9 16:05:49 2010 -0600

If an interpreter of a jsonpickle stream has a different dictionary key
hashing algorithm from the one that produced it, then the problem of a
py/ref before a py/object can occur. If the hashing algorithms are exactly
the same (i.e. dict.keys() produces the same ordering of keys in both the
encoder and decoder domains), then the problem will not occur.

This case occurs when using (for example) Objective-C and NSDictionary
to interpret a jsonpickle stream.

The patch sorts dictionary keys when both encoding and decoding so that
the process is independent of the dict key hashing algorithm.

--

David