jsonify and sqlalchemy support
Raphael Slinckx
With the most recent TG release (b2) support for jsonifying sqlalchemy
maped objects has been added to turbojson.
Previously one had to either add a __json__() method to mapped
classes, or use explicit jsonify.when('isintance(obj, Foo)') to
provide jsonification of a mapped sqlalchemy.
Now, with the upgrade the rule in turbojson is becoming ambiguous with
the two previously mentioned rules, meaning i just can't customize
sqlalchemy mapped object's jsonification, because either i add a
__json__ and dispatch doesn't know whether to use __json__ or the
builtin jsonify_saobject (AmbiguousMethod), or I add an explicit rule,
but the same error appears.
How should I go about customizing the jsonification of my sqlalchemy-
mapped objects without patching turbojson ?
Is there some kind of prioritization of dispatch method in case of
ambiguous method ?
Thanks a lot !
Raphael
Jonathan LaCour
> Now, with the upgrade the rule in turbojson is becoming ambiguous with
> the two previously mentioned rules, meaning i just can't customize
> sqlalchemy mapped object's jsonification, because either i add a
> __json__ and dispatch doesn't know whether to use __json__ or the
> builtin jsonify_saobject (AmbiguousMethod), or I add an explicit rule,
> but the same error appears.
I have been having the same problem. I solved it by doing this before I
defined my own custom jsonify rules:
jsonify.clear()
This will clear out all of the existing jsonification functions, and
yours will take precedence. However, this isn't ideal. What we really
need is a way to resolve the AmbiguousMethod issue by somehow making our
constraints more specific. I don't know how to do that, however.
--
Jonathan LaCour
http://cleverdevil.org
Paul Johnston
>How should I go about customizing the jsonification of my sqlalchemy-
>mapped objects without patching turbojson ?
>
>
A good question. I'm not sure offhand, certainly didn't anticipate this
when I made that change to TurboJson. I have forwarded your message to
Philip J Eby; lets see what he says.
Paul
Phillip J. Eby
> Is there some kind of prioritization of dispatch method in case of
> ambiguous method ?
Add an "and" condition that will be true to the higher-priority
method. E.g. "isinstance(x, Foo) and [some condition involving x that
will be true]".
Alberto Valverde
Take a look at toscawidgets.genericfunctions (has no dependencies with
TW at all). It has a custom combiner that adds a "priority" argument to
when() (and before(), around() and after()) which greatly simplifies
writing rules since you don't need to make one of them more specific to
resolve ambiguities. For rules having the same priority it behaves in
the same way as the standard combiner.
The tests, at tests/test_genericfunctions.py, might shed a light on how
it's used. It's been serving me well for some time :)
Alberto
Raphael Slinckx
> > Is there some kind of prioritization of dispatch method in case of
> > ambiguous method ?
>
> Add an "and" condition that will be true to the higher-priority
> method. E.g. "isinstance(x, Foo) and [some condition involving x that
> will be true]".
That doesn't resolve the ambiguous method problem. Here are the three
rules we are talking about:
jsonify_saobject:
"hasattr(obj, 'c') and
isinstance(obj.c,sqlalchemy.util.OrderedProperties)"
jsonify_explicit:
'hasattr(obj, "__json__")'
Before the patch i was using the explicit method. After the patch that
added saobject, the two conditions became true (since my object is a
saobject, and also has a __json__ method) raising an ambiguous method
error.
Now as you suggested (if i understood correctly), I tried the
following: I removed all the __json__ methods, and i added the
following condition to json.py:
"isinstance(obj, projectname.model.User) and hasattr(obj, 'c') and
isinstance(obj.c,sqlalchemy.util.OrderedProperties)"
So i'm just 'extending' the check for saobject with another condition.
But it seems this function is never called and the other one is used
instead. But even then, I don't think having to manually 'extend' like
this the jsonify checks is a Good Thing..
A temporary workaround would be to make saobject check for "not
hasattr('__json__')" so that in case of ambiguity the explicit
jsonification is used..
Raf
Paul Johnston
>That doesn't resolve the ambiguous method problem. Here are the three
>rules we are talking about:
>
>
I see... sounds like we want to use Alberto's genericfunctions package
in TurboJson, and make the rule that checks for __json__ have a higher
priority.
That's probably the neat and proper solution; an alternative quick fix
is to add NOT __json__ checks to all the other branches, as you suggest.
Let me know what you think,
Paul
Raphael Slinckx
On Nov 5, 8:39 pm, Paul Johnston <p...@pajhome.org.uk> wrote:
> I see... sounds like we want to use Alberto's genericfunctions package
> in TurboJson, and make the rule that checks for __json__ have a higher
> priority.
>
> That's probably the neat and proper solution; an alternative quick fix
> is to add NOT __json__ checks to all the other branches, as you suggest.
be to add (NOT __json__) checks. This also removes yet another
external dependency.
If needed I could take time to prepare such a patch, unless you want
to do it yourself. Is there a SVN/bugtracker URL or something where i
should send the patch ?
Thanks !
Raf
Florent Aide
> If needed I could take time to prepare such a patch, unless you want
> to do it yourself. Is there a SVN/bugtracker URL or something where i
> should send the patch ?
patches should now be against the 1.1 branch and should be proposed in
our trac system: http://trac.turbogears.org
Discussions about the patches and/or issues should preferably occur
here since it is more visible. A reference to the gg thread could be
added later on on the ticket if needed.
Florent.
Raphael Slinckx
I created a ticket and attached a patch here:
http://trac.turbogears.org/ticket/1619
Against trunk turbojson. It adds two clauses to prevent ambiguous
methods
Hope this can be released soonish..
Raf
Diez B. Roggisch
I can make a 1.1.3-release including that. But only if you supply a test-case
for that.
Diez
Raphael Slinckx
> > Hope this can be released soonish..
>
> I can make a 1.1.3-release including that. But only if you supply a test-case
> for that.
requires SQLAlchemy >= 0.4 which isn't reflected in setup.py.
When using it with sqlalchemy 0.3.11, it fails in the
"turbojson.tests.test_sqlalchemy.test_salist" test because older
version didn't return a real list, but a
sqlalchemy.blaba.InstrumentedList thus preventing default list
jsonification.
Coming up with the testcase for __json__ on the ticket soon.
Raf
Diez B. Roggisch
> Hi
>
> > > Hope this can be released soonish..
> >
> > I can make a 1.1.3-release including that. But only if you supply a
> > test-case for that.
>
> I'm working on it. Just a note: it seems the turbojson testcase
> requires SQLAlchemy >= 0.4 which isn't reflected in setup.py.
> When using it with sqlalchemy 0.3.11, it fails in the
> "turbojson.tests.test_sqlalchemy.test_salist" test because older
> version didn't return a real list, but a
> sqlalchemy.blaba.InstrumentedList thus preventing default list
> jsonification.
I don't use SA and just install it with TG because of dependencies. If
somebody who _does_ says that we should use the SA-version in TG, I'm happy
adding that requirement. Maybe you can do so and create a patch for
everything - the fix + testcase + setup.py changes.
Regards,
Diez
Roger Demetrescu
>
> Hi
>
> > > Hope this can be released soonish..
> >
> > I can make a 1.1.3-release including that. But only if you supply a test-case
> > for that.
>
> I'm working on it. Just a note: it seems the turbojson testcase
> requires SQLAlchemy >= 0.4 which isn't reflected in setup.py.
> When using it with sqlalchemy 0.3.11, it fails in the
> "turbojson.tests.test_sqlalchemy.test_salist" test because older
> version didn't return a real list, but a
> sqlalchemy.blaba.InstrumentedList thus preventing default list
> jsonification.
I confirm Raphael's observations:
Running the tests with SA 0.3.10 gives me:
======================================================================
ERROR: turbojson.tests.test_sqlalchemy.test_salist
----------------------------------------------------------------------
Traceback (most recent call last):
File "c:\python24\lib\site-packages\nose-0.10.0-py2.4.egg\nose\case.py",
line 202, in runTest
self.test(*self.arg)
File "C:\TurboJson\turbojson\tests\test_sqlalchemy.py", line 48, in
test_salist
encoded = jsonify.encode(t.test2s)
File "C:\TurboJson\turbojson\jsonify.py", line 115, in encode
return _instance.encode(obj)
File "C:\Python24\lib\site-packages\simplejson-1.3-py2.4.egg\simplejson\encoder.py",
line 270, in encode
File "C:\Python24\lib\site-packages\simplejson-1.3-py2.4.egg\simplejson\encoder.py",
line 231, in _iterencode
File "C:\Python24\lib\site-packages\simplejson-1.3-py2.4.egg\simplejson\encoder.py",
line 237, in _iterencode_default
File "C:\TurboJson\turbojson\jsonify.py", line 107, in default
return jsonify(obj)
File "<string>", line 5, in jsonify
File "_d_speedups.pyx", line 403, in _d_speedups.BaseDispatcher.__getitem__
File "C:\Python24\lib\site-packages\ruledispatch-0.5a0.dev_r2306-py2.4-win32.egg\dispatch\interfaces.py",
line 15, in
__call__
NoApplicableMethods: (([<turbojson.tests.test_sqlalchemy.Test2 object
at 0x012A4850>, <turbojson.tests.test_sqlalchemy.T
est2 object at 0x012A48B0>],), {})
----------------------------------------------------------------------
Ran 6 tests in 0.609s
FAILED (errors=1)
==================================
Running the tests with SA trunk, it gives me no error...
Raphael Slinckx
for the __json__ issue.
http://trac.turbogears.org/ticket/1619
As for the sa 0.4 issue, i'm not sure if sa 0.4 is a strict
requirement for turbogears (i don't think it is, actually).
I guess we shouldn't expect the test_salist to work for sqlaclhemy <
0.4
I don't know how to reflect that in the tests.
Raf
Diez B. Roggisch
The patch works for me, I committed it.
HOwever I'd need to have a consensus on how to deal with the SA-issue. As I
said I don't know nothing about SA, so maybe one of the guys using it &
having both versions running can either patch the problem away or at least
create a warning if the expected SA-version doesn't work and not create a
failure in the tests.
Diez
Paul Johnston
>I created a ticket and attached a patch here:
>http://trac.turbogears.org/ticket/1619
>
>
Good stuff! I can't actually see your patches though, perhaps
something's wrong with Trac.
As for the failing unit test, ideally all unit tests should pass with SA
0.3.10+. What you've described doesn't sound like such a big deal
though; if it's hard to fix perhaps just leave it.
Regards,
Paul
Raphael Slinckx
I just wanted to know when the next turbojson would be released
containing the sqlalchemy fix ?
Thanks a lot!
Raf