QuerySet's __repr__

[Bjørn Stabell]

Hi,

I would like to be able to serialize QuerySets for use as "canned"
queries etc, and looking at QuerySet it's currently delegating __repr__
to its data. I was wondering what the feeling would be to change this
to actually return a Python expression that would evaluate to the value
of the QuerySet, for use in serializing etc. E.g.,

>>> a = MyModel.objects.filter(status=3).distinct()[:3]
>>> a_str = repr(a)
>>> a_str
MyModel.objects.filter(status=3).distinct()[:3]
>>> b = eval(a_str)
>>> b
MyModel.objects.filter(status=3).distinct()[:3]

The object b would now be a clone of a. I haven't found a way to
replicate it using filter() and exclude(), but can do it using
complex_filter(), e.g.,

>>> b
QuerySet(model=MyModel).complex_filter( QAnd( Q(), Q(status=3)
)).distinct()

For some reason there's always an empty Q() as the first element of
_filter.

I've attached a tentative patch for django.db.models.query.py for this.

Index: query.py
===================================================================
--- query.py (revision 5323)
+++ query.py (working copy)
@@ -94,7 +94,19 @@
########################

def __repr__(self):
- return repr(self._get_data())
+ # goal is that self == eval(repr(self))
+ model = self.model and "model=%s" % self.model.__name__ or
""
+ filters = ".complex_filter( %s )" % repr(self._filters)
+ order_by = self._order_by and ".order_by(%s)" %
repr(self._order_by) or ""
+ distinct = self._distinct and ".distinct()" or ""
+ if self._offset or self._limit:
+ start = self._offset or 0
+ end = self._limit and self._limit+start or 0
+ slice = "[%s:%s]" % (start or "", end or "")
+ else:
+ slice = ""
+ # TODO: support _select_related, _select, _where, _params,
_tables
+ return
"QuerySet(%(model)s)%(filters)s%(order_by)s%(distinct)s%(slice)s" %
locals()

def __len__(self):
return len(self._get_data())
@@ -592,6 +604,9 @@
return QAnd(*(self.args+(other,)))
else:
raise TypeError, other
+
+ def __repr__(self):
+ return 'QAnd(%s)' % (", ".join([ repr(arg) for arg in
self.args ]),)

class QOr(QOperator):
"Encapsulates a combined query that uses 'OR'."
@@ -607,6 +622,9 @@
else:
raise TypeError, other

+ def __repr__(self):
+ return 'QOr(%s)' % (", ".join([ repr(arg) for arg in self.args
]),)
+
class Q(object):
"Encapsulates queries as objects that can be combined logically."
def __init__(self, **kwargs):
@@ -621,6 +639,9 @@
def get_sql(self, opts):
return parse_lookup(self.kwargs.items(), opts)

+ def __repr__(self):
+ return 'Q(%s)' % (", ".join([ "%s=%s" % (arg,repr(val)) for
arg,val in self.kwargs.items() ]),)
+
class QNot(Q):
"Encapsulates NOT (...) queries as objects"
def __init__(self, q):
@@ -632,6 +653,9 @@
where2 = ['(NOT (%s))' % " AND ".join(where)]
return joins, where2, params

+ def __repr__(self):
+ return 'QNot(%s)' % repr(self.q)
+
def get_where_clause(lookup_type, table_prefix, field_name, value):
if table_prefix.endswith('.'):
table_prefix = backend.quote_name(table_prefix[:-1])+'.'

[Malcolm Tredinnick]

Hi Bjørn,

This didn't seem to get a response, but it deserves some
acknowledgement...

On Fri, 2006-12-29 at 04:17 +0000, Bjørn Stabell wrote:
> Hi,
>
> I would like to be able to serialize QuerySets for use as "canned"
> queries etc, and looking at QuerySet it's currently delegating __repr__
> to its data. I was wondering what the feeling would be to change this
> to actually return a Python expression that would evaluate to the value
> of the QuerySet, for use in serializing etc. E.g.,
>
> >>> a = MyModel.objects.filter(status=3).distinct()[:3]
> >>> a_str = repr(a)
> >>> a_str
> MyModel.objects.filter(status=3).distinct()[:3]
> >>> b = eval(a_str)
> >>> b
> MyModel.objects.filter(status=3).distinct()[:3]
>
> The object b would now be a clone of a. I haven't found a way to
> replicate it using filter() and exclude(), but can do it using
> complex_filter(), e.g.,
>
> >>> b
> QuerySet(model=MyModel).complex_filter( QAnd( Q(), Q(status=3)
> )).distinct()

This seems like a reasonable idea, although I'm not sure if __repr__ is
the right place for it or not (and that's something that doesn't
interfere with the implementation anyway, so I'm not going to worry
much).

I would suggest/prefer holding off trying to get this perfect until the
QuerySet rewrite is done (something I am doing). As part of that
rewrite, QuerySets will contain a slightly more abstract version of the
SQL being constructed -- the SQL string itself will be formed as part of
the __str__ method. This removes a bunch of the hackery needed to join
queries together and provides some manipulation possibilities we don't
have now. The upshot of this for your work is that it will probably be
easier to extract a serialised form of the query itself. I'm not sure
how much easier/harder it will be to print it out as a Python statement
like the above in all cases, but some form of serialisation that will be
easy to dynamically reconstruct should be possible.

That being said, I see no immediate problems with the patch (realising
it's a prototype, so docs and tests are missing). I just wanted to
possibly avoid a round of debugging for this if it's only going to have
a lifespan of a month or two.

Regards,
Malcolm

[Bjørn Stabell]

Malcolm Tredinnick wrote:
[...]

> This seems like a reasonable idea, although I'm not sure if __repr__ is
> the right place for it or not (and that's something that doesn't
> interfere with the implementation anyway, so I'm not going to worry
> much).

In principle __repr__ should be the right place (according to how
Python defines __repr__), but I understand there could be compatibility
issues.

> I would suggest/prefer holding off trying to get this perfect until the
> QuerySet rewrite is done (something I am doing). As part of that
> rewrite, QuerySets will contain a slightly more abstract version of the
> SQL being constructed -- the SQL string itself will be formed as part of
> the __str__ method. This removes a bunch of the hackery needed to join
> queries together and provides some manipulation possibilities we don't
> have now. The upshot of this for your work is that it will probably be
> easier to extract a serialised form of the query itself. I'm not sure
> how much easier/harder it will be to print it out as a Python statement
> like the above in all cases, but some form of serialisation that will be
> easy to dynamically reconstruct should be possible.

Waiting will probably be better as we (at least I :) won't have to
create backwards compatible constructors to QuerySets serialized the
old way, which could be non-trivial.