Customizable QuerySets
flo...@gmail.com
about aggregate support, the idea struck me that the same types of
discussions were had before with schema evolution, and nobody could
really agree on much. Now that management commands are possible,
schema evolution is able to be implemented in a 3rd party library, of
which there are many. This is the best possible situation, because
the best implementation will get the most support and this can all
happen without jeopardizing the stability or backwards compatibility
of trunk.
So the question is: What would it take to allow for aggregate-enabled
QuerySets to be developed as 3rd party libraries, where the same
benefits would apply?
In my opinion the answer is this:
1. A BaseQuerySet that all internal Django calls could rely on staying
the same.
2. A setting (DEFAULT_QUERYSET_CLASS for example) which, by default,
points to BaseQuerySet, but could be changed to point to any sucblass
of BaseQuerySet.
That's it! Now projects could subclass BaseQuerySet, add any
functionality that they deemed valuable, and Django could use that
whenever any method on a QuerySet is called.
I've implemented a proof of concept in ticket #6875 [1], and tested it
by writing a simple OutputterQuerySet which outputs to stdout every
time you make a call to that QuerySet. It worked for me and I believe
this would be a good addition to Django which might solve a lot of the
debates that we're having right now.
What does everyone think about this?
[1] http://code.djangoproject.com/ticket/6875
Yuri Baburov
applications codes are still compatible with each other, but different
querysets features as 3rd-party libraries could make project code
incompatible, and that's baaad.
--
Best regards, Yuri V. Baburov, ICQ# 99934676, Skype: yuri.baburov,
MSN: bu...@live.com
Empty
> 1. A BaseQuerySet that all internal Django calls could rely on staying
> the same.
> 2. A setting (DEFAULT_QUERYSET_CLASS for example) which, by default,
> points to BaseQuerySet, but could be changed to point to any sucblass
> of BaseQuerySet.
>
> That's it! Now projects could subclass BaseQuerySet, add any
> functionality that they deemed valuable, and Django could use that
> whenever any method on a QuerySet is called.
FWIW, I like the idea. The more "hooks" added into the framework is a
good thing. I also like how your test implementation does not disrupt
current structure that is already in place.
Michael Trier
blog.michaeltrier.com
Nicolas E. Lara G.
think that subclasing a core part of the framework is quite messy and
will require for the developers to know/understand the inner workings
of the framework (and there goes abstraction). With this approach you
would end up, if I understood correctely, writting the sql for the
problem you need to solve wrapped up in a lot of logic to keep it
consistent with the QuerySet workings. This seems more complicated
that just writing your sql in a function for the models or a a manager
and using it anywhere.
Even if the user is not needed to write sql but do it in plain python
still I believe that, if it is going to be done, extending the
QuerySet system should be done in a higher level of abstraction
without having to tangle with the framework inner workings.
James Bennett
> schema-evolution is separable from applications -- it's utility, and
> applications codes are still compatible with each other, but different
> querysets features as 3rd-party libraries could make project code
> incompatible, and that's baaad.
Indeed. If two applications each rely on different QuerySet classes,
something more fine-grained than a global setting will be needed for
them to peaceably coexist; I'd much prefer some standardized way of
extending the functionality, something we already have to an extent
via the ability to override get_query_set() in a custom manager.
--
"Bureaucrat Conrad, you are technically correct -- the best kind of correct."
flo...@gmail.com
> something more fine-grained than a global setting will be needed for
> them to peaceably coexist; I'd much prefer some standardized way of
> extending the functionality, something we already have to an extent
> via the ability to override get_query_set() in a custom manager.
achieve a similar goal, but there's currently no clean way (you could
monkeypatch) of overriding the get_query_set() method on, for example,
the django.contrib.auth.models.User object.
I think that a combination of a customizable default global QuerySet
classes, in conjunction with locally overridable get_query_set()
method on custom managers gets quite close to completing this circle.
> schema-evolution is separable from applications -- it's utility, and
> applications codes are still compatible with each other, but different
> querysets features as 3rd-party libraries could make project code
> incompatible, and that's baaad.
compatible. They could:
1. Simply rely on the functionality provided by BaseQuerySet, which
is more than adequate for the majority of tasks.
2. Ensure that their required QuerySet class is used by overriding
get_query_set() on their own models and doing some sort of an
assertion with isinstance() or issubclass().
The only hole in this strategy is if a 3rd-party library would like to
use a built-in model (User is again a good example) and at the same
time use a non-standard QuerySet. I'm still not sure how to address
this issue, but perhaps a model method is in order to get a custom
manager. Let me provide an example of what I'm thinking:
>>> from django.contrib.auth.models import User
>>> from django.db.models import Manager
>>> import MyCustomQuerySet # For whatever reason, this implements group_by
>>> class MyCustomManager(Manager):
>>> def get_query_set(self):
>>> return MyCustomQuerySet(self.model)
>>> def group_by(self, *args, **kwargs):
>>> return self.get_query_set().group_by(*args, **kwargs)
>>> custom_manager = User.get_custom_manager(MyCustomManager)
>>> print custom_manager.get(username = 'amy') # It still has all of the regular manager methods.
>>> <User: amy>
>>> print custom_manager.group_by(username__startswith=['a', 'b', 'c', 'd'])
>>> [[<User: amy>, <User: anne>, <User: ashley>], [<User: bob>, <User: bill>], ... ]
(Note that I have not inspected the implementation to see if this
method would be easy to create, I'm just thinking about the concept at
this point.)
This would solve that hole that I mentioned earlier, and now third
parties could rely on whichever QuerySet subclass that they wanted to
because they could create their own custom managers and use them with
whichever models they wanted to.
alex....@gmail.com
finding the best ways to solve the problems in django itself, one of
the main differences between django and other python web frameworks is
that Django offers it's own option at every level of the stack, and
people almost exclusively use these options, this is not to say that
this change would in and of itself, overnight, change django into a
mixed soup of options, but it is on that path. We should, and
inevitably will, be able to find a good solution to the aggregates
debate. I am not opposed to providing hooks in django, however as
James said, providing a custom manager and defining get_query_set can
do this on a more granular basis, and I think a good solution to
providing this on the included models(such as User or others in the
contrib apps), is to have a setting USER_MODEL (or whichever, although
USER is by far the most common case), and have it take a subclass of
User(once qs-rf merges), or any other model that offers the necessary
attributes and methods, and this will not only make it easy to provide
a custom manager(and queryset if necessary), but will be more flexible
in every respect.
Alex
Russell Keith-Magee
>
> On Fri, Mar 28, 2008 at 5:54 AM, Yuri Baburov <bur...@gmail.com> wrote:
> > schema-evolution is separable from applications -- it's utility, and
> > applications codes are still compatible with each other, but different
> > querysets features as 3rd-party libraries could make project code
> > incompatible, and that's baaad.
>
> Indeed. If two applications each rely on different QuerySet classes,
> something more fine-grained than a global setting will be needed for
> them to peaceably coexist; I'd much prefer some standardized way of
> extending the functionality, something we already have to an extent
> via the ability to override get_query_set() in a custom manager.
Agreed. In fact, this exact suggestion was made during the recent
discussions about adding aggregates. I'd much rather see per-manager
customization of Query Sets, rather than a global setting.
Yours,
Russ Magee %-)
flo...@gmail.com
> discussions about adding aggregates. I'd much rather see per-manager
> customization of Query Sets, rather than a global setting.
get_custom_manager(MyManager) function on Model objects? (See the end
of my previous post to this list for an example of what I mean).
The more and more I think about it, it seems like this could be the
better solution, not my earlier proposed setting. It also jives well
with the notion that you and James have for keeping it granular.
Malcolm Tredinnick
On Sun, 2008-03-30 at 13:31 -0700, flo...@gmail.com wrote:
> > Agreed. In fact, this exact suggestion was made during the recent
> > discussions about adding aggregates. I'd much rather see per-manager
> > customization of Query Sets, rather than a global setting.
>
> What do you think about my latest proposal of a
> get_custom_manager(MyManager) function on Model objects? (See the end
> of my previous post to this list for an example of what I mean).
I don't like this, since it's not actually usable (in the sense that it
doesn't add anything) in this form. Your earlier example had you
attaching a get_custom_manager() method to, e.g., the User model, but
there's no way to say what that method should return.
If you want to be able to specify a different default manager for some
particular use of an existing model, you're sort of after a third type
of model subclassing that I've thought about but haven't implemented:
subclassing an existing model and explicitly telling Django that this is
only Python-level (and not database-/ORM-level) inheritance. So all the
database interactions are part of the parent class(es) and the child
class simply adds extra functional pieces (such as a new default
manager). Nothing existing so far rules out adding this, so it's not
something that has to be resolved now, which is why I haven't wasted any
brain cells on it so far. This might be fairly easy to add once we work
out how to spell it, since it's only saying "create absolutely no fields
for this model, not even links back to the parent model and definitely
don't create a database table."
Then something that wanted to use a different manager (which is really
what you are meaning when you say "different queryset") with a model
would subclass it in this fashion and add their own manager to the
subclass. Existing code doesn't care about the new behaviour of your
custom manager and can safely rely on the existing behaviour it is used
to (which is a good thing: you don't sabotage existing code. Existing
code uses the original model class). Your new code can use the subclass
to be explicit that it wants the new queryset and that's fine, since
it's new code that you are writing to take advantage this, so it's
hardly a burden to have to write UserSubclass instead of User.
Malcolm
--
Atheism is a non-prophet organization.
http://www.pointy-stick.com/blog/
flo...@gmail.com
> particular use of an existing model, you're sort of after a third type
> of model subclassing that I've thought about but haven't implemented:
> subclassing an existing model and explicitly telling Django that this is
> only Python-level (and not database-/ORM-level) inheritance.
seems like the cleanest way of getting custom functionality onto
models while leaving existing code alone.
Marty Alchin
<mal...@pointy-stick.com> wrote:
> If you want to be able to specify a different default manager for some
> particular use of an existing model, you're sort of after a third type
> of model subclassing that I've thought about but haven't implemented:
> subclassing an existing model and explicitly telling Django that this is
> only Python-level (and not database-/ORM-level) inheritance. So all the
> database interactions are part of the parent class(es) and the child
> class simply adds extra functional pieces (such as a new default
> manager). Nothing existing so far rules out adding this, so it's not
> something that has to be resolved now, which is why I haven't wasted any
> brain cells on it so far. This might be fairly easy to add once we work
> out how to spell it, since it's only saying "create absolutely no fields
> for this model, not even links back to the parent model and definitely
> don't create a database table."
I've invested a few brain cells on this already, though not enough for
a complete solution. It'd be off-topic for this discussion, but
if/when you get around to it, feel free to hit me up if you're looking
for another opinion on how it could be done.
-Gul