Admin: Allow FilterSpecs to return Q-likes

[steve yeago]

https://code.djangoproject.com/ticket/27303#trac-add-comment

Hi all,

I am looking to drum up more feedback about this issue as I think an important design decision needs to be made regarding filtering in the admin. At the heart of this issue is the syntax simplicity of repeating filter() clauses on a queryset breaks down pretty quickly when you are talking about traversing relations. The current FilterSpec API was a huge improvement, but shuttling filter() clauses between objects leaves a lot to be desired in terms of joining and leaves what I see as a critical inconsistency when it comes to adding several filters across relations (the way it is now, each filter gets its own JOIN and the result is a rather ambiguous OR-like clause).

There are also related issues at play, such as providing hooks for developers to apply filters that do more than just AND. While my ticket doesn't hope to address that comprehensively, I do think that some interesting options come into play when we stop authoring FilterSpecs to chain filters() and instead return Q-like objects which are then applied by the ModelAdmin or AdminSite.

-Steve

[Andy Baker]

In a few cases I've had to do filtering in Python because it wasn't possible purely at the db level. (mainly in cases where I'm expected small result sets obviously). I'd like to see this remain possible with any future changes.

[Andy Baker]

Actually - my recollection was faulty. As the queryset method always has to return a queryset (dur) I am not sure that I'm actually doing anything that couldn't be expressed as a Q object. I'm just doing some funky stuff to get my queryset in shape.

So I suppose my question is this - are there any operations that return a queryset that couldn't be captured in a Q object?

[charettes]

> So I suppose my question is this - are there any operations that return a queryset that couldn't be captured in a Q object?

Annotations are a good example of operations that cannot be expressed in a Q object.

On the ticket I suggested relying on the "sticky filter" feature of the ORM to implement a combinable filter spec.

Sticky filters allow chained filter() calls to reuse aliases, making filter(lookup=1).filter(lookup=2) equivalent to filter(lookup=1, lookup=2) while the queryset is marked a "sticky"

They are only used internally by the ORM but I remember a ticket where Anssii mentioned we wight want to expose the feature through a more adequate API.

e.g.

with queryset.reuse_aliases() as qs:
    qs = qs.filter(lookup=1).filter(lookup=2)

assert qs == queryset.filter(lookup=1, lookup=2)

This would make it easy to implement a "combinable" filter spec and allow users to solve the issues that have surfaced recently about how the admin does search and filtering through multiple joins. For example, one could override get_search_results() to reuse aliases easily by relying on this method and implementing the "combinable" filter spec class would only be a matter of creating a composite filter spec that reuse aliases in queryset() before delegating to each of its nested filters.

Simon