Feature requests for queryset

[Laurent Lyaudet]

Hello,

Below are 2 feature requests :
- the first would just be to add a constant object for ignoring some filters,
to avoid lists of "if ... is not None: q = q.filter(...)" like I must
do right now,
q = q.filter(id_in=[1,2,3]) filters something
q = q.filter(id_in=IGNORE_FILTER) does not filter
- the second would be to have "in-place" variants of filter and exclude, like :
q.filter_in_place(id_lt=5)
instead of
q = q.filter(id_lt=5)
Maybe other people asked for one or both of these features before,
but I did check the search here :
https://code.djangoproject.com/search
and also the archive here :
https://groups.google.com/g/django-developers/search?q=ignore%20filter
etc.
and found nothing related.

I proposed this morning the following class with better example at my
team at work,
but the decision was took that we stay on bare Django.
Nevertheless, some of my teammates agreed that the IGNORE_FILTER
feature was nice.

class EnhancedQuerySet:
"""
Extends the functionalities of Django QuerySet
Example:
- before:
def some_function(
fk1_ids: Optional[List[int]] = None,
fk2_ids: Optional[List[int]] = None,
fk3_ids: Optional[List[int]] = None,
fk4_ids: Optional[List[int]] = None,
pk_ids: Optional[List[int]] = None,
...
):
some_instances_queryset = SomeModelClass.objects.all()
if fk1_ids is not None:
some_instances_queryset =
some_instances_queryset.filter(fk1_id__in=fk1_ids)
if fk2_ids is not None:
some_instances_queryset =
some_instances_queryset.filter(fk2_id__in=fk2_ids)
if fk3_ids is not None:
some_instances_queryset =
some_instances_queryset.filter(fk3_id__in=fk3_ids)
if fk4_ids is not None:
some_instances_queryset =
some_instances_queryset.filter(fk4_id__in=fk4_ids)
if pk_ids is not None:
some_instances_queryset =
some_instances_queryset.filter(id__in=pk_ids)
for some_instance in some_instances_queryset:
...
- after:
def some_function(
fk1_ids: Optional[List[int]] = EnhancedQuerySet.IGNORE_FILTER,
fk2_ids: Optional[List[int]] = EnhancedQuerySet.IGNORE_FILTER,
fk3_ids: Optional[List[int]] = EnhancedQuerySet.IGNORE_FILTER,
fk4_ids: Optional[List[int]] = EnhancedQuerySet.IGNORE_FILTER,
pk_ids: Optional[List[int]] = EnhancedQuerySet.IGNORE_FILTER,
...
):
some_instances_queryset = EnhancedQuerySet(SomeModelClass.objects.all())
some_instances_queryset.filter(
fk1_id__in=fk1_ids,
fk2_id__in=fk2_ids,
fk3_id__in=fk3_ids,
fk4_id__in=fk4_ids,
id__in=pk_ids,
)
for some_instance in some_instances_queryset:
...
You may alias EnhancedQuerySet for shorter code :
from ... import EnhancedQuerySet as EQS
def some_function(
fk1_ids: Optional[List[int]] = EQS.IGNORE_FILTER,
fk2_ids: Optional[List[int]] = EQS.IGNORE_FILTER,
fk3_ids: Optional[List[int]] = EQS.IGNORE_FILTER,
fk4_ids: Optional[List[int]] = EQS.IGNORE_FILTER,
pk_ids: Optional[List[int]] = EQS.IGNORE_FILTER,
...
):
some_instances_queryset = EQS(SomeModelClass.objects.all())
some_instances_queryset.filter(
fk1_id__in=fk1_ids,
fk2_id__in=fk2_ids,
fk3_id__in=fk3_ids,
fk4_id__in=fk4_ids,
id__in=pk_ids,
)
for some_instance in some_instances_queryset:
...
"""

IGNORE_FILTER = object() # a sentinel for ignoring some filter to
distinguish from None

def __init__(self, queryset):
self.queryset = queryset

def _get_true_kwargs(self, kwargs: dict):
true_kw_args = {}
for key, value in kwargs.items():
if value is EnhancedQuerySet.IGNORE_FILTER:
continue
true_kw_args[key] = value
return true_kw_args

def filter(self, *args, **kwargs):
"""
Enhanced filter function with 2 features :
- ignore some filters given with some default value
- never forget the affectation after filtering, i.e. avoid this mistake:
queryset.filter()
instead of queryset = queryset.filter()
"""
true_kw_args = self._get_true_kwargs(kwargs)
self.queryset = self.queryset.filter(*args, **true_kw_args)

def exclude(self, *args, **kwargs):
"""
Similar to filter method but with exclude
"""
true_kw_args = self._get_true_kwargs(kwargs)
self.queryset = self.queryset.exclude(*args, **true_kw_args)


Best regards,
Laurent Lyaudet

[Jason Johns]

filter takes in a decomposed dict of keys and values, ie 

filter_dict = {

    "id_in": SomeIterable

}

qs = SomeModel.objects.filter(**filter_dict)

from your example, IGNORE_FILTER does not return an iterable, so how is that to work with a SQL query?

Seems to me that ths might be an x-y problem

[Laurent Lyaudet]

Hello Jason,

Thanks for your answer.

First I apologize because my webmail removed all indentation from
class EnhancedQuerySet:

Thus it may not be easily understood without taking the time to reindent it.

I created it here : https://djangosnippets.org/snippets/10867/

so that everyone can easily read the code.

Nevertheless, it contains the reasons I suggest this and clear examples of the problems that explain my point.

Your solution with a dict seems to me inferior to my solution.

It does not avoid the boring sequence of :

if filter_value is not None:

      # add the value in the filter_dict

The repetitions of "if" brings nothing to readability, it is plain boilerplate code that a framework helps to remove normally.

So the real problem is to remove all this boilerplate code (and avoid silly bugs for the other feature request),

and it is explained in my previous email, if you are interested in x-y problem.

> from your example, IGNORE_FILTER does not return an iterable, so how is that to work with a SQL query?

Reading my first email and my code, it is clear that this constant object is detected by Django before creating the SQL query,

since its goal is to ignore a particular filter, thus it has no other requirement than to be identifiable by Django code and distinct from all other valid values of filters.

Making it a constant object created by Django satisfies this requirements.

Sorry, but your remark shows that my feature request was not understood.

I hope that now that my code is readable on djangosnippets, you will understand how it works.

Best regards,

    Laurent Lyaudet