[Django] #34603: ~Q() incorrectly interpreted as full rather than empty
Django
-------------------------------------+-------------------------------------
Reporter: Anders | Owner: nobody
Kaseorg |
Type: Bug | Status: new
Component: Database | Version: 4.2
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Since `Q()` is ([https://en.wikipedia.org/wiki/Empty_product#In_logic
correctly]) interpreted as matching every row, its negation `~Q()` should
be interpreted as matching no rows. However, Django incorrectly interprets
`~Q()` as matching every row like `Q()`.
{{{
>>> from django.db.models import Q
>>> from my_app.models import Client
>>> Client.objects.count()
12
>>> Client.objects.filter(Q()).count() # good, expected 12
12
>>> Client.objects.filter(~Q()).count() # bad, expected 0
12
}}}
Related: #5261, which introduced this incorrect behavior and added an
incorrect test for it.
--
Ticket URL: <https://code.djangoproject.com/ticket/34603>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: closed
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution: invalid
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => invalid
Comment:
Thanks for the ticket, however, IMO the current behavior is correct. An
empty `Q()` negated or not shouldn't change the `WHERE` statement as it
doesn't contain any filters.
--
Ticket URL: <https://code.djangoproject.com/ticket/34603#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: Anders Kaseorg | Owner: nobody
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: closed => new
* resolution: invalid =>
Comment:
That isn’t how logic works.
The expected meaning of `.filter(~Q(k_1=v_1, k_2=v_2, …, k_n=v_n))` is to
exclude those rows `r` such that for all `1 ≤ i ≤ n`, row `r` satisfies
`k_i=v_i`. If we plug in `n = 0`, then ''every'' row `r`
[https://en.wikipedia.org/wiki/Vacuous_truth vacuously] satisfies the
condition that for all `1 ≤ i ≤ n`, row `r` satisfies `k_i=v_i`, so every
row should be excluded.
----
Here’s an example. Consider this 3-column table with 8 rows:
||= **a** =||= **b** =||= **c** =||
|| 0 || 0 || 0 ||
|| 0 || 0 || 1 ||
|| 0 || 1 || 0 ||
|| 0 || 1 || 1 ||
|| 1 || 0 || 0 ||
|| 1 || 0 || 1 ||
|| 1 || 1 || 0 ||
|| 1 || 1 || 1 ||
Then `.filter(~Q(a=1, b=1, c=1))` gives 7 rows—those that don’t match `1 1
1`:
||= **a** =||= **b** =||= **c** =||
|| 0 || 0 || 0 ||
|| 0 || 0 || 1 ||
|| 0 || 1 || 0 ||
|| 0 || 1 || 1 ||
|| 1 || 0 || 0 ||
|| 1 || 0 || 1 ||
|| 1 || 1 || 0 ||
`.filter(~Q(a=1, b=1))` gives 6 rows—those that don’t match `1 1 *`:
||= **a** =||= **b** =||= **c** =||
|| 0 || 0 || 0 ||
|| 0 || 0 || 1 ||
|| 0 || 1 || 0 ||
|| 0 || 1 || 1 ||
|| 1 || 0 || 0 ||
|| 1 || 0 || 1 ||
`.filter(~Q(a=1))` gives 4 rows—those that don’t match `1 * *`:
||= **a** =||= **b** =||= **c** =||
|| 0 || 0 || 0 ||
|| 0 || 0 || 1 ||
|| 0 || 1 || 0 ||
|| 0 || 1 || 1 ||
and we should expect `.filter(~Q())` to give 0 rows—those that don’t match
`* * *`:
||= **a** =||= **b** =||= **c** =||
----
Another explanation comes from the invariant that the meaning of `Q(…)`
should be unchanged if we add a criterion that matches every row. For
example, in a table with an `id` column of nonnegative integers,
`.filter(~Q(k_1=v_1, k_2=v_2))` is equivalent to `.filter(~Q(k_1=v_1,
k_2=v_2, id__gte=0))`. So, similarly, `.filter(~Q())` should be equivalent
to `.filter(~Q(id__gte=0))`. Yet Django gives
{{{
>>> Client.objects.filter(~Q()).count()
12
>>> Client.objects.filter(~Q(id__gte=0)).count()
0
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34603#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: Anders Kaseorg | Owner: nobody
Component: Database layer | Version: 4.2
(models, ORM) |
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => wontfix
Comment:
I understand how the current logic works (there is no need for a long
examples), and I'm not sure how it justifies that `~Q()` should return an
empty queryset. `Q()` is "nothing", if you will negate "nothing" you will
still get "nothing". This is rather a philosophical discourse.
I appreciate you'd like to reopen the ticket, but please
[https://docs.djangoproject.com/en/stable/internals/contributing/triaging-
tickets/#closing-tickets follow the triaging guidelines with regards to
wontfix tickets] and take this to DevelopersMailingList, where you'll
reach a wider audience and see what other think.
--
Ticket URL: <https://code.djangoproject.com/ticket/34603#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: Anders Kaseorg | Owner: nobody
Type: Bug | Status: closed
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Anders Kaseorg):
There’s an important difference between ''including'' nothing and
''excluding'' nothing. We can’t just say these are the same because they
both involve “nothing”.
- `Q()` matches every row that satisfies no particular restrictions—i.e.,
it matches every row. (In Python, `all([]) == True`.)
- `~Q()` should match everything ''other than'' the rows that satisfy no
particular restrictions–i.e., it should match no rows. (In Python,
`not(all([])) == False`.)
I know this is a dense explanation that’s hard to digest in this abstract
form. But the rules of formal logic do give us a right answer here; this
is not some ambiguous question of philosophy.
--
Ticket URL: <https://code.djangoproject.com/ticket/34603#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: Anders Kaseorg | Owner: nobody
Type: Bug | Status: closed
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak):
Replying to [comment:4 Anders Kaseorg]:
> There’s an important difference between ''including'' nothing and
''excluding'' nothing.
TBH, I don't see any difference between including and excluding a void.
> I know this is a dense explanation that’s hard to digest in this
abstract form. But the rules of formal logic do give us a right answer
here; this is not some ambiguous question of philosophy.
As far as I'm aware, that's not true. Rules of mathematical logic don't
give us a clear answer how to negate "nothing". For example, `NOT NULL` is
`NULL`, it doesn't have a logical evaluation. Again, start a discussion on
the Django Forum or on the mailing list, if you don't agree.
--
Ticket URL: <https://code.djangoproject.com/ticket/34603#comment:5>