[Django] #32809: Filtering with Q and OR gets duplicated entries
Django
-------------------------------------+-------------------------------------
Reporter: Ismael | Owner: nobody
Jerez |
Type: Bug | Status: new
Component: Database | Version: 3.2
layer (models, ORM) | Keywords: queryset, Q,
Severity: Normal | filtering, OR, annotation, |
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Hi:
Django version: 3.2.4
Python version: 3.8
I cannot reproduce this bug in a simple example, so I can only show you
screenshots from my Pycharm Debug Tool, I am sorry.
I have one queryset with 3 objects which contains annotations, CharField
and ManyToManyField attributes. Then I filter by a query using Q like
this:
{{{
(OR: ('created_date_dt_custom__icontains', 'a'), ('id__icontains', 'a'),
('title__unaccent__icontains', 'a'), ('created_date_dt_custom__icontains',
'a'), ('id__icontains', 'a'), ('title__unaccent__icontains', 'a'),
('promoters__in', <QuerySet [<User: admin>, <User: u********>]>))
}}}
And gets a queryset with 4 objects as a result, one of the objects is
duplicated. If I delete the last filter ('promoters__in', <QuerySet
[<User: admin>, <User: u********>]>) it works fine, so the problem is on
this filter or maybe the concatenation of filters with this kind.
I have no problem on Django version ~= 2.
Here are the screenshots of my debug tool showing the problem:
- Screenshot of the code: https://i.ibb.co/Fnb53rn/ss1.png
- Screenshot of debug tool with variable values (Project object (19)
duplicated): https://i.ibb.co/b34FwSM/ss2.png
Thank you in advance, hope it is fixed soon,
Ismael.
--
Ticket URL: <https://code.djangoproject.com/ticket/32809>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: queryset, Q, | Triage Stage:
filtering, OR, annotation, | | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* Attachment "ss1.png" added.
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: queryset, Q, | Triage Stage:
filtering, OR, annotation, | | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* Attachment "ss2.png" added.
Django
-------------------------------------+-------------------------------------
Component: Database layer | Version: 3.2
(models, ORM) |
Keywords: queryset, Q, | Triage Stage:
filtering, OR, annotation, | | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => needsinfo
Comment:
Thanks for this report, unfortunately we will not be able to reproduce the
issue or confirm that it's a regression without a queryset. It looks like
a regression in c8b659430556dca0b2fe27cf2ea0f8290dbafecd but again it's
hard to confirm without a queryset.
Can you provide models and filters? Can you confirm that it's a regression
in 3.2.1?
--
Ticket URL: <https://code.djangoproject.com/ticket/32809#comment:1>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
filtering, OR, annotation, | | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: closed => new
* resolution: needsinfo =>
Comment:
Hi again:
I am sorry for not being able to reproduce this before. Here is a simple
example:
1) models.py:
{{{
from django.contrib.auth.models import User
from django.db import models
class Example(models.Model):
example_attr = models.CharField(max_length=100)
users = models.ManyToManyField(User)
}}}
2) I created two User's instances from admin site:
- username = 'admin'
- username = 'other'
And two Group's instances from admin site:
- name = 'Group1'
- name = 'Group2'
Associate Group1 and Group2 to users 'admin' and 'other' from admin site
too.
3) Now the code:
{{{
from example.models import Example
from django.db.models import Q
from django.contrib.auth.models import User, Group
example = Example.objects.create(example_attr='have a nice day')
user1 = User.objects.get(username='admin')
user2 = User.objects.get(username='other')
example.users.add(user1, user2)
filters = Q(example_attr__icontains='a') |
Q(users__in=User.objects.filter(groups__in=[Group.objects.get(name='Group1').pk]))
print(Example.objects.count()) # This prints "1"
print(Example.objects.filter(filters).count()) # This prints "2"
}}}
Hope this helps to reproduce the bug.
This bug is happening since Django 3.2.0 until 3.2.4.
Thanks in advance,
Ismael.
--
Ticket URL: <https://code.djangoproject.com/ticket/32809#comment:2>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 3.2
(models, ORM) |
Severity: Normal | Resolution:
Keywords: queryset, Q, | Triage Stage:
filtering, OR, annotation, | | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Ismael Jerez):
Forgot to mention that the bug is only happening when the second Q filter
is used but not with simple '__icontains' filters concatenated by OR
expressions.
--
Ticket URL: <https://code.djangoproject.com/ticket/32809#comment:3>
Django
-------------------------------------+-------------------------------------
Component: Database layer | Version: 3.2
(models, ORM) |
Keywords: queryset, Q, | Triage Stage:
filtering, OR, annotation, | | Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => needsinfo
Comment:
Thanks for extra details, however the described behavior is expected,
[https://code.djangoproject.com/ticket/28292#comment:1 documented], and
has not been changed in Django 3.2. Django < 3.2 behaves the same.
Please reopen the ticket if you can debug your issue and provide way to
reproduce a regression in Django 3.2
--
Ticket URL: <https://code.djangoproject.com/ticket/32809#comment:4>