[Django] #30000: Passing QuerySet constructed with .union() causes ModelMultipleChoiceField to return wrong values
Django
to return wrong values
-----------------------------------------+------------------------
Reporter: thoha | Owner: nobody
Type: Uncategorized | Status: new
Component: Forms | Version: 1.11
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+------------------------
I have a form that displays large lists of selectable records using
ModelMultipleChoiceFields. These fields need to show a predetermined list
of records, obtained through some database queries performed within the
Form's class __init__ block.
The QuerySet required is constructed by reaching through a few ForeignKey
relationships to retrieve the required records and then using the .union()
method to assemble the QuerySet.
e.g.:
{{{
# Retrieve original record:
base = BaseModel.objects.filter(source_name=source_name).first()
# Get related record via ForeignKey relationship:
related_record = base.related_record
# related_record.other_related_things has ManyToMany relationship to
another Model (one or more records).
# So get them:
other_related_things = related_record.other_related_things.all()
# Loop through these:
union_queryset = None
for other_related_thing in other_related_things:
# Each other_related_thing has a set of SelectableRecords associated
with it, via a ForeignKey relationship from the latter model to the former
one. This gets that QuerySet:
other_related_thing_queryset =
other_related_thing.selectablerecords_set.all()
# Build union_queryset:
if union_queryset is None:
union_queryset = other_related_thing_queryset
else:
union_queryset =
union_queryset_object.union(other_related_thing_queryset)
# The resulting union_queryset is a unified queryset containing
all of the required fields.
# If only one other_related_thing exists, the result of evaluating
the union_queryset QuerySet ought to be the
# same as the result of evaluating SelectableRecords.objects.all()
}}}
However, when I pass the union_queryset to my ModelMultipleChoiceFields,
**validating the form returns all of the records contained in
union_queryset instead of the records selected on the form**. The form
displays correctly.
The code used for this is:
{{{
self.fields['selectable_records'] =
forms.ModelMultipleChoiceField(queryset=union_queryset, required=False,
label='Selectable
records',
widget=forms.CheckboxSelectMultiple)
}}}
Do take note that the documentation for the .union() method
([https://docs.djangoproject.com/en/2.1/ref/models/querysets/#union here])
indicates there are some limits to the QuerySet returned, and I assume my
problem has to do with this.
--
Ticket URL: <https://code.djangoproject.com/ticket/30000>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
to return wrong values
Reporter: thoha | Owner: nobody
Type: Uncategorized | Status: new
Component: Forms | Version: 1.11
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Description changed by thoha:
Old description:
New description:
problem may have to do with this. Still, it would be welcome if this were
explained somewhere (should this behaviour be normal), as it is rather
unexpected.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/30000#comment:1>
Django
to return wrong values
Reporter: thoha | Owner: nobody
Type: Uncategorized | Status: new
Component: Forms | Version: 1.11
Keywords: queryset, union, | Triage Stage:
forms | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by thoha):
* keywords: => queryset, union, forms
--
Ticket URL: <https://code.djangoproject.com/ticket/30000#comment:2>
Django
to return wrong values
Reporter: thoha | Owner: nobody
Type: Uncategorized | Status: new
Component: Forms | Version: 1.11
Keywords: queryset, union, | Triage Stage:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by thoha):
After some further testing it appears the unexpected behaviour comes from
the fact that UNION querysets cannot be filtered, which the
ModelMultipleChoiceField apparently does out of sight. Still, I believe
that the observed behaviour is undesired, and the field should either
accept the union queryset and work properly, or raise an exception if a
union queryset is passed.
I also found that using
{{{
union_queryset =
SelectableRecords.objects.filter(other_related_thing_name__in=other_related_things)
}}}
instead of the for-loop results in a queryset that actually works properly
with the ModelMultipleChoiceField. Based on
[https://stackoverflow.com/questions/50638442/django-queryset-union-
return-broken-queryset-filter-and-get-return-every this StackOverflow
question].
--
Ticket URL: <https://code.djangoproject.com/ticket/30000#comment:3>
Django
to return wrong values
Reporter: thoha | Owner: nobody
Type: Bug | Status: new
Component: Forms | Version: 1.11
Severity: Normal | Resolution:
Keywords: queryset, union, | Triage Stage:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by thoha):
* type: Uncategorized => Bug
--
Ticket URL: <https://code.djangoproject.com/ticket/30000#comment:4>
Django
filter() attempt.
-------------------------------------+-------------------------------------
Reporter: thoha | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.11
(models, ORM) |
Severity: Normal | Resolution:
Keywords: queryset union | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Simon Charette):
* keywords: queryset, union, forms => queryset union
* component: Forms => Database layer (models, ORM)
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/30000#comment:5>
Django
filter() attempt.
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 1.11
(models, ORM) |
Severity: Normal | Resolution:
Keywords: queryset union | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Prabakaran Kumaresshan):
* cc: Prabakaran Kumaresshan (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/30000#comment:6>
Django
filter() attempt.
-------------------------------------+-------------------------------------
| Ramezani
Type: Bug | Status: assigned
Component: Database layer | Version: 1.11
(models, ORM) |
Severity: Normal | Resolution:
Keywords: queryset union | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Hasan Ramezani):
* owner: nobody => Hasan Ramezani
* status: new => assigned
Comment:
should we raise an exception for all `filter()` attempts? or just for
unsupported one?
if we should we raise an exception just for unsupported, is there any way
to find them?
--
Ticket URL: <https://code.djangoproject.com/ticket/30000#comment:7>
Django
filter() attempt.
-------------------------------------+-------------------------------------
Reporter: thoha | Owner: Hasan
| Ramezani
Type: Bug | Status: assigned
Component: Database layer | Version: 1.11
(models, ORM) |
Severity: Normal | Resolution:
Keywords: queryset union | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by thoha):
Replying to [comment:7 Hasan Ramezani]:
> should we raise an exception for all `filter()` attempts? or just for
unsupported one?
> if we should we raise an exception just for unsupported, is there any
way to find them?
Unsupported ones, I'd think. For my original problem, a simple check would
be to do a count on the number of selections made in the form and
comparing that to the number of records returned - normally they should
match.
--
Ticket URL: <https://code.djangoproject.com/ticket/30000#comment:8>
Django
filter() attempt.
-------------------------------------+-------------------------------------
Reporter: thoha | Owner: Hasan
| Ramezani
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution: duplicate
Keywords: queryset union | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by felixxm):
* status: assigned => closed
* version: 1.11 => master
* resolution: => duplicate
Comment:
`filter()` and `exclude()` are unsupported but doesn't raise an error yet.
As [https://docs.djangoproject.com/en/stable/ref/models/querysets/#union
per the documentation], ''"only LIMIT, OFFSET, COUNT(*), ORDER BY, and
specifying columns (i.e. slicing, count(), order_by(), and
values()/values_list()) are allowed on the resulting QuerySet."''.
Duplicate of #27995.
See also a ticket #28519 to support these features.
--
Ticket URL: <https://code.djangoproject.com/ticket/30000#comment:9>