[Django] #31940: Order of WHERE clauses do not match the order of the arguments to .filter()

[Django]

#31940: Order of WHERE clauses do not match the order of the arguments to .filter()
-------------------------------------+-------------------------------------
Reporter: Tommy Li | Owner: nobody
Type: Bug | Status: new
Component: Database | Version: 3.0
layer (models, ORM) |
Severity: Normal | Keywords: orm
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
With the following sample model

{{{#!python
class MyModel(models.Model):
a_field = models.IntegerField()
b_field = models.IntegerField()
}}}

If I run the following query:
{{{#!python
MyModel.objects.filter(b_field = 1, a_field = 2)
}}}
the SQL that it generates is
{{{
SELECT "mymodel"."a_field", "mymodel"."b_field
FROM "mymodel"
WHERE ("mymodel"."a_field" = 2 AND "mymodel"."b_field" = 1)
}}}

I would expect the order of the clauses in the WHERE statement to match
the order of arguments passed to the filter statement. In my case,
`b_field` has a much higher cardinality than `a_field` and I want the
query to use an index I have on `(b_field, a_field)`

After some experimentation, it looks like the WHERE clauses are ordered
alphabetically by column name. Is that explicitly intended, or is it just
a side effect of some implementation detail?

--
Ticket URL: <https://code.djangoproject.com/ticket/31940>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#31940: Order of WHERE clauses do not match the order of the arguments to .filter()
-------------------------------------+-------------------------------------
Reporter: Tommy Li | Owner: nobody
Type: Bug | Status: new

Component: Database layer | Version: 3.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: orm | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Description changed by Tommy Li:

Old description:

> With the following sample model
>
> {{{#!python
> class MyModel(models.Model):
> a_field = models.IntegerField()
> b_field = models.IntegerField()
> }}}
>
> If I run the following query:
> {{{#!python
> MyModel.objects.filter(b_field = 1, a_field = 2)
> }}}
> the SQL that it generates is
> {{{
> SELECT "mymodel"."a_field", "mymodel"."b_field
> FROM "mymodel"
> WHERE ("mymodel"."a_field" = 2 AND "mymodel"."b_field" = 1)
> }}}
>
> I would expect the order of the clauses in the WHERE statement to match
> the order of arguments passed to the filter statement. In my case,
> `b_field` has a much higher cardinality than `a_field` and I want the
> query to use an index I have on `(b_field, a_field)`
>
> After some experimentation, it looks like the WHERE clauses are ordered
> alphabetically by column name. Is that explicitly intended, or is it just
> a side effect of some implementation detail?

New description:

With the following sample model

{{{#!python
class MyModel(models.Model):
a_field = models.IntegerField()
b_field = models.IntegerField()
}}}

If I run the following query:
{{{#!python
MyModel.objects.filter(b_field = 1, a_field = 2)
}}}
the SQL that it generates is
{{{
SELECT "mymodel"."a_field", "mymodel"."b_field
FROM "mymodel"
WHERE ("mymodel"."a_field" = 2 AND "mymodel"."b_field" = 1)
}}}

I would expect the order of the clauses in the WHERE statement to match
the order of arguments passed to the filter statement. In my case,
`b_field` has a much higher cardinality than `a_field` and I want the
query to use an index I have on `(b_field, a_field)`

After some experimentation, it looks like the WHERE clauses are ordered
alphabetically by column name. Is that explicitly intended, or is it just
a side effect of some implementation detail?

A workaround is to run
{{{#!python
MyModel.objects.filter(b_field = 1).filter(a_field = 2)
}}}
But this is pretty inconvenient if you just want to run a .get, you would
have to build out a verbose .filter().filter().first() query

--

--
Ticket URL: <https://code.djangoproject.com/ticket/31940#comment:1>

[Django]

#31940: Order of WHERE clauses do not match the order of the arguments to

.filter().

-------------------------------------+-------------------------------------
Reporter: Tommy Li | Owner: nobody

Type: | Status: closed
Cleanup/optimization |

Component: Database layer | Version: 3.0
(models, ORM) |

Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by felixxm):

* keywords: orm =>
* status: new => closed
* type: Bug => Cleanup/optimization
* resolution: => wontfix

Comment:

Thanks for this ticket.

> After some experimentation, it looks like the WHERE clauses are ordered
alphabetically by column name. Is that explicitly intended, or is it just
a side effect of some implementation detail?

`QuerySet.filter()` uses `Q()` objects. We added sorting to `kwargs` to
make `Q.deconstruct()` deterministic on Python 3.5, see #29125. Django
3.0+ supports only Python 3.6+, so we could theoretically revert this fix,
but this will generate migrations for existing project. I think it isn't
worth.

> I would expect the order of the clauses in the WHERE statement to match
the order of arguments passed to the filter statement. In my case, b_field
has a much higher cardinality than a_field and I want the query to use an
index I have on (b_field, a_field)

Clauses ordering should not affect indexes, as far as I'm aware databases
will use a `(b, a)`-index in both cases `b=1 AND a=2` and `a=2 AND b=1`.

--
Ticket URL: <https://code.djangoproject.com/ticket/31940#comment:2>