[Django] #32684: Annotate query with Subquery with None value cannot be reliably filtered (or excluded)

[Django]

#32684: Annotate query with Subquery with None value cannot be reliably filtered
(or excluded)
-------------------------------------+-------------------------------------
Reporter: Gerben | Owner: nobody
Morsink |
Type: Bug | Status: new
Component: Database | Version: 3.1
layer (models, ORM) |
Severity: Normal | Keywords: annotate, Subquery
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Let's say I have the following query

{{{
last_location = Subquery(PersonLocation.objects.filter(
person=OuterRef('id'),
departure_date=None).order_by('-arrival_date').values('location')[:1])

persons1 =
Person.objects_with_inactive.annotate(current_location=last_location)
}}}
Where PersonLocation is the table connecting the Person to a Location (to
keep track of the location history of a person).

And further in the code I do:

{{{
persons2 = persons1.exclude(current_location=location.id)
}}}
This will always return an empty queryset. Even if in persons1 there are
person objects that have `current_location = None` annotated.
This is very counter-intuitive, right?

I'm also not aware of a way to fix this. I tried
`persons1.filter(~Q(current_location=location.id))`, but the result is the
same.

--
Ticket URL: <https://code.djangoproject.com/ticket/32684>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#32684: Annotate query with Subquery with None value cannot be reliably filtered
(or excluded)
-------------------------------------+-------------------------------------

Reporter: Gerben Morsink | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution:

Keywords: annotate, Subquery | Triage Stage:

| Unreviewed
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Description changed by Gerben Morsink:

Old description:

> Let's say I have the following query
>
> {{{
> last_location = Subquery(PersonLocation.objects.filter(
> person=OuterRef('id'),
> departure_date=None).order_by('-arrival_date').values('location')[:1])
>
> persons1 =
> Person.objects_with_inactive.annotate(current_location=last_location)
> }}}
> Where PersonLocation is the table connecting the Person to a Location (to
> keep track of the location history of a person).
>
> And further in the code I do:
>

> {{{
> persons2 = persons1.exclude(current_location=location.id)
> }}}
> This will always return an empty queryset. Even if in persons1 there are
> person objects that have `current_location = None` annotated.
> This is very counter-intuitive, right?
>
> I'm also not aware of a way to fix this. I tried
> `persons1.filter(~Q(current_location=location.id))`, but the result is
> the same.

New description:

Let's say I have the following query

{{{
last_location = Subquery(PersonLocation.objects.filter(
person=OuterRef('id'),
departure_date=None).order_by('-arrival_date').values('location')[:1])

persons1 =
Person.objects_with_inactive.annotate(current_location=last_location)
}}}
Where PersonLocation is the table connecting the Person to a Location (to
keep track of the location history of a person).

And further in the code I do:

{{{
persons2 = persons1.exclude(current_location=location.id)
}}}
This will always return an empty queryset. Even if in persons1 there are
person objects that have `current_location = None` annotated.
This is very counter-intuitive, right?

I'm also not aware of a way to fix this. I tried
`persons1.filter(~Q(current_location=location.id))`, but the result is the
same.

NOTE: the location.id is a UUID, not sure if that makes a difference
though.

--

--
Ticket URL: <https://code.djangoproject.com/ticket/32684#comment:1>

[Django]

#32684: Annotate query with Subquery with None value cannot be reliably filtered
(or excluded)
-------------------------------------+-------------------------------------

Reporter: Gerben Morsink | Owner: nobody

Type: Bug | Status: closed

Component: Database layer | Version: 3.1
(models, ORM) |

Severity: Normal | Resolution: duplicate

Keywords: annotate, Subquery | Triage Stage:

| Unreviewed
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mariusz Felisiak):

* status: new => closed
* resolution: => duplicate

Comment:

Duplicate of #32398.

--
Ticket URL: <https://code.djangoproject.com/ticket/32684#comment:2>

[Django]

#32684: Annotate query with Subquery with None value cannot be reliably filtered
(or excluded)
-------------------------------------+-------------------------------------

Reporter: Gerben Morsink | Owner: nobody
Type: Bug | Status: closed
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution: duplicate

Keywords: annotate, Subquery | Triage Stage:

| Unreviewed
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Gerben Morsink):

Replying to [comment:2 Mariusz Felisiak]:
> Duplicate of #32398.

I'm sorry I didn't find that issue. Thank you.

--
Ticket URL: <https://code.djangoproject.com/ticket/32684#comment:3>