[Django] #35344: GeneratedField get_col output_field bug

[Django]

#35344: GeneratedField get_col output_field bug
-------------------------------------+-------------------------------------
Reporter: us77ipis | Owner: us77ipis
Type: Bug | Status: assigned
Component: Database | Version: 5.0
layer (models, ORM) |
Severity: Release | Keywords:
blocker |
Triage Stage: | Has patch: 1
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Related issue #34838.

For generated fields, `get_col` gets called with `output_field=<instance
of the generated field>`. Consequently, if an `alias` is specified, the
`output_field` of the generated `Col` is of type `GeneratedField` instead
of the actual `output_field` of the `GeneratedField`, since the current
code only handles the case where `get_col`'s `output_field` parameter is
`None`, but not when it is `self`.

**Error**
{{{
File "django/contrib/postgres/fields/ranges.py", line 253, in
as_postgresql
cast_internal_type =
self.lhs.output_field.base_field.get_internal_type()
AttributeError: 'GeneratedField' object has no attribute 'base_field'
}}}

**Patch**

{{{#!diff
diff --git a/django/db/models/fields/generated.py
b/django/db/models/fields/generated.py
index 257feeeba2..5b6b188df0 100644
--- a/django/db/models/fields/generated.py
+++ b/django/db/models/fields/generated.py
@@ -39,7 +39,7 @@ class GeneratedField(Field):
return Col(self.model._meta.db_table, self, self.output_field)

def get_col(self, alias, output_field=None):
- if alias != self.model._meta.db_table and output_field is None:
+ if alias != self.model._meta.db_table and output_field in (None,
self):
output_field = self.output_field
return super().get_col(alias, output_field)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35344>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35344: GeneratedField get_col output_field bug
-------------------------------------+-------------------------------------
Reporter: us77ipis | Owner: us77ipis
Type: Bug | Status: assigned

Component: Database layer | Version: 5.0
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by us77ipis):

PR https://github.com/django/django/pull/18034.
--
Ticket URL: <https://code.djangoproject.com/ticket/35344#comment:1>

[Django]

#35344: GeneratedField get_col output_field bug
-------------------------------------+-------------------------------------
Reporter: us77ipis | Owner: us77ipis
Type: Bug | Status: assigned

Component: Database layer | Version: dev

(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by us77ipis):

* version: 5.0 => dev

--
Ticket URL: <https://code.djangoproject.com/ticket/35344#comment:2>

[Django]

#35344: GeneratedField get_col output_field bug
-------------------------------------+-------------------------------------
Reporter: us77ipis | Owner: us77ipis
Type: Bug | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Release blocker | Resolution:

Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Simon Charette):

* stage: Unreviewed => Accepted

Comment:

Nice catch and thank you for submitting a MR.

Pending a mention in the 5.0.4 release notes this LGTM!
--
Ticket URL: <https://code.djangoproject.com/ticket/35344#comment:3>

[Django]

#35344: GeneratedField get_col output_field bug
-------------------------------------+-------------------------------------

Reporter: Johannes Westphal | Owner: Johannes
| Westphal

Type: Bug | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Johannes Westphal):

Should I update the release notes in the PR?
--
Ticket URL: <https://code.djangoproject.com/ticket/35344#comment:4>

[Django]

#35344: GeneratedField get_col output_field bug
-------------------------------------+-------------------------------------
Reporter: Johannes Westphal | Owner: Johannes
| Westphal
Type: Bug | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Johannes Westphal):

Well, I added a mention in the 5.0.4 release notes in the PR.
--
Ticket URL: <https://code.djangoproject.com/ticket/35344#comment:5>

[Django]

#35344: GeneratedField get_col output_field bug
-------------------------------------+-------------------------------------
Reporter: Johannes Westphal | Owner: Johannes
| Westphal
Type: Bug | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Simon Charette):

yep, that' how it's done!
--
Ticket URL: <https://code.djangoproject.com/ticket/35344#comment:6>

[Django]

#35344: GeneratedField get_col output_field bug
-------------------------------------+-------------------------------------
Reporter: Johannes Westphal | Owner: Johannes
| Westphal
Type: Bug | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Natalia Bidart):

Thank you Simon and Johannes for your help, the PR looks good, I'm
completing the review and I'll merge soon.

While I understand how the provided test is relevant for the fix, I would
like to understand a bit better the high-level use case. Would you have an
example of a query producing the reported error?
--
Ticket URL: <https://code.djangoproject.com/ticket/35344#comment:7>

[Django]

#35344: GeneratedField get_col output_field bug
-------------------------------------+-------------------------------------
Reporter: Johannes Westphal | Owner: Johannes
| Westphal
Type: Bug | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Release blocker | Resolution:

Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Natalia Bidart):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/35344#comment:8>

[Django]

#35344: GeneratedField get_col output_field bug
-------------------------------------+-------------------------------------
Reporter: Johannes Westphal | Owner: Johannes
| Westphal
Type: Bug | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Johannes Westphal):

Here a minimal example to reproduce the crash:

**Model**

{{{#!python
from django.contrib.gis.db import models
from django.contrib.postgres.fields import DateTimeRangeField
from django.contrib.postgres.functions import TransactionNow

class DateTimeRange(models.Func):
function = 'tstzrange'
template = "%(function)s(%(expressions)s)"
output_field = DateTimeRangeField()

def __init__(self, a, b):
super().__init__(a, b)

class P(models.Model):
start = models.DateTimeField(db_default=TransactionNow())
finish = models.DateTimeField(null=True)
timerange =
models.GeneratedField(expression=DateTimeRange(models.F('start'),
models.F('finish')), output_field=DateTimeRangeField(), db_persist=True)

class T(models.Model):
p1 = models.ForeignKey(P, on_delete=models.CASCADE,
related_name='tp1s')
p2 = models.ForeignKey(P, on_delete=models.CASCADE,
related_name='tp2s')
}}}

**Query**

{{{#!python
from django.utils import timezone

now = timezone.now()
T.objects.filter(
p1__timerange__contains=now,
p2__timerange__contains=now,
).count()
}}}


**Traceback**

{{{
Traceback (most recent call last):
File "test.py", line 14, in <module>
).count()
File "django/db/models/query.py", line 620, in count
return self.query.get_count(using=self.db)
File "django/db/models/sql/query.py", line 629, in get_count
return obj.get_aggregation(using, {"__count": Count("*")})["__count"]
File "django/db/models/sql/query.py", line 615, in get_aggregation
result = compiler.execute_sql(SINGLE)
File "django/db/models/sql/compiler.py", line 1549, in execute_sql
sql, params = self.as_sql()
File "django/db/models/sql/compiler.py", line 764, in as_sql
self.compile(self.where) if self.where is not None else ("", [])
File "django/db/models/sql/compiler.py", line 546, in compile
sql, params = node.as_sql(self, self.connection)
File "django/db/models/sql/where.py", line 151, in as_sql
sql, params = compiler.compile(child)
File "django/db/models/sql/compiler.py", line 544, in compile
sql, params = vendor_impl(self, self.connection)

File "django/contrib/postgres/fields/ranges.py", line 253, in
as_postgresql
cast_internal_type =
self.lhs.output_field.base_field.get_internal_type()
AttributeError: 'GeneratedField' object has no attribute 'base_field'
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/35344#comment:9>

[Django]

#35344: GeneratedField get_col output_field bug
-------------------------------------+-------------------------------------
Reporter: Johannes Westphal | Owner: Johannes
| Westphal

Type: Bug | Status: closed

Component: Database layer | Version: dev
(models, ORM) |

Severity: Release blocker | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Johannes Westphal <jojo@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"5f180216409d75290478c71ddb0ff8a68c91dc16" 5f18021]:
{{{#!CommitTicketReference repository=""
revision="5f180216409d75290478c71ddb0ff8a68c91dc16"
Fixed #35344, Refs #34838 -- Corrected output_field of resolved columns
for GeneratedFields in aliased tables.

Thanks Simon Charette for the review.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35344#comment:10>

[Django]

#35344: GeneratedField get_col output_field bug
-------------------------------------+-------------------------------------
Reporter: Johannes Westphal | Owner: Johannes
| Westphal
Type: Bug | Status: closed
Component: Database layer | Version: dev
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Natalia <124304+nessita@…>):

In [changeset:"14ab15d69ab2171ef26b295dcbd561b723b7eb7e" 14ab15d6]:
{{{#!CommitTicketReference repository=""
revision="14ab15d69ab2171ef26b295dcbd561b723b7eb7e"
[5.0.x] Fixed #35344, Refs #34838 -- Corrected output_field of resolved

columns for GeneratedFields in aliased tables.

Thanks Simon Charette for the review.

Backport of 5f180216409d75290478c71ddb0ff8a68c91dc16 from main
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35344#comment:11>

[Django]

#35344: GeneratedField get_col output_field bug
-------------------------------------+-------------------------------------
Reporter: Johannes Westphal | Owner: Johannes
| Westphal
Type: Bug | Status: closed
Component: Database layer | Version: dev
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Johannes Westphal):

Thank you Natalia and Simon for the fast processing of my bug report and
pull request.
--
Ticket URL: <https://code.djangoproject.com/ticket/35344#comment:12>