[Django] #33229: BaseDatabaseOperations.adapt_datetimefield_value and adapt_timefield_value() crash on expressions
Django
adapt_timefield_value() crash on expressions
-------------------------------------+-------------------------------------
Reporter: Tim | Owner: nobody
Graham |
Type: Bug | Status: new
Component: Database | Version: dev
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 1
UI/UX: 0 |
-------------------------------------+-------------------------------------
While building a backend for Snowflake, I saw these test failures:
{{{
======================================================================
ERROR: test_in_lookup_allows_F_expressions_and_expressions_for_datetimes
(expressions.tests.IterableLookupInnerExpressionsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/tim/code/django/django/db/backends/utils.py", line 87, in
_execute
return self.cursor.execute(sql, params)
File "/home/tim/.virtualenvs/django39/lib/python3.9/site-
packages/snowflake/connector/cursor.py", line 777, in execute
Error.errorhandler_wrapper(
File "/home/tim/.virtualenvs/django39/lib/python3.9/site-
packages/snowflake/connector/errors.py", line 255, in errorhandler_wrapper
handed_over = Error.hand_to_other_handler(
File "/home/tim/.virtualenvs/django39/lib/python3.9/site-
packages/snowflake/connector/errors.py", line 310, in
hand_to_other_handler
cursor.errorhandler(connection, cursor, error_class, error_value)
File "/home/tim/.virtualenvs/django39/lib/python3.9/site-
packages/snowflake/connector/errors.py", line 189, in default_errorhandler
raise error_class(
snowflake.connector.errors.ProgrammingError: Timestamp
'Col(expressions_ExPeRiMeNt, expressions.Experiment.start)' is not
recognized
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/tim/code/django/tests/expressions/tests.py", line 959, in
test_in_lookup_allows_F_expressions_and_expressions_for_datetimes
self.assertSequenceEqual(queryset, [r1])
File "/opt/python3.9.5/lib/python3.9/unittest/case.py", line 949, in
assertSequenceEqual
len1 = len(seq1)
File "/home/tim/code/django/django/db/models/query.py", line 262, in
__len__
self._fetch_all()
File "/home/tim/code/django/django/db/models/query.py", line 1324, in
_fetch_all
self._result_cache = list(self._iterable_class(self))
File "/home/tim/code/django/django/db/models/query.py", line 51, in
__iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch,
chunk_size=self.chunk_size)
File "/home/tim/code/django/django/db/models/sql/compiler.py", line
1175, in execute_sql
cursor.execute(sql, params)
File "/home/tim/code/django/django/db/backends/utils.py", line 101, in
execute
return super().execute(sql, params)
File "/home/tim/code/django/django/db/backends/utils.py", line 69, in
execute
return self._execute_with_wrappers(sql, params, many=False,
executor=self._execute)
File "/home/tim/code/django/django/db/backends/utils.py", line 78, in
_execute_with_wrappers
return executor(sql, params, many, context)
File "/home/tim/code/django/django/db/backends/utils.py", line 87, in
_execute
return self.cursor.execute(sql, params)
File "/home/tim/code/django/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/tim/code/django/django/db/backends/utils.py", line 87, in
_execute
return self.cursor.execute(sql, params)
File "/home/tim/.virtualenvs/django39/lib/python3.9/site-
packages/snowflake/connector/cursor.py", line 777, in execute
Error.errorhandler_wrapper(
File "/home/tim/.virtualenvs/django39/lib/python3.9/site-
packages/snowflake/connector/errors.py", line 255, in errorhandler_wrapper
handed_over = Error.hand_to_other_handler(
File "/home/tim/.virtualenvs/django39/lib/python3.9/site-
packages/snowflake/connector/errors.py", line 310, in
hand_to_other_handler
cursor.errorhandler(connection, cursor, error_class, error_value)
File "/home/tim/.virtualenvs/django39/lib/python3.9/site-
packages/snowflake/connector/errors.py", line 189, in default_errorhandler
raise error_class(
django.db.utils.ProgrammingError: Timestamp 'Col(expressions_ExPeRiMeNt,
expressions.Experiment.start)' is not recognized
======================================================================
ERROR: test_expressions_in_lookups_join_choice
(expressions.tests.IterableLookupInnerExpressionsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/tim/code/django/tests/expressions/tests.py", line 872, in
test_expressions_in_lookups_join_choice
self.assertSequenceEqual(queryset, [s1])
File "/opt/python3.9.5/lib/python3.9/unittest/case.py", line 949, in
assertSequenceEqual
len1 = len(seq1)
File "/home/tim/code/django/django/db/models/query.py", line 262, in
__len__
self._fetch_all()
File "/home/tim/code/django/django/db/models/query.py", line 1324, in
_fetch_all
self._result_cache = list(self._iterable_class(self))
File "/home/tim/code/django/django/db/models/query.py", line 51, in
__iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch,
chunk_size=self.chunk_size)
File "/home/tim/code/django/django/db/models/sql/compiler.py", line
1162, in execute_sql
sql, params = self.as_sql()
File "/home/tim/code/django/django/db/models/sql/compiler.py", line 528,
in as_sql
where, w_params = self.compile(self.where) if self.where is not None
else ("", [])
File "/home/tim/code/django/django/db/models/sql/compiler.py", line 445,
in compile
sql, params = node.as_sql(self, self.connection)
File "/home/tim/code/django/django/db/models/sql/where.py", line 81, in
as_sql
sql, params = compiler.compile(child)
File "/home/tim/code/django/django/db/models/sql/compiler.py", line 445,
in compile
sql, params = node.as_sql(self, self.connection)
File "/home/tim/code/django/django/db/models/lookups.py", line 194, in
as_sql
rhs_sql, rhs_params = self.process_rhs(compiler, connection)
File "/home/tim/code/django/django/db/models/lookups.py", line 248, in
process_rhs
return self.batch_process_rhs(compiler, connection)
File "/home/tim/code/django/django/db/models/lookups.py", line 261, in
batch_process_rhs
pre_processed = super().batch_process_rhs(compiler, connection, rhs)
File "/home/tim/code/django/django/db/models/lookups.py", line 58, in
batch_process_rhs
_, params = self.get_db_prep_lookup(rhs, connection)
File "/home/tim/code/django/django/db/models/lookups.py", line 217, in
get_db_prep_lookup
[get_db_prep_value(v, connection, prepared=True) for v in value]
File "/home/tim/code/django/django/db/models/lookups.py", line 217, in
<listcomp>
[get_db_prep_value(v, connection, prepared=True) for v in value]
File "/home/tim/code/django/django/db/models/fields/__init__.py", line
2297, in get_db_prep_value
return connection.ops.adapt_timefield_value(value)
File "/home/tim/code/django/django/db/backends/base/operations.py", line
511, in adapt_timefield_value
if timezone.is_aware(value):
File "/home/tim/code/django/django/utils/timezone.py", line 220, in
is_aware
return value.utcoffset() is not None
AttributeError: 'Col' object has no attribute 'utcoffset'
}}}
The `if hasattr(value, 'resolve_expression'):` guard added in
4f138fe5a496a81115c4fba6615a517fc62c3b17 wasn't added to
`BaseDatabaseOperations.adapt_datetimefield_value` (for the first test)
and `adapt_timefield_value()` (for the second test).
--
Ticket URL: <https://code.djangoproject.com/ticket/33229>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
adapt_timefield_value() crash on expressions
-------------------------------------+-------------------------------------
| Pandey
Type: Bug | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: nobody => Vishal Pandey
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/33229#comment:1>
Django
adapt_timefield_value() crash on expressions
-------------------------------------+-------------------------------------
| Pandey
Type: Bug | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Unreviewed => Accepted
Comment:
Thanks for the report. We should probably add the same to the
`BaseDatabaseOperations.adapt_datefield_value()`.
--
Ticket URL: <https://code.djangoproject.com/ticket/33229#comment:2>
Django
adapt_timefield_value() crash on expressions
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: Vishal Pandey => (none)
* status: assigned => new
--
Ticket URL: <https://code.djangoproject.com/ticket/33229#comment:3>
Django
adapt_timefield_value() crash on expressions
-------------------------------------+-------------------------------------
| SwastikTripathi
Type: Bug | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: (none) => SwastikTripathi
* status: new => assigned
* has_patch: 0 => 1
Comment:
I'm adding {{{ if hasattr(value, 'resolve_expression'):}}} the condition
to adapt_datetimefield_value() and adapt_timefield_value() in
django/db/backends/postgresql/operations.py and
django/db/backends/base/operations.py
Do you know why it wasn't added there?
--
Ticket URL: <https://code.djangoproject.com/ticket/33229#comment:4>
Django
adapt_timefield_value() crash on expressions
-------------------------------------+-------------------------------------
| SwastikTripathi
Type: Bug | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
Comment:
[https://github.com/django/django/pull/15081 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/33229#comment:5>
Django
adapt_timefield_value() crash on expressions
-------------------------------------+-------------------------------------
| SwastikTripathi
Type: Bug | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/33229#comment:6>
Django
adapt_timefield_value() crash on expressions
-------------------------------------+-------------------------------------
| SwastikTripathi
Component: Database layer | Version: dev
(models, ORM) |
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"a7e7043c8746933dafce652507d3b821801cdc7d" a7e7043c]:
{{{
#!CommitTicketReference repository=""
revision="a7e7043c8746933dafce652507d3b821801cdc7d"
Fixed #33229 -- Fixed
BaseDatabaseOperations.adapt_datetimefield_value()/adapt_timefield_value()
crash with expressions.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33229#comment:7>