[Django] #22536: Exceptions using date-based generic views with MySQL, USE_TZ=True and bad data
Django
data
-------------------------------+-------------------
Reporter: acdha | Owner: acdha
Type: Bug | Status: new
Component: Generic views | Version: 1.5
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+-------------------
I ran into a non-obvious exception while using the date-based generic
views. get_previous_day / get_next_day were raising an AttributeError
('NoneType' object has no attribute 'astimezone') in the following code
when the view was called with the earliest item in the system:
try:
result = getattr(qs[0], date_field)
except IndexError:
return None
# Convert datetimes to dates in the current time zone.
if generic_view.uses_datetime_field:
if settings.USE_TZ:
result = timezone.localtime(result)
result = result.date()
This error wasn't immediately obvious because the query shouldn't have
been able to return a NULL. Looking at the data revealed the problem: a
small number of records on my test instance had invalid dates in MySQL
(0000-00-00) which were included by the __lt query but subsequently
converted into None by the ORM.
The nature of the bug suggests that the cleanest patch might simply be an
immediate "if not result: return None", perhaps with a warning about
invalid data.
--
Ticket URL: <https://code.djangoproject.com/ticket/22536>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
data
Reporter: acdha | Owner: acdha
Type: Bug | Status: new
Component: Generic views | Version: 1.5
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by aaugustin):
* needs_better_patch: => 0
* stage: Unreviewed => Accepted
* needs_tests: => 0
* needs_docs: => 0
Old description:
> I ran into a non-obvious exception while using the date-based generic
> views. get_previous_day / get_next_day were raising an AttributeError
> ('NoneType' object has no attribute 'astimezone') in the following code
> when the view was called with the earliest item in the system:
>
> try:
> result = getattr(qs[0], date_field)
> except IndexError:
> return None
>
> # Convert datetimes to dates in the current time zone.
> if generic_view.uses_datetime_field:
> if settings.USE_TZ:
> result = timezone.localtime(result)
> result = result.date()
>
> This error wasn't immediately obvious because the query shouldn't have
> been able to return a NULL. Looking at the data revealed the problem: a
> small number of records on my test instance had invalid dates in MySQL
> (0000-00-00) which were included by the __lt query but subsequently
> converted into None by the ORM.
>
> The nature of the bug suggests that the cleanest patch might simply be an
> immediate "if not result: return None", perhaps with a warning about
> invalid data.
New description:
I ran into a non-obvious exception while using the date-based generic
views. get_previous_day / get_next_day were raising an AttributeError
('NoneType' object has no attribute 'astimezone') in the following code
when the view was called with the earliest item in the system:
{{{
try:
result = getattr(qs[0], date_field)
except IndexError:
return None
# Convert datetimes to dates in the current time zone.
if generic_view.uses_datetime_field:
if settings.USE_TZ:
result = timezone.localtime(result)
result = result.date()
}}}
This error wasn't immediately obvious because the query shouldn't have
been able to return a NULL. Looking at the data revealed the problem: a
small number of records on my test instance had invalid dates in MySQL
(0000-00-00) which were included by the __lt query but subsequently
converted into None by the ORM.
The nature of the bug suggests that the cleanest patch might simply be an
immediate "if not result: return None", perhaps with a warning about
invalid data.
--
Comment:
We discussed this ticket on IRC. I believe it's an acceptable solution,
provided there's an adequate comment and a MySQL-specific test. The code
isn't complex, so this change is low risk.
--
Ticket URL: <https://code.djangoproject.com/ticket/22536#comment:1>
Django
data
Reporter: acdha | Owner: acdha
Type: Bug | Status: new
Component: Generic views | Version: 1.5
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by mk):
If we want to be nice there's another case which could be handled here. I
just lost hours to debugging a crash which happened because `result` was a
naive datetime. This was probably the result of a conversion of MySQL to
PostgreSQL done without the help of Django, because (as the documentation
says) Django always created datetime fields with timezones.
A thing which would have helped me would have been an error message along
the lines of "Time zone missing on %(result)s. Your database is broken."
The error message for later reference: `ValueError: astimezone() cannot be
applied to a naive datetime`.
(At least this comment might point someone else in the correct direction.)
--
Ticket URL: <https://code.djangoproject.com/ticket/22536#comment:2>
Django
data
Reporter: Chris Adams | Owner: Johanan
| Oppong Amoateng
Type: Bug | Status: assigned
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Changes (by Johanan Oppong Amoateng):
* has_patch: 0 => 1
* owner: Chris Adams => Johanan Oppong Amoateng
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/22536#comment:3>