[Django] #18766: Models with datetime fields fail to save when date is below 1900 and USE_TZ is True
Django
is True
-------------------------------+--------------------------
Reporter: void | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: master
Severity: Normal | Keywords: timezone, tz
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------
I have model with datetime field in it. When I save this model in admin
site and the filled date is below 1900, I am getting the following
traceback:
{{{
File "/Users/voidus/Documents/workspace/django-
trunk/django/contrib/admin/options.py", line 709, in save_model
obj.save()
File "/Users/voidus/Documents/workspace/django-
trunk/django/db/models/base.py", line 524, in save
force_update=force_update, update_fields=update_fields)
File "/Users/voidus/Documents/workspace/django-
trunk/django/db/models/base.py", line 595, in save_base
rows = manager.using(using).filter(pk=pk_val)._update(values)
File "/Users/voidus/Documents/workspace/django-
trunk/django/db/models/query.py", line 573, in _update
return query.get_compiler(self.db).execute_sql(None)
File "/Users/voidus/Documents/workspace/django-
trunk/django/db/models/sql/compiler.py", line 991, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/Users/voidus/Documents/workspace/django-
trunk/django/db/models/sql/compiler.py", line 813, in execute_sql
sql, params = self.as_sql()
File "/Users/voidus/Documents/workspace/django-
trunk/django/db/models/sql/compiler.py", line 956, in as_sql
val = field.get_db_prep_save(val, connection=self.connection)
File "/Users/voidus/Documents/workspace/django-
trunk/django/db/models/fields/__init__.py", line 303, in get_db_prep_save
prepared=False)
File "/Users/voidus/Documents/workspace/django-
trunk/django/db/models/fields/__init__.py", line 834, in get_db_prep_value
value = self.get_prep_value(value)
File "/Users/voidus/Documents/workspace/django-
trunk/django/db/models/fields/__init__.py", line 820, in get_prep_value
if value is not None and settings.USE_TZ and timezone.is_naive(value):
File "/Users/voidus/Documents/workspace/django-
trunk/django/utils/timezone.py", line 271, in is_naive
return value.tzinfo is None or value.tzinfo.utcoffset(value) is None
File "/Users/voidus/Documents/workspace/django-
trunk/django/utils/timezone.py", line 71, in utcoffset
if self._isdst(dt):
File "/Users/voidus/Documents/workspace/django-
trunk/django/utils/timezone.py", line 89, in _isdst
stamp = _time.mktime(tt)
ValueError: year out of range
}}}
I think that is because `utcoffset` and some other functions use `_isdst`
function, which calls `time.mktime` and it have some limitation on the
date range it can handle.
`time` module can handle dates in the more narrow range (1969-2038) then
`datetime` module (1-9999).
--
Ticket URL: <https://code.djangoproject.com/ticket/18766>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
is True
Type: Bug | Status: new
Component: Uncategorized | Version: master
Needs tests: 0 | Patch needs improvement: 0
Changes (by void):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
As [[http://en.wikipedia.org/wiki/Daylight_saving_time|wikipedia article]]
says, DST was not used before at least 1916. So, maybe simple check like
{{{
if dt.year < 1916:
return False
}}}
will work
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:1>
Django
is True
Reporter: void | Owner: aaugustin
Component: Uncategorized | Version: master
Changes (by aaugustin):
* owner: nobody => aaugustin
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:2>
Django
is True
Reporter: void | Owner: aaugustin
Component: Uncategorized | Version: master
Comment (by void):
The other corner case is dates after 2038. While simple check for `< 1916`
will work for dates before 1900, as there were no DST at that time, after
2038 we will probably still have DST. 2038 is not near, but I can imagine
a calendar application with events in future.
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:3>
Django
is True
Reporter: void | Owner: aaugustin
Component: Uncategorized | Version: master
Comment (by aaugustin):
Indeed, this is a limitation of `django.utils.tzinfo.LocalTimezone`.
Could you try installing `pytz` (as strongly recommeded by the docs)? If
that fixes the problem, I'd lean towards a documentation fix rather than
extending the huge hack that is `LocalTimezone`.
Note that dates after 2038 are handled by returning incorrect data rather
than crashing. I deeply dislike that, but I can't change it easily, for
backwards-compatibility reasons.
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:4>
Django
is True
Reporter: void | Owner: aaugustin
Component: Uncategorized | Version: master
Comment (by aaugustin):
See also #17262.
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:5>
Django
is True
Reporter: void | Owner: aaugustin
Component: Uncategorized | Version: master
Severity: Normal | Resolution: needsinfo
Changes (by aaugustin):
* status: new => closed
* resolution: => needsinfo
Comment:
Waiting to know if installing pytz resolves the problem...
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:6>
Django
is True
Reporter: void | Owner: aaugustin
Type: Bug | Status: closed
Severity: Normal | Resolution: needsinfo
Comment (by anonymous):
Yes, installing pytz resolves the problem.
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:7>
Django
is True
Reporter: void | Owner: aaugustin
Type: Bug | Status: closed
Severity: Normal | Resolution: needsinfo
Comment (by void):
Sorry, that was me, was logged out somehow.
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:8>
Django
is True
Reporter: void | Owner: aaugustin
Type: | Status: reopened
Cleanup/optimization | Version: master
Component: Documentation | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: timezone, tz | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by aaugustin):
* status: closed => reopened
* component: Uncategorized => Documentation
* type: Bug => Cleanup/optimization
* resolution: needsinfo =>
* stage: Unreviewed => Accepted
Comment:
I'm accepting the ticket primarily as a documentation issue. This is
related to #17262. I still have to decide how to deal with this.
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:9>
Django
is True
Reporter: void | Owner: aaugustin
Type: | Status: reopened
Cleanup/optimization | Version: master
Component: Documentation | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: timezone, tz | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by void):
Maybe it would be OK to require pytz installed if user set USE_I18N = True
in settings.py?
Or maybe we can even set a hard dependency? I think it's now commonly
consented that django should make more use of pip - at least localflavor
will be separated, etc. So, maybe pytz would be the first thing to
require?
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:10>
Django
is True
Reporter: void | Owner: aaugustin
Type: | Status: reopened
Cleanup/optimization | Version: master
Component: Documentation | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: timezone, tz | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by aaugustin):
`USE_TZ = True` is in the default settings files (for a good reason) and
we want to keep Django usable without having to learn pip / virtualenv
first (also for a good reason). I went to great lengths to ensure that
pytz wouldn't be a mandatory.
I think it's more promising to raise a more explicit error in this case,
ie. "value outside of supported range, please install pytz".
I
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:11>
Django
is True
Reporter: void | Owner: aaugustin
Type: | Status: reopened
Cleanup/optimization | Version: master
Component: Documentation | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: timezone, tz | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by void):
Yeah, it may be a good solution. But may this behaviour be considered as a
regression? Imagine, you have some values outside of supported range in
database, then upgrade django one version upper, set USE_TZ=True, as docs
insist. And live being happy until someday you see in logs such message
about pytz.
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:12>
Django
is True
Reporter: void | Owner: aaugustin
Cleanup/optimization | Version: master
Component: Documentation | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: timezone, tz | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by dracos):
Just to back up void's comment, I have just upgraded a site from Django
1.1 to 1.5. One of the few things that only bit me once it went live (and
thanks as ever for the good documentation on upgrades and consistency :) )
was that I have some very old datetimes in the live database (back to the
17th century) that were working fine, that then started to throw
"ValueError: year out of range" 500s due to this issue. It does indeed
resolve if pytz is installed (or if USE_TZ is set back to False, which
I've done for now).
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:14>
Django
is True
Reporter: void | Owner: aaugustin
Cleanup/optimization | Version: master
Component: Documentation | Resolution: fixed
Severity: Normal | Triage Stage: Accepted
Keywords: timezone, tz | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"ded11aa620e5b38c78ff92e691a459d1977a2539"]:
{{{
#!CommitTicketReference repository=""
revision="ded11aa620e5b38c78ff92e691a459d1977a2539"
Fixed #18766 -- Pointed to pytz when LocalTimezone fails.
Thanks void for the report.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:15>
Django
is True
Reporter: void | Owner: aaugustin
Type: | Status: closed
Cleanup/optimization | Version: master
Component: Documentation | Resolution: fixed
Severity: Normal | Triage Stage: Accepted
Keywords: timezone, tz | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by Aymeric Augustin <aymeric.augustin@…>):
In [changeset:"7c31e195db963ad1ebb28265e721097000fa84d1"]:
{{{
#!CommitTicketReference repository=""
revision="7c31e195db963ad1ebb28265e721097000fa84d1"
[1.6.x] Fixed #18766 -- Pointed to pytz when LocalTimezone fails.
Thanks void for the report.
Backport of ded11aa6 from master.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:16>
Django
is True
Reporter: void | Owner: aaugustin
Type: | Status: closed
Cleanup/optimization | Version: master
Component: Documentation | Resolution: fixed
Severity: Normal | Triage Stage: Accepted
Keywords: timezone, tz | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by Aymeric Augustin <aymeric.augustin@…>):
In [changeset:"1a1e14786aa35b4d80b46be747b5128800cf0f97"]:
{{{
#!CommitTicketReference repository=""
revision="1a1e14786aa35b4d80b46be747b5128800cf0f97"
Hardened the test introduced in ded11aa6. Refs #18766.
Inputs acceptable to time.mktime are platform-dependent.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:17>
Django
is True
Reporter: void | Owner: aaugustin
Type: | Status: closed
Cleanup/optimization | Version: master
Component: Documentation | Resolution: fixed
Severity: Normal | Triage Stage: Accepted
Keywords: timezone, tz | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by Aymeric Augustin <aymeric.augustin@…>):
In [changeset:"0035a0ce2ec243ff9255157117c20caf41a61284"]:
{{{
#!CommitTicketReference repository=""
revision="0035a0ce2ec243ff9255157117c20caf41a61284"
[1.6.x] Hardened the test introduced in ded11aa6. Refs #18766.
Inputs acceptable to time.mktime are platform-dependent.
Backport of 1a1e1478 from master.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:18>
Django
is True
Reporter: void | Owner: aaugustin
Type: | Status: closed
Cleanup/optimization | Version: master
Component: Documentation | Resolution: fixed
Severity: Normal | Triage Stage: Accepted
Keywords: timezone, tz | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by Aymeric Augustin <aymeric.augustin@…>):
In [changeset:"c687bf0620d67e0579302515904ffe169581d036"]:
{{{
#!CommitTicketReference repository=""
revision="c687bf0620d67e0579302515904ffe169581d036"
Further hardening. Refs #18766.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:19>
Django
is True
Reporter: void | Owner: aaugustin
Type: | Status: closed
Cleanup/optimization | Version: master
Component: Documentation | Resolution: fixed
Severity: Normal | Triage Stage: Accepted
Keywords: timezone, tz | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by Aymeric Augustin <aymeric.augustin@…>):
In [changeset:"b085e7c3033d49e4d7525bacb1f34340730918a8"]:
{{{
#!CommitTicketReference repository=""
revision="b085e7c3033d49e4d7525bacb1f34340730918a8"
[1.6.x] Further hardening. Refs #18766.
Backport of c687bf0 from master.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/18766#comment:20>