RuntimeWarning "DateTimeField received a naive datetime" with TIMESTAMP MySQL field

[Sencha]

I'm running into an issue with the use of the MySQL `TIMESTAMP` field compared to `DATETIME` field.

Django is run with timezone support and default timezone of `UTC`.

I have 2 fields in my MySQL table, one is type `DATETIME` and the other is `TIMESTAMP`. Both are set in the model as a `DateTimeField`. The data is already populated in this example.

When the data is loaded into the model by Django, I can see in the `DateTimeField`'s `get_prep_value` method that the `DATETIME` field value come through as:

datetime.datetime(2012, 11, 16, 16, 9, 25, tzinfo=<UTC>)

and the `TIMESTAMP` comes through as:

datetime.datetime(2012, 11, 16, 15, 51, 32)

So the value from the timestamp MySQL field is coming through not initiated with the default timezone like the datetime field is.

This becomes a problem with re-saving the model, even after making no changes. The warning is being thrown up:

File "/Users/me/.virtualenvs/momento/lib/python2.7/site-packages/django/db/models/fields/__init__.py", line 809, in get_prep_value RuntimeWarning)

RuntimeWarning: DateTimeField received a naive datetime (2012-11-16 15:51:32) while time zone support is active.

Is this a bug? Any recommendations how to overcome this?

[Timster]

It's well-documented. Check out the Django documentation page on time zones:

https://docs.djangoproject.com/en/1.4/topics/i18n/timezones/

When time zone support is enabled, Django uses time-zone-aware datetime objects. If your code creates datetime objects, they should be aware too. In this mode, the example above becomes:

import datetime
from django.utils.timezone import utc

now = datetime.datetime.utcnow().replace(tzinfo=utc)

[Sencha]

Yes I'm aware of this, however I'm not generating the datetime objects, it's all coming from Django (hence me wondering whether or not it's a bug or not). All I need to do to replicate the bug is get the object from the db and then save is straight away; then warning appears.

Django abstracts away the difference between MySQL's datetime field and timestamp field, so it should work!

[Tom Evans]

On Fri, Nov 16, 2012 at 5:50 PM, Sencha <mic...@d3i.com> wrote:
> Yes I'm aware of this, however I'm not generating the datetime objects, it's
> all coming from Django (hence me wondering whether or not it's a bug or
> not). All I need to do to replicate the bug is get the object from the db
> and then save is straight away; then warning appears.
>
> Django abstracts away the difference between MySQL's datetime field and
> timestamp field, so it should work!
>

Oh really? How have you determined that django is producing the naive
datetime? This is a loaded question, since I know django does not
produce naive datetimes when USE_TZ=True! See the end of the email for
the easiest way to determine where the naive datetime is actually
coming from.

All the purported solutions suggested in this thread have missed that
TZ support in django adds some nifty shortcuts, prime being
timezone.now():

https://docs.djangoproject.com/en/1.4/ref/utils/#django.utils.timezone.now

This will always DTRT, producing naive datetimes if USE_TZ is false,
and aware datetimes otherwise. You should replace everywhere you call
'datetime.now()' with 'timezone.now()', so that all your datetimes are
TZ aware.

I would thoroughly recommend re-reading the TZ docs. There is an
excellent migration guide, FAQs and troubleshooting hints (one of
which covers this exact issue):

https://docs.djangoproject.com/en/1.4/topics/i18n/timezones/

The easiest way to determine where the naive time is coming from is to
turn the Warning into a RuntimeWarning, so that as soon as it happens
Django will explode and you will get a lovely traceback showing
precisely where it was created. Add this to your settings.py:

import warnings
warnings.filterwarnings(
'error', r"DateTimeField received a naive datetime",
RuntimeWarning, r'django\.db\.models\.fields')

Cheers

Tom

[Tom Evans]

On Mon, Nov 19, 2012 at 2:29 PM, Tom Evans <teva...@googlemail.com> wrote:
> On Fri, Nov 16, 2012 at 5:50 PM, Sencha <mic...@d3i.com> wrote:
>> Yes I'm aware of this, however I'm not generating the datetime objects, it's
>> all coming from Django (hence me wondering whether or not it's a bug or
>> not). All I need to do to replicate the bug is get the object from the db
>> and then save is straight away; then warning appears.
>>
>> Django abstracts away the difference between MySQL's datetime field and
>> timestamp field, so it should work!
>>
>
> Oh really? How have you determined that django is producing the naive
> datetime? This is a loaded question, since I know django does not
> produce naive datetimes when USE_TZ=True! See the end of the email for
> the easiest way to determine where the naive datetime is actually
> coming from.

Write in haste, repent at leisure :/

I'm almost certainly wrong here. The issue is that (apart from me not
being able to read) is in two parts. Firstly, TIMESTAMP field in MySQL
has no TZ info associated with it - MySQL will always convert a
TIMESTAMP field to UTC, and return it as UTC, and secondly, Django has
no understanding of a TIMESTAMP field, and so doesn't have semantics
to cope with it being returned in UTC.

You can't store an aware datetime in a TIMESTAMP, because it will
always lose it's tzinfo, and will be UTC when extracted, so it isn't
suitable for use with TZ aware datetimes. If the TZ itself isn't
important, but keeping it as a TIMESTAMP is (presumably for auto
updating?), I would subclass DateTimeField so that it 'fixes up' non
aware datetimes as extracted from the DB to UTC.

Cheers

Tom