Re: [Django] #9596: Comparing a DateTimeField to a date is too hard

Django

unread,

Jun 8, 2011, 4:26:09 PM6/8/11

to django-...@googlegroups.com

#9596: Comparing a DateTimeField to a date is too hard
-------------------------------------+-------------------------------------
Reporter: django@… | Owner: nobody
Type: New | Status: new
feature | Component: Database layer
Milestone: | (models, ORM)
Version: SVN | Severity: Normal
Resolution: | Keywords: lookup_type date
Triage Stage: Design | datetimefield compare comparison
decision needed | query_term field lookup
Needs documentation: 0 | Has patch: 1
Patch needs improvement: 1 | Needs tests: 0
UI/UX: 0 | Easy pickings: 0
-------------------------------------+-------------------------------------
Changes (by JMagnusson):

* cc: JMagnusson (added)
* ui_ux: => 0
* easy: => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:20>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,

Jun 9, 2011, 9:11:38 AM6/9/11

to django-...@googlegroups.com

#9596: Comparing a DateTimeField to a date is too hard
-------------------------------------+-------------------------------------
Reporter: django@… | Owner: nobody
Type: New | Status: new
feature | Component: Database layer
Milestone: | (models, ORM)
Version: SVN | Severity: Normal
Resolution: | Keywords: lookup_type date
Triage Stage: Design | datetimefield compare comparison
decision needed | query_term field lookup
Needs documentation: 0 | Has patch: 1
Patch needs improvement: 1 | Needs tests: 0
UI/UX: 0 | Easy pickings: 0
-------------------------------------+-------------------------------------

Comment (by UloPe):

Related: #16187 Refactor of the lookup system

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:21>

Django

unread,

Aug 26, 2011, 9:34:34 PM8/26/11

to django-...@googlegroups.com

#9596: Comparing a DateTimeField to a date is too hard
-------------------------------------+-------------------------------------
Reporter: django@… | Owner: nobody
Type: New | Status: new
feature | Component: Database layer
Milestone: | (models, ORM)
Version: SVN | Severity: Normal
Resolution: | Keywords: lookup_type date
Triage Stage: Design | datetimefield compare comparison
decision needed | query_term field lookup
Needs documentation: 0 | Has patch: 1
Patch needs improvement: 1 | Needs tests: 0
UI/UX: 0 | Easy pickings: 0
-------------------------------------+-------------------------------------

Changes (by curtis@…):

* cc: curtis@… (added)

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:22>

Django

unread,

Jan 22, 2012, 5:46:49 AM1/22/12

to django-...@googlegroups.com

#9596: Comparing a DateTimeField to a date is too hard
-------------------------------------+-------------------------------------
Reporter: django@… | Owner: nobody

* stage: Design decision needed => Accepted

Comment:

This ticket is in DDN because the current technique fails if the `__date`
lookup is used on a `DateField`.

There is exactly the same problem with the `year` lookup; its
implementation depends on the field it's applied to:
{{{
# django/db/models/fields/__init__.py
elif lookup_type == 'year':
if self.get_internal_type() == 'DateField':
return
connection.ops.year_lookup_bounds_for_date_field(value)
else:
return connection.ops.year_lookup_bounds(value)
}}}
The implementation of `year_lookup_bounds` and
`year_lookup_bounds_for_date_field` is database dependent. This allows
setting the upper bound to `YYYY-12-31` instead of `YYYY-12-31
23:59:59.999999` for `DateField`s under Oracle.

I think we could mirror this solution for the `__date` lookup. This means
implementing `connection.ops.date_lookup_bounds(value)`
`connection.ops.date_lookup_bounds_for_date_field(value)` (in a backend
dependent fashion if necessary).

It would be even better to convert the lookup to an exact match for
`DateField`, but I don't believe that's doable with the current
implementation.

Note that the tests in the patch need to be rewritten as unittests.

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:23>

Django

unread,

Jan 18, 2014, 4:54:55 AM1/18/14

to django-...@googlegroups.com

#9596: Comparing a DateTimeField to a date is too hard
-------------------------------------+-------------------------------------
Reporter: django@… | Owner: nobody
Type: New feature | Status: new

Component: Database layer | Version: master

Comment (by akaariai):

Now that #16187 is fixed this should be relatively easy to implement:

{{{
class DateTransform(Transform):
lookup_name = 'as_date' # or just date, implementer's choice
output_type = DateField()

def as_sql(self, qn, connection):
lhs, lhs_params = qn.compile(self.lhs)
# Biggest problem is the following line - implement date cast
template
# for different backends. MySQL seems to be date(%s), PostgreSQL
%s::date.
date_cast_sql = connection.ops.date_cast_template
return date_cast_template % lhs, lhs_params
DateTimeField.register_lookup(DateTransform)
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:24>

Django

unread,

Jun 7, 2014, 11:25:49 AM6/7/14

to django-...@googlegroups.com

#9596: Comparing a DateTimeField to a date is too hard
-------------------------------------+-------------------------------------
Reporter: django@… | Owner: nobody
Type: New feature | Status: new
Component: Database layer | Version: master
(models, ORM) | Resolution:
Severity: Normal | Triage Stage: Accepted
Keywords: lookup_type date | Needs documentation: 0
datetimefield compare comparison | Patch needs improvement: 1
query_term field lookup | UI/UX: 0
Has patch: 1 |
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------

Comment (by aaugustin):

Still, I would support adding proper support for this feature in Django,
if only to make sure time zones are handled correctly when USE_TZ is True.
It's very easy to mess them up!

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:25>

Django

unread,

Aug 31, 2014, 4:15:57 PM8/31/14

to django-...@googlegroups.com

#9596: Comparing a DateTimeField to a date is too hard
-------------------------------------+-------------------------------------

Reporter: django@… | Owner: aaugustin
Type: New feature | Status: assigned

Changes (by aaugustin):

* owner: nobody => aaugustin
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:26>

Django

unread,

Mar 7, 2015, 6:28:17 PM3/7/15

to django-...@googlegroups.com

#9596: Comparing a DateTimeField to a date is too hard
-------------------------------------+-------------------------------------
Reporter: django@… | Owner: aaugustin
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |

Severity: Normal | Resolution:
Keywords: lookup_type date | Triage Stage: Accepted
datetimefield compare comparison |
query_term field lookup |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by jdufresne):

* cc: jon.dufresne@… (added)

Comment:

> Now that #16187 is fixed this should be relatively easy to implement:

> ...
> DateTimeField.register_lookup(DateTransform)

I've started down the path suggested above with commit:
https://github.com/jdufresne/django/commit/5bb3f1881982bc1248d5891a887725eeff7841bd

However, I'm not sure where this code should live if added to Django
generally. What is the correct area of the code to call
`register_lookup()`?

From the [https://docs.djangoproject.com/en/dev/howto/custom-lookups/
docs]:

> We can now use `foo__ne` for any field foo. You will need to ensure that
this registration happens before you try to create any querysets using it.
You could place the implementation in a `models.py` file, or register the
lookup in the `ready()` method of an `AppConfig`.

As far as I can tell, these options don't really apply to *built-in*
transforms.

I notice there is a `default_lookups` in `django/db/models/lookups.py`,
but I'm not sure how to make this work for field specific transforms,
instead of baisc lookups.

I tried putting `register_lookup()` inside `django/db/fields/__init__.py`,
however this lead to issues with settings.

```
django.core.exceptions.ImproperlyConfigured: Requested setting
DEFAULT_INDEX_TABLESPACE, but settings are not configured. You must either
define the environment variable DJANGO_SETTINGS_MODULE or call
settings.configure() before accessing settings.
```

Any thoughts?

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:27>

Django

unread,

Mar 8, 2015, 5:47:35 AM3/8/15

to django-...@googlegroups.com

#9596: Comparing a DateTimeField to a date is too hard
-------------------------------------+-------------------------------------
Reporter: django@… | Owner:

Type: New feature | Status: new

Changes (by aaugustin):

* owner: aaugustin =>
* status: assigned => new

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:28>

Django

unread,

Mar 8, 2015, 5:56:22 AM3/8/15

to django-...@googlegroups.com

#9596: Comparing a DateTimeField to a date is too hard
-------------------------------------+-------------------------------------
Reporter: django@… | Owner:
Type: New feature | Status: new
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution:
Keywords: lookup_type date | Triage Stage: Accepted
datetimefield compare comparison |
query_term field lookup |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by aaugustin):

The code looks correct when USE_TZ = False. You also need to handle the
more complicated case when USE_TZ = True.

See the current implementation `datetime_trunc_sql` for an idea of what's
needed.

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:29>

Django

unread,

Mar 8, 2015, 3:25:12 PM3/8/15

to django-...@googlegroups.com

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by jdufresne):

* needs_better_patch: 1 => 0

Comment:

Created a PR that is ready for review.

https://github.com/django/django/pull/4281

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:30>

Django

unread,

Mar 28, 2015, 9:17:04 AM3/28/15

to django-...@googlegroups.com

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by timgraham):

* needs_better_patch: 0 => 1

Comment:

Some test failures on Oracle need to be fixed.

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:31>

Django

unread,

Mar 31, 2015, 1:29:10 AM3/31/15

to django-...@googlegroups.com

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by jdufresne):

* needs_better_patch: 1 => 0

Comment:

Test fixes and other feedback has been incorporated into the latest
version of the PR. Additional feedback is welcome.

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:32>

Django

unread,

Apr 15, 2015, 11:08:12 AM4/15/15

to django-...@googlegroups.com

#9596: Add a __date lookup for DateTimeField

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by timgraham):

* needs_better_patch: 0 => 1

Comment:

Left some mostly cosmetic comments, and tests are still failing on Oracle
(but passing on my local version of Oracle, so might need some expertise
about Oracle version differences to resolve that).

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:33>

Django

unread,

Jun 1, 2015, 9:12:43 PM6/1/15

to django-...@googlegroups.com

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by jdufresne):

* needs_better_patch: 1 => 0

Comment:

Updated patch for Oracle based on feedback. All tests are now passing.

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:34>

Django

unread,

Jun 2, 2015, 8:49:46 AM6/2/15

to django-...@googlegroups.com

#9596: Add a __date lookup for DateTimeField
-------------------------------------+-------------------------------------

Reporter: django@… | Owner: Tim
| Graham <timograham@…>
Type: New feature | Status: closed

Component: Database layer | Version: master
(models, ORM) |

Severity: Normal | Resolution: fixed

Changes (by Tim Graham <timograham@…>):

* owner: => Tim Graham <timograham@…>
* status: new => closed
* resolution: => fixed

Comment:

In [changeset:"44f3ee77166bd5c0e8a4604f2d96015268dce100" 44f3ee77]:
{{{
#!CommitTicketReference repository=""
revision="44f3ee77166bd5c0e8a4604f2d96015268dce100"
Fixed #9596 -- Added date transform for DateTimeField.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/9596#comment:35>

Django

unread,

Jun 2, 2015, 9:51:47 AM6/2/15

to django-...@googlegroups.com

#9596: Add a __date lookup for DateTimeField
-------------------------------------+-------------------------------------
Reporter: django@… | Owner: Tim
| Graham <timograham@…>
Type: New feature | Status: closed
Component: Database layer | Version: master
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: lookup_type date | Triage Stage: Accepted
datetimefield compare comparison |
query_term field lookup |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------