[Django] #29142: OuterRef not being treated as an F Expression
Django
-------------------------------------+-------------------------------------
Reporter: | Owner: nobody
michaeljohnbarr |
Type: Bug | Status: new
Component: Database | Version: 2.0
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
[https://docs.djangoproject.com/en/2.0/ref/models/expressions
/#referencing-columns-from-the-outer-queryset According to the
documentation on models.OuterRef]:
"It acts like an F expression except that the check to see if it refers to
a valid field isn’t made until the outer queryset is resolved."
I am experiencing an issue with this using the following example:
{{{
class ExampleModel(models.Model):
date = models.DateField()
value = models.IntegerField()
subquery = ExampleModel.objects.filter(
date__gte=models.OuterRef('date') - timedelta(days=2),
date__lte=models.OuterRef('date') + timedelta(days=2),
value__lte=5
)
queryset = ExampleModel.objects.annotate(
value_is_lte_five=models.Exists(subquery)
)
}}}
The result that I am getting is:
AttributeError: 'ResolvedOuterRef' object has no attribute
'relabeled_clone'
From my understanding, if this is similar to an F Expression, you should
be able to perform the timedelta on the OuterRef.
[https://stackoverflow.com/questions/48230771/is-it-possible-to-do-
arithmetic-operation-on-outerref-expression I also found someone else on
StackOverflow with the same issue].
What I would like to know is if this is a bug (e.g. should it be
supported), or a documentation issue.
--
Ticket URL: <https://code.djangoproject.com/ticket/29142>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 2.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Matthew Schinckel (added)
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/29142#comment:1>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 2.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Matthew Schinckel):
Yeah, I think this is a bug.
I'm not sure where `F` gets it's `relabeled_clone` from, because
`OuterRef` inherits directly from `F`.
--
Ticket URL: <https://code.djangoproject.com/ticket/29142#comment:2>
Django
-------------------------------------+-------------------------------------
Type: Bug | Status: new
Component: Database layer | Version: 2.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Matthew Schinckel):
Ah, is it that `F` gets turned into a `Ref`?
--
Ticket URL: <https://code.djangoproject.com/ticket/29142#comment:3>
Django
-------------------------------------+-------------------------------------
| Schinckel
Type: Bug | Status: assigned
Component: Database layer | Version: 2.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => assigned
* owner: nobody => Matthew Schinckel
--
Ticket URL: <https://code.djangoproject.com/ticket/29142#comment:4>
Django
-------------------------------------+-------------------------------------
| Schinckel
Type: Bug | Status: assigned
Component: Database layer | Version: 2.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Matthew Schinckel):
PR: https://github.com/django/django/pull/9722
--
Ticket URL: <https://code.djangoproject.com/ticket/29142#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: Michael Barr | Owner: Matthew
| Schinckel
Type: Bug | Status: assigned
Component: Database layer | Version: 2.0
(models, ORM) |
Severity: Normal | Resolution:
| checkin
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/29142#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: Michael Barr | Owner: Matthew
| Schinckel
Component: Database layer | Version: 2.0
(models, ORM) |
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"c412926a2e359afb40738d8177c9f3bef80ee04e" c412926a]:
{{{
#!CommitTicketReference repository=""
revision="c412926a2e359afb40738d8177c9f3bef80ee04e"
Fixed #29142 -- Fixed crash when OuterRef is used with an operator.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29142#comment:7>
Django
-------------------------------------+-------------------------------------
Reporter: Michael Barr | Owner: Matthew
| Schinckel
Component: Database layer | Version: 2.0
(models, ORM) |
Severity: Normal | Resolution:
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: closed => new
* resolution: fixed =>
Comment:
Issue still exists in django 2.0.5. Performing arithmetic operation on
OuterRef returns the following.
**'ResolvedOuterRef' object has no attribute 'relabeled_clone'**
--
Ticket URL: <https://code.djangoproject.com/ticket/29142#comment:8>
Django
-------------------------------------+-------------------------------------
Reporter: Michael Barr | Owner: Matthew
| Schinckel
Component: Database layer | Version: 2.0
(models, ORM) |
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => fixed
Comment:
The patch doesn't qualify for a backport to the stable/2.0.x branch based
on our [https://docs.djangoproject.com/en/dev/internals/release-process
/#supported-versions supported versions policy].
--
Ticket URL: <https://code.djangoproject.com/ticket/29142#comment:9>
Django
-------------------------------------+-------------------------------------
Reporter: Michael Barr | Owner: Matthew
| Schinckel
Type: Bug | Status: closed
Component: Database layer | Version: 2.0
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Brenton Partridge):
For anyone needing a backport, it's relatively easy to patch in your
userland code.
{{{
# Patch from
https://github.com/django/django/commit/c412926a2e359afb40738d8177c9f3bef80ee04e
# https://code.djangoproject.com/ticket/29142
django.db.models.F.relabeled_clone = lambda self, relabels: self
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29142#comment:10>
Django
-------------------------------------+-------------------------------------
Reporter: Michael Barr | Owner: Matthew
| Schinckel
Type: Bug | Status: closed
Component: Database layer | Version: 2.0
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Dmitry Mugtasimov):
The backport does not seem to work for me
{{{
class CustomResolvedOuterRef(ResolvedOuterRef):
contains_aggregate = False
def relabeled_clone(self, relabels):
return self
class CustomOuterRef(OuterRef):
def resolve_expression(self, query=None, allow_joins=True, reuse=None,
summarize=False, for_save=False,
simple_col=False):
if isinstance(self.name, self.__class__):
return self.name
return CustomResolvedOuterRef(self.name)
queryset.annotate(
title_vector=SearchVector(
'title', config=SIMPLE_PG_SEARCH_CONFIGURATION)
).annotate(
has_role=Exists(
JobTitle.objects.annotate(
title_vector=CustomOuterRef('title_vector')
).filter(
title_vector=SearchQuery(
F('name'),
config=SIMPLE_PG_SEARCH_CONFIGURATION)
)
)
)
}}}
get this:
{{{
Environment:
Request Method: GET
Request URL: http://127.0.0.1:8000/admin/jobs/job/
Django Version: 2.2.2
Python Version: 3.7.3
Installed Applications:
['core',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.sites',
'django.contrib.postgres',
'allauth',
'allauth.account',
'drf_yasg',
'rest_auth',
'rest_auth.registration',
'rest_framework',
'rest_framework.authtoken',
'colorful',
'gm2m',
'explorer',
'allauth.socialaccount',
'accounts',
'questions',
'colleges',
'geo',
'jobs',
'advices',
'referral']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware']
Traceback:
File "/usr/local/lib/python3.7/site-
packages/django/core/handlers/exception.py" in inner
34. response = get_response(request)
File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py"
in _get_response
115. response = self.process_exception_by_middleware(e,
request)
File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py"
in _get_response
113. response = wrapped_callback(request,
*callback_args, **callback_kwargs)
File "/usr/local/lib/python3.7/contextlib.py" in inner
74. return func(*args, **kwds)
File "/usr/local/lib/python3.7/site-
packages/django/contrib/admin/options.py" in wrapper
606. return self.admin_site.admin_view(view)(*args,
**kwargs)
File "/usr/local/lib/python3.7/site-packages/django/utils/decorators.py"
in _wrapped_view
142. response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python3.7/site-
packages/django/views/decorators/cache.py" in _wrapped_view_func
44. response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python3.7/site-
packages/django/contrib/admin/sites.py" in inner
223. return view(request, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/django/utils/decorators.py"
in _wrapper
45. return bound_method(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/django/utils/decorators.py"
in _wrapped_view
142. response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python3.7/site-
packages/django/contrib/admin/options.py" in changelist_view
1672. cl = self.get_changelist_instance(request)
File "/usr/local/lib/python3.7/site-
packages/django/contrib/admin/options.py" in get_changelist_instance
744. sortable_by,
File "/usr/local/lib/python3.7/site-
packages/django/contrib/admin/views/main.py" in __init__
45. self.root_queryset = model_admin.get_queryset(request)
File "/code/jobs/admin.py" in get_queryset
76. return queryset.annotate_with_has_role()
File "/code/jobs/models/job.py" in annotate_with_has_role
88. F('name'),
config=SIMPLE_PG_SEARCH_CONFIGURATION)
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py" in
filter
892. return self._filter_or_exclude(False, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py" in
_filter_or_exclude
910. clone.query.add_q(Q(*args, **kwargs))
File "/usr/local/lib/python3.7/site-
packages/django/db/models/sql/query.py" in add_q
1290. clause, _ = self._add_q(q_object, self.used_aliases)
File "/usr/local/lib/python3.7/site-
packages/django/db/models/sql/query.py" in _add_q
1318. split_subq=split_subq, simple_col=simple_col,
File "/usr/local/lib/python3.7/site-
packages/django/db/models/sql/query.py" in build_filter
1207. condition = self.build_lookup(lookups,
reffed_expression, value)
File "/usr/local/lib/python3.7/site-
packages/django/db/models/sql/query.py" in build_lookup
1104. lookup_class = lhs.get_lookup(lookup_name)
Exception Type: AttributeError at /admin/jobs/job/
Exception Value: 'CustomResolvedOuterRef' object has no attribute
'get_lookup'
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29142#comment:11>
Django
-------------------------------------+-------------------------------------
Reporter: Michael Barr | Owner: Matthew
| Schinckel
Type: Bug | Status: closed
Component: Database layer | Version: 2.0
(models, ORM) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Matthew Schinckel):
Replying to [comment:11 Dmitry Mugtasimov]:
> The backport does not seem to work for me
Does the backport work if you just use the standard OuterRef class? I
think perhaps it's something else, as this issue was about the missing
'relabeled_clone', not a missing 'get_lookup'.
Perhaps this is a different issue entirely?
--
Ticket URL: <https://code.djangoproject.com/ticket/29142#comment:12>