[Django] #34015: "Related Field got invalid lookup: startswith" on CharField ForeignKey
Django
-------------------------------------+-------------------------------------
Reporter: Thomas | Owner: nobody
Type: Bug | Status: new
Component: Database | Version: 4.1
layer (models, ORM) |
Severity: Normal | Keywords: ORM lookup
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Hello,
I have a model, let's call it `Parent`, with a field called `object_id`. I
have another model, let's call it `Child`, which has a `ForeignKey` field
called `parent_object[_id]` pointing to `Parent.object_id`. I need to do a
lookup on `Child` where the FK starts with a certain character (it's a
normalized value so, in the context of my app, it makes sense... also, I
didn't design this schema and changing it is not a possibility ATM).
The problem is that if I do:
{{{#!python
qs = Child.objects.filter(parent_object_id__startswith='c')
}}}
I get:
{{{
django.core.exceptions.FieldError: Related Field got invalid lookup:
startswith
}}}
The only way I could make it work is:
{{{#!python
qs = Child.objects.filter(parent_object__object_id__startswith='c')
}}}
but it forces a join between the table and the view and that's a no-no in
my case (way too costly).
Here's the MCVE (tested on Python 3.9 + Django 4.0.7 and Python 3.10 +
Django 4.1.1):
{{{
#!python
import django
django.setup()
from django.db import models
class Parent(models.Model):
class Meta:
app_label = 'test'
object_id = models.CharField('Object ID', max_length=20, unique=True)
class Child(models.Model):
class Meta:
app_label = 'test'
parent_object = models.ForeignKey(
Parent, to_field='object_id', related_name='%(class)s_set',
on_delete=models.CASCADE
)
if __name__ == '__main__':
qs = Child.objects.filter(parent_object_id__startswith='c') # fails
with `FieldError: Related Field got invalid lookup: startswith`
qs = Child.objects.filter(parent_object__object_id__startswith='c') #
works but forces a costly join
}}}
And the error:
{{{
Traceback (most recent call last):
File "/opt/src/_sandbox/django_bug.py", line 23, in <module>
qs = Child.objects.filter(media_object_id__startswith='c')
File "/opt/venv/lib/python3.9/site-
packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/opt/venv/lib/python3.9/site-packages/django/db/models/query.py",
line 1071, in filter
return self._filter_or_exclude(False, args, kwargs)
File "/opt/venv/lib/python3.9/site-packages/django/db/models/query.py",
line 1089, in _filter_or_exclude
clone._filter_or_exclude_inplace(negate, args, kwargs)
File "/opt/venv/lib/python3.9/site-packages/django/db/models/query.py",
line 1096, in _filter_or_exclude_inplace
self._query.add_q(Q(*args, **kwargs))
File "/opt/venv/lib/python3.9/site-
packages/django/db/models/sql/query.py", line 1502, in add_q
clause, _ = self._add_q(q_object, self.used_aliases)
File "/opt/venv/lib/python3.9/site-
packages/django/db/models/sql/query.py", line 1532, in _add_q
child_clause, needed_inner = self.build_filter(
File "/opt/venv/lib/python3.9/site-
packages/django/db/models/sql/query.py", line 1448, in build_filter
condition = self.build_lookup(lookups, col, value)
File "/opt/venv/lib/python3.9/site-
packages/django/db/models/sql/query.py", line 1262, in build_lookup
raise FieldError(
django.core.exceptions.FieldError: Related Field got invalid lookup:
startswith
}}}
Thanks for your help,
Regards,
--
Ticket URL: <https://code.djangoproject.com/ticket/34015>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Reporter: Thomas | Owner: nobody
Type: Bug | Status: new
(models, ORM) |
Severity: Normal | Resolution:
Keywords: ORM lookup | Triage Stage:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
New description:
Hello,
File "/opt/src/orm_test.py", line 26, in <module>
qs = Child.objects.filter(parent_object_id__startswith='c')
File "/opt/src/venv/lib/python3.10/site-
packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/opt/src/venv/lib/python3.10/site-
packages/django/db/models/query.py", line 1420, in filter
return self._filter_or_exclude(False, args, kwargs)
File "/opt/src/venv/lib/python3.10/site-
packages/django/db/models/query.py", line 1438, in _filter_or_exclude
clone._filter_or_exclude_inplace(negate, args, kwargs)
File "/opt/src/venv/lib/python3.10/site-
packages/django/db/models/query.py", line 1445, in
_filter_or_exclude_inplace
self._query.add_q(Q(*args, **kwargs))
File "/opt/src/venv/lib/python3.10/site-
packages/django/db/models/sql/query.py", line 1532, in add_q
clause, _ = self._add_q(q_object, self.used_aliases)
File "/opt/src/venv/lib/python3.10/site-
packages/django/db/models/sql/query.py", line 1562, in _add_q
child_clause, needed_inner = self.build_filter(
File "/opt/src/venv/lib/python3.10/site-
packages/django/db/models/sql/query.py", line 1478, in build_filter
condition = self.build_lookup(lookups, col, value)
File "/opt/src/venv/lib/python3.10/site-
packages/django/db/models/sql/query.py", line 1292, in build_lookup
raise FieldError(
django.core.exceptions.FieldError: Related Field got invalid lookup:
startswith
}}}
Thanks for your help,
Regards,
--
--
Ticket URL: <https://code.djangoproject.com/ticket/34015#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: Thomas | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: ORM lookup | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Thomas (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/34015#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: Thomas | Owner: nobody
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: ORM lookup | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* type: Bug => New feature
Comment:
Thanks for the report. Django 4.2
(cd1afd553f9c175ebccfc0f50e72b43b9604bd97) allows
[https://docs.djangoproject.com/en/dev/ref/models/lookups/#django.db.models.lookups.RegisterLookupMixin.register_lookup
registering lookups per field instances], so you will be able to register
`__startswith` for `parent_object_id`, e.g.
{{{#!python
parent_field = Child._meta.get_field("parent_object_id")
with register_lookup(parent_field, StartsWith):
Child.objects.filter(parent_object_id__startswith='c')
}}}
Duplicate of #29799.
--
Ticket URL: <https://code.djangoproject.com/ticket/34015#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: Thomas | Owner: nobody
Component: Database layer | Version: 4.1
(models, ORM) |
Keywords: ORM lookup | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):
* status: new => closed
* resolution: => duplicate
--
Ticket URL: <https://code.djangoproject.com/ticket/34015#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: Thomas | Owner: nobody
Type: New feature | Status: closed
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution: duplicate
Keywords: ORM lookup | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Alex Morega):
Also, Thomas, it makes sense to assume there is a JOIN in the second
queryset, but apparently there isn't:
{{{#!python
>>>
print(Child.objects.filter(parent_object__object_id__startswith='c').query)
SELECT "test_child"."id", "test_child"."parent_object_id" FROM
"test_child" WHERE "test_child"."parent_object_id" LIKE c% ESCAPE '\'
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34015#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: Thomas | Owner: nobody
Type: New feature | Status: closed
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution: duplicate
Keywords: ORM lookup | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Thomas):
@Mariusz Felisiak: Thanks for the heads-up, I hadn't found that ticket in
my searches.
@Alex Morega: Thank you. I should have checked the SQL of my example for
the join. I relied on my findings on my existing code base which uses
`parent_object__object_id__startswith` to circumvent the RelatedField
lookup problem. I have the join there but it must be coming from somewhere
else.
--
Ticket URL: <https://code.djangoproject.com/ticket/34015#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: Thomas | Owner: nobody
Type: New feature | Status: closed
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution: duplicate
Keywords: ORM lookup | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Simon Charette):
This also has similarities with this very old ticket
https://code.djangoproject.com/ticket/2331#comment:7 so it's likely that
`parent_object_id__startswith='c'` was actually never supported which is
surprising to me.
--
Ticket URL: <https://code.djangoproject.com/ticket/34015#comment:7>