[Django] #32484: Can't access inherited fields from multi-table inherited model when using apps from MigrationExecutor

103 views

Skip to first unread message

Django

unread,

Feb 26, 2021, 8:17:36 AM2/26/21

to django-...@googlegroups.com

#32484: Can't access inherited fields from multi-table inherited model when using
apps from MigrationExecutor
-------------------------------------+-------------------------------------
Reporter: Max N | Owner: nobody
Type: Bug | Status: new
Component: Database | Version: 3.1
layer (models, ORM) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
I came across an issue when using the `apps` value from the
`MigrationExecutor`. If I use this `apps.get_model` I can't use fields
from a multi-table inherited model.

I have two applications (`api` and `labels`), with two models
(`DocumentLabel` and `Label` respectively). `DocumentLabel` uses multi-
table inheritance from `Label`. If I use the `MigrationExecutor` to load a
migration, and then use `apps.get_model` from that state
(`executor.loader.project_state(migrate_from).apps`) then the returned
model from `get_model` of `DocumentLabel` behaves incorrectly, it doesn't
allow me to reference the fields from the inherited `Label` model. It also
throws errors if I try to access the `label_ptr` field.

If I use `apps` from `django.apps` it works as expected.

For example, `api.models` contains:

{{{#!python
class DocumentLabel(LabelsLabel):
"""Label for organising documents."""

assigned_to = models.ForeignKey(
settings.AUTH_USER_MODEL, null=True, on_delete=models.SET_NULL
)
workspace = models.ForeignKey("Workspace", on_delete=models.SET_NULL,
null=True)

class Meta:
constraints: List[BaseConstraint] = []
}}}

The `DocumentLabel` inherits from the `Label` class from another app
`labels.models`:

{{{#!python
class Label(CreatedAtUpdatedAt): # type: ignore
"""The label model should be used as default"""

name = models.TextField()
type = models.ForeignKey(
f"{LABEL_TYPE_MODEL_APP}.{LABEL_TYPE_MODEL_NAME}",
on_delete=models.CASCADE,
)

class Meta:
constraints = [
UniqueConstraint(
fields=["name", "type"],
name="%(app_label)s_label_unique_name_and_type",
)
]
}}}

I am testing a migration, so I am using the MigrationExecutor to go to a
certain migration, it does not change the models in question. When I try
to access the `type` field of the `Label` model through the
`DocumentLabel` model, something that is possible usually, I get an
exception saying that `type` is not set on the `DocumentLabel` model. The
code:

{{{#!python
executor = MigrationExecutor(connection)
old_apps = executor.loader.project_state(migrate_from).apps
executor.migrate(migrate_from)

LabelType = old_apps.get_model("api", "LabelType")
DocumentLabel = old_apps.get_model("api", "DocumentLabel")
project = LabelType.objects.create(name="Project")
label = DocumentLabel.objects.create(type=project,
name="test",workspace=workspace)
}}}

It throws the following:

{{{
Traceback (most recent call last):
File "/opt/project/api/tests/test_migrations.py", line 131, in
test_projects_data_migration
label_1 = DocumentLabel.objects.create(type=project,
name="test",workspace=workspace)
File "/usr/local/lib/python3.8/site-
packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py",
line 445, in create
obj = self.model(**kwargs)
File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py",
line 501, in __init__
raise TypeError("%s() got an unexpected keyword argument '%s'" %
(cls.__name__, kwarg))
TypeError: DocumentLabel() got an unexpected keyword argument 'type'
}}}

Interestingly, `LabelType` is also a model that is using multi-table
inheritance from the same application, but that works fine. The only
difference I can think of is that `Label` has a `ForeignKey` field in it,
but the inherited `LabelType` field has no foreign key.

I also tried to directly set the `label_ptr`:

{{{#!python
LabelType = old_apps.get_model("api", "LabelType")
Label = old_apps.get_model("label", "Label")
DocumentLabel = old_apps.get_model("api", "DocumentLabel")

project = LabelType.objects.create(project=True, name="Project",
workspace=workspace, allow_multiple=True)
label_label_1 = Label.objects.create(name="test", type=project)
label_1 = DocumentLabel.objects.create(label_ptr=label_label_1,
workspace=workspace)

for label in DocumentLabel.objects.all():
print(label.label_ptr)
}}}

It throws the exception:

{{{
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-
packages/django/db/models/fields/related_descriptors.py", line 173, in
__get__
rel_obj = self.field.get_cached_value(instance)
File "/usr/local/lib/python3.8/site-
packages/django/db/models/fields/mixins.py", line 15, in get_cached_value
return instance._state.fields_cache[cache_name]
KeyError: 'label_ptr'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/project/api/tests/test_migrations.py", line 136, in
test_projects_data_migration
print(label.label_ptr)
File "/usr/local/lib/python3.8/site-
packages/django/db/models/fields/related_descriptors.py", line 187, in
__get__
rel_obj = self.get_object(instance)
File "/usr/local/lib/python3.8/site-
packages/django/db/models/fields/related_descriptors.py", line 302, in
get_object
kwargs = {field: getattr(instance, field) for field in fields}
File "/usr/local/lib/python3.8/site-
packages/django/db/models/fields/related_descriptors.py", line 302, in
<dictcomp>
kwargs = {field: getattr(instance, field) for field in fields}
AttributeError: 'DocumentLabel' object has no attribute 'id'
}}}

I looked into the model at that point, and the `label_ptr` is not found. I
also looked into the code of `ForwardManyToOneDescriptor` and found that
`get_cached_value` throws a `KeyError` exception, which triggers some
logic. I tried to 'pre-cache' the related model by doing the following:

{{{#!python
DocumentLabel.objects.all().prefetch_related("label_ptr")
}}}

This actually causes the `label_ptr` to be set, but it doesn't allow the
'transparent' ability to query `Label` fields as if they are part of the
`DocumentLabel` model.

--
Ticket URL: <https://code.djangoproject.com/ticket/32484>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,

Mar 2, 2021, 5:18:08 AM3/2/21

to django-...@googlegroups.com

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Carlton Gibson):

* status: new => closed
* resolution: => needsinfo

Comment:

Hello. I need to ask you to provide a sample project with exact steps
here to have a chance to reproduce this. There's simply not enough detail
otherwise. Sorry.

The only thing that catches my eye is this:

> old_apps = executor.loader.project_state(migrate_from).apps

I'd suspect you'd want the project state **after** the migration… — but as
I say, without being able to look in depth, it's not really possible to
say.
Thanks.