[Django] #21699: Provide a way to define a model without being registered into AppConfig
Django
------------------------------+-------------------------
Reporter: mitar | Owner: nobody
Type: New feature | Status: new
Component: Core (Other) | Version: 1.6
Severity: Normal | Keywords: app-loading
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-------------------------
Provide a way to define a model without being registered into AppConfig.
This is useful when you are creating dynamic models to be returned from
querysets which have dynamic virtual fields (for example, created from
data which can have arbitrary virtual fields stored). You do not want all
those models to be stored in AppCache.
--
Ticket URL: <https://code.djangoproject.com/ticket/21699>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: mitar | Owner: nobody
Type: New feature | Status: new
Component: Core (Other) | Version: 1.6
Keywords: app-loading | Triage Stage: Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by aaugustin):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Comment:
I know I'm insisting heavily, but can you formulate this feature request
in terms of effects on the public API of the app registry, or any other
public API of Django?
For example:
- "I would like app_config.get_model(xxx) to return None when I've created
a model in this way."
- "I would like Django not to crash miserably with this stack trace when
I've created a model in that way."
A request on the internal implementation of the app registry doesn't make
sense because only public APIs are guaranteed to keep working as
documented across versions. Why would you care that the model is or isn't
referenced somewhere in the app registry, provided it's never returned or
used by any API? In fact, if this feature gets implemented, it's likely
that Django will register this model with some sort of "hidden" flag.
--
Ticket URL: <https://code.djangoproject.com/ticket/21699#comment:1>
Django
Reporter: mitar | Owner: nobody
Type: New feature | Status: new
Component: Core (Other) | Version: 1.6
Keywords: app-loading | Triage Stage: Unreviewed
Easy pickings: 0 | UI/UX: 0
Comment (by charettes):
I'm also curious to know why you'd need such a feature? There was no way
of doing this with the legacy `django.db.models.loading` AFAIK.
Wouldn't making a `django.apps.registry.Apps` no-op subclass and
specifying it as your `DynamicModel.Meta.apps` work?
{{{
#!python
class Void(django.apps.registry.Apps):
def register_model(self, app_label, model):
pass
void = Void()
Meta = type('Meta', (), {'apps': void})
MyDynamicModel = type('MyDynamicModel', (django.db.models.Model,),
{'Meta': Meta})
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/21699#comment:2>
Django
Reporter: mitar | Owner: nobody
Type: New feature | Status: new
Component: Core (Other) | Version: 1.6
Keywords: app-loading | Triage Stage: Unreviewed
Easy pickings: 0 | UI/UX: 0
Comment (by aaugustin):
Setting Model._meta.auto_created most likely does exactly what the OP
wants, but since they didn't specify, I can't be sure.
https://github.com/django/django/blob/98b52ae2/django/apps/registry.py#L222
--
Ticket URL: <https://code.djangoproject.com/ticket/21699#comment:3>
Django
Reporter: mitar | Owner: nobody
Type: New feature | Status: new
Component: Core (Other) | Version: 1.6
Keywords: app-loading | Triage Stage: Unreviewed
Easy pickings: 0 | UI/UX: 0
Comment (by aaugustin):
`Model._deferred` might be even more suitable in that case, since it's
what Django does for its own dynamic models.
Of course it would be better to provide a formal API.
--
Ticket URL: <https://code.djangoproject.com/ticket/21699#comment:4>
Django
Reporter: mitar | Owner: nobody
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: app-loading | Triage Stage: Unreviewed
Easy pickings: 0 | UI/UX: 0
Changes (by aaugustin):
* version: 1.6 => master
Comment:
Maybe the goal is to avoid leaking memory when creating a large number of
such models?
--
Ticket URL: <https://code.djangoproject.com/ticket/21699#comment:5>
Django
Reporter: mitar | Owner: nobody
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: app-loading | Triage Stage: Unreviewed
Easy pickings: 0 | UI/UX: 0
Comment (by mitar):
I am just copying from #21698, to make things more clear here. So my
comment how I see that unregistering could help:
> So there are two ways to allow making custom subclasses of models which
should not be registered. One is to somehow allow some configuration in
Meta (#21682), another is to call current metaclass which adds it, but be
able to remove it from registry immediately afterwards (a bad side-effect
is that registration signal has been already made).
I didn't know about "no-op subclass" approach. This seems interesting
idea.
> Maybe the goal is to avoid leaking memory when creating a large number
of such models?
Yes. So what we are doing is developing support for dynamic schema. Users
can add dynamically fields from the many hop away models. In a way we make
proxy fields to things you would need multiple hops and we do one big join
instead in the background to not have to fetch them again and again as you
are hopping around. (Those hops can cross content type relations as well.)
(Which mean you can change concrete implementation of a model, but the
rest of your code can still work as it is referencing them through
registered names.) We add those fields as virtual fields into base model
and create a dummy proxy model to keep them, when we return a new queryset
for them. So the syntax is something like:
{{{
queryset = models.Node.objects.regpoint('config').registry_fields(
name='core.general#name',
type='core.type#type',
router_id='core.routerid#router_id',
project='core.project#project.name',
).regpoint('monitoring').registry_fields(
last_seen='core.general#last_seen',
network_status='core.status#network',
monitored_status='core.status#monitored',
health_status='core.status#health',
peers='network.routing.topology#link_count',
).order_by('type')
}}}
So, after you fetch those additional fields, returned queryset contains a
dynamically created proxy model to `models.Node`, with `name` and others
added as virtual fields. As this proxy model is useful only for this
particular queryset, we do not want to register it globally. `core.status`
is a registered name for a particular registry point, which can be
implemented by any model which announces that it is implementing it. So a
concrete related model can change (be extended by an app), but your
queryset does not have to change.
All this is a bit involved and we do not yet have good documentation, but
you can check [https://dev.wlan-si.net/wiki/Nodewatcher/Registry this
description] and our
[https://github.com/wlanslovenija/nodewatcher/blob/77ec018255fa0546b362376976db2f7b874968c1/nodewatcher/core/registry/lookup.py#L138
current implantation of creating a dynamic proxy model and unregistered
immediately afterwards is here]. We do not want that such models pile up.
I know this is not public API, but it would still be good to have some
clear way to: or not register a model when created, or be able to remove
it from the registry. As you are planing on rewriting AppConfig, I just
wanted to raise a point that some internal API (which I understand can
change at any time) would be useful here.
--
Ticket URL: <https://code.djangoproject.com/ticket/21699#comment:6>
Django
Reporter: mitar | Owner: nobody
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: app-loading | Triage Stage: Unreviewed
Easy pickings: 0 | UI/UX: 0
Comment (by aaugustin):
So we're talking of hacking the internals heavily here ;-)
The implementation of Apps has become very straightforward if you ignore
all the deprecated code. In the short term I'd just go for `del
apps.all_models[app_label][model_name]` or `del
apps.get_app_config(app_label).models[model_name]`, followed by
`apps.clear_cache()`. The first one is marginally faster, the second one
is marginally less likely to change. Make sure `model_name` is lowercase.
Yesterday I tried to stop storing deferred models with the app registry.
It would have solved your use case nicely -- what you're doing looks a lot
like deferred models. Sadly it didn't work.
This request has larger ramifications. Currently ModelBase.__new__ always
returns a class registered in the app registry. It's quite a hack, and
it's done for bad reasons that no longer exist since Django 1.4 (new
project layout). I'm going to keep the ticket open with the following
scope: get rid of get_registered_model, and then figure out what the
registration API becomes.
--
Ticket URL: <https://code.djangoproject.com/ticket/21699#comment:7>
Django
Reporter: mitar | Owner: nobody
Type: New feature | Status: new
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by aaugustin):
* stage: Unreviewed => Someday/Maybe
--
Ticket URL: <https://code.djangoproject.com/ticket/21699#comment:8>
Django
Reporter: mitar | Owner: nobody
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: app-loading | Triage Stage: Someday/Maybe
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by mitar):
> So we're talking of hacking the internals heavily here ;-)
Yes. :-) But from such hacking new use cases emerge. :-)
> Currently ModelBase.new always returns a class registered in the app
registry.
Yes. This is why we register, and then try to unregister. So maybe then
#21698 is a better approach. The issue there is that you still send a
signal when a new model is registered, despite then removing it almost
immediately. Maybe an alternative would be that API could provide a way to
register a model without sending a signal, and then to unregister. But
this is just a hack for current approach with `get_registered_model`.
--
Ticket URL: <https://code.djangoproject.com/ticket/21699#comment:9>
Django
Reporter: mitar | Owner: nobody
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: app-loading | Triage Stage: Someday/Maybe
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by aaugustin):
`ModelBase.__new__` cannot return a class previously registered in the app
registry any more.
`get_registered_model` is still used in `ModelSignal.connect` and in
`add_lazy_relation`. In both cases it triggers different code paths
depending on whether the target model was already created and its
`class_prepared` signal was sent.
--
Ticket URL: <https://code.djangoproject.com/ticket/21699#comment:10>
Django
registry / Get rid of get_registered_model
------------------------------+-----------------------------------------
Reporter: mitar | Owner: nobody
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: app-loading | Triage Stage: Someday/Maybe
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by aaugustin):
mitar, how should relations (foreign keys, one to one, many to many)
between such a model and other models behave?
--
Ticket URL: <https://code.djangoproject.com/ticket/21699#comment:11>
Django
registry / Get rid of get_registered_model
------------------------------+-----------------------------------------
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: app-loading | Triage Stage: Someday/Maybe
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by kostko):
> mitar, how should relations (foreign keys, one to one, many to many)
between such a model and other models behave?
Do you see any specific problems here?
--
Ticket URL: <https://code.djangoproject.com/ticket/21699#comment:12>
Django
registry / Get rid of get_registered_model
------------------------------+-----------------------------------------
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: app-loading | Triage Stage: Someday/Maybe
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by aaugustin):
Yes. If you have multiple variants of a model with relations to another
model, the reverse relations conflict on the other model.
--
Ticket URL: <https://code.djangoproject.com/ticket/21699#comment:13>
Django
registry / Get rid of get_registered_model
------------------------------+-----------------------------------------
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: app-loading | Triage Stage: Someday/Maybe
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by John Cronan):
Hi! This is a very old ticket, but I've tested this same approach in 3.2
and it still works. I'm doing a form-builder type of app, and I decided to
use dynamic models.
I get the warning message about "reloading models is not advised as it can
lead to inconsistencies." In my case, I've ensured that the dynamic models
don't use any foreign keys (except between each other).
Considering the scope, "figure out what the registration API becomes":
just as someone using this capability, I'd suggest that there could be a
way to suppress the warning when you know that the side effects are not
harmful. True unregistering, where the side effects with related models
and such are removed automatically, would be cool but sounds hard.
--
Ticket URL: <https://code.djangoproject.com/ticket/21699#comment:14>
Django
registry / Get rid of get_registered_model
------------------------------+-----------------------------------------
Severity: Normal | Resolution:
Keywords: app-loading | Triage Stage: Someday/Maybe
Has patch: 0 | Needs documentation: 0
Changes (by Ülgen Sarıkavak):
* cc: Ülgen Sarıkavak (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/21699#comment:15>