[Django] #29554: Missing feature: load apps dynamically at runtime
Django
----------------------------------------------+------------------------
Reporter: Christian González | Owner: nobody
Type: New feature | Status: new
Component: Core (Other) | Version: 2.0
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
----------------------------------------------+------------------------
I stumbled upon a problem I am not able to solve, and I think this is a
feature that should be in Django core AFAIKT.
Until Django 1.x it was possible to add apps dynamically to INSTALLED_APPS
during runtime, and reload the cache, see here:
https://stackoverflow.com/questions/19833393/django-dynamically-add-apps-
as-plugin-building-urls-and-other-settings-automat
Even if I think this is a bit of a hack and it was never really meant that
way (?), nevertheless there should be an official possibility to do so.
I have made a Django application with dynamically apps at server startup
time - this is no problem, just search for a plugins dir during
settings.py and add the dotted paths to INSTALLED_APPS. Works fine, but
one problem remains:
If I want to enable dynamic app installation - like an "app store" in my
application - users can download apps from my "store", and enable them
using my web interface - there currently is no way to achieve that besides
restarting the Django server, right?
During setup, Apps.populate() is called - and its docstring says that the
method is idempotent. Is it possible/allowed/recommended to add some paths
to django.conf.settings.INSTALLED_APPS and call that populate() again
later during runtime?
--
Ticket URL: <https://code.djangoproject.com/ticket/29554>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
------------------------------------+--------------------------------------
Reporter: Christian González | Owner: nobody
Type: New feature | Status: new
Component: Core (Other) | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* cc: Aymeric Augustin (added)
Comment:
As the author of the app loading refactor, Aymeric may have some input.
--
Ticket URL: <https://code.djangoproject.com/ticket/29554#comment:1>
Django
------------------------------------+--------------------------------------
Type: New feature | Status: new
Component: Core (Other) | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Aymeric Augustin):
This proposal removes the guarantees of deterministic behavior that the
app-loading refactor brought. One example among many: if two apps override
the same template, which app wins? The order of `INSTALLED_APPS` defines
this. What happens in the scenario you're proposing?
I'd like to see a thorough discussion of the pros and cons before making
this decision as well as an analysis of which caches need to be
invalidated and how this could happen.
I wrote extensively about this topic on the django-developers mailing-list
during the app-loading refactor. As far as I can tell, all this material
is still relevant, so you should be able to get the list of topics to
discuss from there. You can also check out my conference talks about app-
loading.
--
Ticket URL: <https://code.djangoproject.com/ticket/29554#comment:2>
Django
------------------------------------+--------------------------------------
Component: Core (Other) | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* status: new => closed
* resolution: => needsinfo
--
Ticket URL: <https://code.djangoproject.com/ticket/29554#comment:3>
Django
------------------------------------+--------------------------------------
Component: Core (Other) | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Christian González):
Should I discuss this on the mailing list then, or here?
Just to answer your questions:
* regarding the INSTALLED_APPS order: Sure, if you have a stati
application, this order makes sense. If you have an application that uses
plugins, the application itself is responsible for the order the plugins
are loaded. To make this deterministic, would be nonsense IMHO, as
different applications use different plugin methods (Django provides
None). So It could be that there is a "plugin resolution order" where
dependendant plugins are loaded after their dependencies - which makes
sense, or the app could decide that plugins define an integer "weight"
(which is no good practice IMHO), etc. So, that's the sense of "dynamic"
apps.
* Regarding templates, and which app wins - I always fount the templating
system a bit difficult - IMHO app templates should have implicit
namespaces. But that would break virtually any existing Django install -
so forget it. But: Same as before: The loading order decides, like in
{{{INSTALLED_APPS}}}. Let the application decide in which order the
plugins should be loaded.
In my simple mind, I call {{{apps.populate([myfirstapp_plugin,
other_plugin, last_plugin_app])}}} like setup.py calls it with
{{{settings.INSTALLED_APPS}}}. And populate works through all he apps **in
this order** and merges them into the system.
I wrote a Django application that merely consists of the plugin system.
The "real" main application is already a "Core" plugin (with a version
etc) which is listed in INSTALLED_APPS as app, because it is always loaded
and part of the base system. And all other plugins are loaded dynamically.
So they can have dependencies like "core>=1.5.2" and it won't load it when
the core version is below that. It's pre-alpha, just look here for the
extension system
[https://gitlab.com/nerdocs/medux/MedUX/tree/develop/medux/extensionsystem]
(e.g. the PluginManager, is in development. I'm no professional software
engineer, so please look away if you see horrible coding style ;-) - or
better tell me)
I tried really many ways to accomplish that, and some of the first ones
were plugins that are no django apps. But I came across some issues, and
had to rewrite much of the stuff Django does already perfectly, like
loading models, migrations, etc. So in the sense of DRY, I wanted to have
plugins being Django apps. But there the problem I describe here starts: I
have to restart the server, rewriting the settings.py - which is nonsense
in my eyes.
--
Ticket URL: <https://code.djangoproject.com/ticket/29554#comment:4>
Django
------------------------------------+--------------------------------------
Component: Core (Other) | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Christian González):
Just wanted to state that IMHO this is a real *regression* in the Django
evolvement, as in earlier Django versions this was possible.
How can I help here? Should I start a thread on the mailing list?
--
Ticket URL: <https://code.djangoproject.com/ticket/29554#comment:5>
Django
------------------------------------+--------------------------------------
Component: Core (Other) | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Christian González):
Aymeric Augustin: Please just tell me where I should discuss this topic:
here or in the devel mailing list. IMHO this is a real gap in the Django
environment.
And I'd like to contribute to help close it. But tell me where - I don't
want to pollute the devel mailing list with things that don't belong
there.
But I can't understand why this issue is closed that fast.
--
Ticket URL: <https://code.djangoproject.com/ticket/29554#comment:6>
Django
------------------------------------+--------------------------------------
Component: Core (Other) | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Aymeric Augustin):
django-developers is the right place for discussing app loading. See
https://groups.google.com/forum/#!searchin/django-developers/app-
loading%7Csort:date/django-developers/_iggZzrYtJQ/FWFPgCflSnkJ for the
last episode.
Unfortunately this is a poor example of open-source development. App-
loading is such a complex issue that I felt unable to handle it fully in
public. Instead I had lots private discussions with a small group of
active core devs, I merged patches once I had a rough agreement from that
group, and I handled complaints after the fact. Apparently enough people
felt the result was worth it that I didn't get kicked out but I decided I
wouldn't do that again.
Anyway, that mailing list discussion provides some context. You should
make sure you understand why the app registry population is designed the
way it is before attempting to change it. You should also demonstrate that
you have reviewed past issues related to app loading order and especially
unstable import ordering.
--
Ticket URL: <https://code.djangoproject.com/ticket/29554#comment:7>
Django
------------------------------------+--------------------------------------
Component: Core (Other) | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Christian González):
Thanks. I'll do my best there.
--
Ticket URL: <https://code.djangoproject.com/ticket/29554#comment:8>
Django
------------------------------------+--------------------------------------
Component: Core (Other) | Version: 2.0
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham):
[https://groups.google.com/d/topic/django-
developers/Ebgwvb4AV7U/discussion django-developers discussion]
--
Ticket URL: <https://code.djangoproject.com/ticket/29554#comment:9>