app loading

[Alex Kamedov]

Hi all! Happy new year!

Sorry for my bad english.

On this weekend I reviewed app-loading branch[1] and want to propose some improvements.

Now app-loading branch resolve how to load application, store its meta data and load its models.

I plan to extend it to get simple way for replacing the same type applications. This problem is 

know as Lazy Foreign Key and now can be resolve through settings, but django contrib modules

currently is not supported it. I propose more beautiful way to resolve this problem. 

The main goal of my improvements is adding ability to simple replace one application to another

with the same type but another functionality.  For example, if you need replace `django.contrib.auth`

on your own application you must rewrite all used 3rd-party applications linked with it now. I 

propose way to resolve this problem and I think It is not hard to implement with saving backward 

compatibilities.

1. Change INSTALLED_APPS form tuple to dictionary or may be it's better to use something with 

the same interface as tuple and dictionary 

    INSTALLED_APPS = {

        'auth': 'django.contrib.auth',

        'comments': 'django.contrib.comments'.

        ...

    }

For backward compatibility it checks on setting load stage and convert to dictionary if it need.

INSTALLED_APPS will be low level API to install applications. I propose install application through

`install_applications` function like this:

    from django.core.apps import install_applications

    INSTALLED_APPS = install_applications(

        'django.contrib.contenttypes',

        'django.contrib.sessions',

        ...

        auth = 'django.contrib.auth',

        comments = 'django.contrib.comments'

        gallery = App(path='my_gallery', db_prefix='new_gallery', verbose_name=u'My new gallery')

    )

There are args and kwargs passed to `install_applications`. It can be python module names of 

django applications. Keys of kwargs are system names of the installed applications. This names 

will be used to get application instance in other applications (see next for more details).

For applications passed through args keys will be auto generated. 

The main benefit is a manage installed applications in project settings module without make 

changes in this applications. 

Application loading process will be as listed below:

 * iterate over INSTALLED_APPS items

 * if item is string - try to load application definition (`Application`class) from python package 

    defined in this string and create its instance. If application definition is not provided, it will be 

    auto generated. `django.core.apps.App` class will be used for it.

 * if item is `App` instance, check existence of python package at `path` attribute of `App` instance

   and `ImproperlyConfigured` exception will be raised if this python package is not installed. 

2. Way to organize relations between models of different applications (something like this it 

already exist in app-loading branch and trunk. I show this only for example and demonstrate 

syntax)

    from django.core.apps import get_app, get_model

    auth = get_app('auth')

    class MyModel(models.Model):

        user = models.ForeignKey(auth.models.User)

    or 

    class MyModel(models.Model):

        user = models.ForeignKey(get_model('auth.User'))

3. Add new exceptions: `AppIsNotInstalled` (instead of `ImproperlyConfigured`) will be raised if it is 

try to load not installed applications through `get_app`.

[1] http://code.djangoproject.com/wiki/SummerOfCode2010#Apploading

Waiting for your comments.

Cheers!

[Łukasz Rekucki]

What about application order? It's important in some parts of Django,
like template loading order.

> For backward compatibility it checks on setting load stage and convert to
> dictionary if it need.
> INSTALLED_APPS will be low level API to install applications.

There is another aspect of backwards-compatibility: there are many 3rd
party tools that assume INSTALLED_APPS to be a tuple with application
names, so changing it to a dictionary will break them.

> I propose install application through `install_applications` function like this:
>     from django.core.apps import install_applications
>     INSTALLED_APPS = install_applications(
>         'django.contrib.contenttypes',
>         'django.contrib.sessions',
>         ...
>         auth = 'django.contrib.auth',
>         comments = 'django.contrib.comments'
>         gallery = App(path='my_gallery', db_prefix='new_gallery',
> verbose_name=u'My new gallery')
>     )

Same problem as above, keyword arguments have no order.

> There are args and kwargs passed to `install_applications`. It can be python
> module names of
> django applications. Keys of kwargs are system names of the installed
> applications. This names
> will be used to get application instance in other applications (see next for
> more details).
> For applications passed through args keys will be auto generated.
> The main benefit is a manage installed applications in project settings
> module without make
> changes in this applications.

Isn't this already done ? You can't assign arbitrary names to
applications, but if you use "myproject.auth" instead of
"django.contrib.auth", get_app() and get_model() will work as you
expected.

--
Łukasz Rekucki

[Alexander Dutton]

I don't know how useful this will be as it's mostly a 'this is how we
tackled the problem'. I haven't worked out how your parameters are
passed to the views, so it's possible you may be interested

On 02/01/11 11:17, Alex Kamedov wrote:
> INSTALLED_APPS will be low level API to install applications. I
> propose install application through
> `install_applications` function like this:
>
> from django.core.apps import install_applications
>
> INSTALLED_APPS = install_applications(
> 'django.contrib.contenttypes',
> 'django.contrib.sessions',
> ...
> auth = 'django.contrib.auth',
> comments = 'django.contrib.comments'
> gallery = App(path='my_gallery', db_prefix='new_gallery',
> verbose_name=u'My new gallery')
> )
>

> [snip]

I developed something akin to this for the Molly project¹ (a
framework-on-a-framework), albeit using an APPLICATION setting instead
of the INSTALLED_APPS setting (so as not to have to hack Django itself):

> APPLICATIONS = [
> … Application('molly.apps.z3950', 'library', 'Library search',
> verbose_name = 'Oxford Library Information System',
> host = 'library.ox.ac.uk',
> database = 'MAIN*BIBMAST',
> ),
> ]

The signature is Application(app_path, app_name, title, **params).

INSTALLED_APPS then has the app paths from APPLICATIONS appended to it
to keep the rest of Django happy ('normal' Django apps are handled as
before). We stuck with a list as the order is when displaying apps on
the homepage.

We took this route so that we could parameterize apps without having a
profusion of settings. It also lets us have multiple instances of apps
with different parameters. Application objects can be retrieved by name
or package path, and they have a urls attribute that contains a resolver.

It's hacky, but the urls attribute is constructing by walking the app's
urls.urlpatterns and initialising each class-based view with the params.
This manifests as a conf attribute on view instances.

If you're interested, see an example root urlconf², our hacky
infrastructure³, and an app using this stuff⁴.

Hope this is of some use!

Alex

¹ http://github.com/mollyproject/mollyproject
²
https://github.com/mollyproject/mollyproject/blob/6ab2d4f/demos/molly_oxford/urls.py
³
https://github.com/mollyproject/mollyproject/tree/6ab2d4f/molly/conf/settings.py
⁴
https://github.com/mollyproject/mollyproject/blob/6ab2d4f/molly/apps/contact/views.py

[Alex Kamedov]

2011/1/2 Łukasz Rekucki <lrek...@gmail.com>

What about application order? It's important in some parts of Django,
like template loading order.

Can you describe this problem with more details? 

Usualy each applications has own path for its templates.

> For backward compatibility it checks on setting load stage and convert to
> dictionary if it need.
> INSTALLED_APPS will be low level API to install applications.

There is another aspect of backwards-compatibility: there are many 3rd
party tools that assume INSTALLED_APPS to be a tuple with application
names, so changing it to a dictionary will break them.

I've say it's no hard to provide interface like dict and compatible with tuple.

I show only proposed syntax.

Same problem as above, keyword arguments have no order.

Can you share pruf link or provide more details of described problem?

--

skype: kamedov

www: kamedov.ru

[Łukasz Rekucki]

On 2 January 2011 19:27, Alex Kamedov <kam...@gmail.com> wrote:
>
> 2011/1/2 Łukasz Rekucki <lrek...@gmail.com>
>>
>> What about application order? It's important in some parts of Django,
>> like template loading order.
>
> Can you describe this problem with more details?
> Usualy each applications has own path for its templates.

Yes, but sometimes you want to override a template from another
application, like grapelli[1]. The AppDir template loader searches in
order of installed applications. By putting grapelli in front of
contrib.auth, you're able to override the original admin templates.

>>
>> > For backward compatibility it checks on setting load stage and convert
>> > to
>> > dictionary if it need.
>> > INSTALLED_APPS will be low level API to install applications.
>>
>> There is another aspect of backwards-compatibility: there are many 3rd
>> party tools that assume INSTALLED_APPS to be a tuple with application
>> names, so changing it to a dictionary will break them.
>
> I've say it's no hard to provide interface like dict and compatible with
> tuple.
> I show only proposed syntax.
>>
>> Same problem as above, keyword arguments have no order.
>
> Can you share pruf link or provide more details of described problem?

I'm not sure I know what you mean by "pruf link". If you provide
keyword arguments to a function, you have no way of telling in which
order they were provided, because all you get is a dictionary (which
has no meaningful order):

>>> def a(**kwargs):
... print kwargs
...
>>> a(a=1, b=2, c=3)
{'a': 1, 'c': 3, 'b': 2}

So you can't use that syntax and keep ordering. If the template
example doesn't convince you, take a look at the "syncdb" command. You
need a way to specify in which order in which custom SQL or fixtures
are loaded from each application. I don't see a one with a dictionary
or keyword arguments.

[1]: http://code.google.com/p/django-grappelli/wiki/Installation

--
Łukasz Rekucki

[Alex Kamedov]

Thank you. I see the problem now.

What about this?

INSTALLED_APPS = (

    'django.contrib.sessions',

    ('comments', 'django.contrib.comments'),

    (gallery, App(path='my_gallery', db_prefix='new_gallery', verbose_name=u'My new gallery')),

    ...

)

2011/1/3 Łukasz Rekucki <lrek...@gmail.com>

--
Łukasz Rekucki

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Arthur Koziel]

Hey Alex,

thanks for taking the time to review the code. However, we've agreed not to construct the app instances in the settings.py file as it would require imports to be used, and stick to the convention of dotted paths. You might want to read this thread [0], but especially this [1] reply from Jannis Leidel.

We also discussed this whole thing at the DjangoCon sprints, which led us to the current implementation. There are now 2 settings with which apps can be installed: 

- APP_CLASSES: which is a list of dotted paths pointing to App classes

- INSTALLED_APPS: stays the same, dotted paths to python modules

Arthur

[0]: http://groups.google.com/group/django-developers/browse_thread/thread/4cca2086dd485879/

[1]: http://groups.google.com/group/django-developers/msg/fd2e3bc0593f4158

--

[Alex Kamedov]

Hello Arthur, thanks for good work on this branch.

I've knew about APP_CLASSES on starting the topic, but I not understood why it was not merged with INSTALLED_APPS.

Thank you for explanation. Now I see.

I found some problems with contrib applications and managment commands and can produce patch to fix it.

Cheers!

--

Alex Kamedov

skype: kamedov    www: kamedov.ru