Towards version 1.0

[Simone Deponti]

Hello all,

this is going to be a long post. After almost one year of being there, I re-evaluated some of the choices made with djc.recipe. The first, and quite distinguishing choice, was that djc.recipe did allow you to control your settings.py from within the buildout, using a templating mechanism.
I'm not quite sure anyone but me has ever used this feature or found it cool, but the basic reasoning behind it was more or less the following:

settings.py always irked me a bit. It's configuration, and lives in a code module. This can get extremely annoying to manage especially for sysadmins. For example, in the company where I work, it is very likely that the sysadmin actually manages the production.cfg buildout config, usually setting the proper configuration values (from simple things such as db connections to more complicated things like memcached setups) and having to modify a code module was not very well received. Hence the "settings as template" approach and the somewhat silly loading of them on-the-fly before starting Django.

Another minor issue was the the secret key in settings.py. This is largely demanded to human intervention in a standard Django setup, and it is bad. Very bad. It's very easy to be lax, and not resort to a script and go production with secrets like f00b4r and sp4msp4m3ggs. djc.recipe managed the secret key for you, creating a fairly secure one on the fly the first time you run the buildout. All this led me to choose the templating approach.

Sadly, using it in quite some projects brought up several defects in this approach:

1. it's not very productive, when compared to the standard Django approach, as far as adding some custom value in the settings.py is concerned (a fairly common task). In standard Django, you just add the value in settings.py. With the djc.recipe approach, you have to provide a custom template or extension, add the value there, and optionally use a variable there so you can put the real value in the buildout config. Eeek.
2. sometimes you shoot yourself in the foot. We (and by this I mean me and my colleagues) took the nasty habit to quickly add the value in the generated settings.py, and only after the feature was done to take the bother to do all that was explained in #1. Only sometimes you forgot about this and ran buildout. Oops.
3. the static-origin feature was very good, but did not help for speedy development. In particular, you have to rerun buildout each time you modify a resource file (such as a CSS), which doesn't quite help when you are developing JS or doing CSS work. A link feature was provided but it still had several rough edges (particularly, it linked files one-by-one, which was sub-optimal). Gnaaah.

To address these problems, several (potentially) backwards-incompatible changes are needed, in particular:

1. going back to a standard Django setup with settings.py being a real code module, usually in a package that contains just that and url definitions (the "project package")
2. providing some sort of API (to be loaded by settings.py) to manage the secret key: so for example you would put in your settings.py something like from djc.recipe import secure_key; SECRET_KEY = secure_key()
3. providing the static-origin feature as a separate recipe within the same package, so that it would have its separate part in the buildout and therefore enabling you to run just that part with bin/buildout install copy_resources
4. providing the feature above with a smarter linking algorythm that does link entire directories if there is no need to link single files

Now, the proposed changes are very likely backwards incompatible. Would it be a problem if from version 0.8 we switched to a new API? All existing projects can keep using the old (0.7) serie that will still be maintained, while new projects can have less hassle. Thoughts?

[Aaron VanDerlip]

Hi,

I am currently using djc.recipe for one of my projects. I am primarily
using it for creating different settings.py files for development,
staging and production. For instance is useful to set have a prod.cfg
file that overrides some default parameters. However I do agree that
adding custom values to settings.py is troublesome, as my colleague
would say "I have to rerun buildout and change 3 files?!?!" Are you
considering removing the templating feature?

Aaron

On Mar 30, 6:04 pm, Simone Deponti <shywolf9...@gmail.com> wrote:
> Hello all,
>
> this is going to be a long post. After almost one year of being there, I

> re-evaluated some of the choices made with *djc.recipe*. The first, and
> quite distinguishing choice, was that *djc.recipe* did allow you to control
> your *settings.py* from within the buildout, using a templating mechanism.

> I'm not quite sure anyone but me has ever used this feature or found it
> cool, but the basic reasoning behind it was more or less the following:
>

> *settings.py* always irked me a bit. It's configuration, and lives in a code

> module. This can get extremely annoying to manage especially for sysadmins.
> For example, in the company where I work, it is very likely that the

> sysadmin actually manages the *production.cfg* buildout config, usually

> setting the proper configuration values (from simple things such as db

> connections to more complicated things like *memcached* setups) and having

> to modify a code module was not very well received. Hence the "settings as
> template" approach and the somewhat silly loading of them on-the-fly before

> starting *Django*.
>
> Another minor issue was the the secret key in *settings.py*. This is largely

> demanded to human intervention in a standard Django setup, and it is bad.
> Very bad. It's very easy to be lax, and not resort to a script and go

> production with secrets like *f00b4r* and *sp4msp4m3ggs*. *djc.recipe*managed the secret key for you, creating a fairly secure one on the fly the

> first time you run the buildout. All this led me to choose the templating
> approach.
>
> Sadly, using it in quite some projects brought up several defects in this
> approach:
>

>    1. it's not very productive, when compared to the standard Django
>    approach, as far as adding some custom value in the *settings.py* is

>    concerned (a fairly common task). In standard Django, you just add the value

>    in *settings.py*. With the *djc.recipe* approach, you have to provide a

>    custom template or extension, add the value there, and optionally use a
>    variable there so you can put the real value in the buildout config. Eeek.

>    2. sometimes you shoot yourself in the foot. We (and by this I mean me

>    and my colleagues) took the nasty habit to quickly add the value in the

>    generated *settings.py*, and only after the feature was done to take the

>    bother to do all that was explained in #1. Only sometimes you forgot about
>    this and ran buildout. Oops.

>    3. the *static-origin* feature was very good, but did not help for speedy

>    development. In particular, you have to rerun buildout each time you modify
>    a resource file (such as a CSS), which doesn't quite help when you are
>    developing JS or doing CSS work. A link feature was provided but it still
>    had several rough edges (particularly, it linked files one-by-one, which was
>    sub-optimal). Gnaaah.
>
> To address these problems, several (potentially) backwards-incompatible
> changes are needed, in particular:
>

>    1. going back to a standard Django setup with *settings.py* being a real

>    code module, usually in a package that contains just that and url
>    definitions (the "project package")

>    2. providing some sort of API (to be loaded by *settings.py*) to manage
>    the secret key: so for example you would put in your *settings.py*something like from

>    djc.recipe import secure_key; SECRET_KEY = secure_key()

>    3. providing the *static-origin* feature as a separate recipe within the

>    same package, so that it would have its separate part in the buildout and
>    therefore enabling you to run just that part with bin/buildout install
>    copy_resources

>    4. providing the feature above with a smarter linking algorythm that does

[Simone Deponti]

> Are you considering removing the templating feature?

Yes, but nothing has been decided yet. Templating has pros and cons,
and I am evaluating both approaches.

So far, the pros of templating are:
* Clear distinction between production and development
* The secret key is auto-managed

The cons are:
* It's not very Django-ish
* It's somewhat longer than it should be, development-wise, to add new
settings
* The code for loading the settings sucks