Re: Changes to django's settings module

[Alex Ogier]

I find the value of separate settings modules is not splitting them by topic, but overriding them in specific contexts, like staging, production and development. Your implementation (and, I think, any solution that compiles multiple settings modules independently) don't have a way to specify orderings in a non-magical way. That's why I prefer explicitly importing some common settings into one module and overriding them piecemeal. The Zen of Python says "Explicit is better than implicit" and I think that is the case here. Packages provide settings and reasonable defaults. If you want to modularize them, you are free to do so yourself. I think composing settings internally is just added complexity for little benefit.

Best,

Alex Ogier

On Wed, Mar 13, 2013 at 12:27 PM, Omer Katz <omer...@gmail.com> wrote:

Lately I implemented some changes for django's settings module .

I refactored the whole module in order to have more extension points.

With #20040 it is now possible to inject the Settings class that will be used by the LazySettings object and I introduced a new class called SettingsCollector that allows the developer to costumize how django will collect it's settings.

This enabled me to write the real code that will allow loading the settings either as a package or as a module.

Instead of having one big monolitic settings.py you can now have a package that has configuration modules that are seperated by topic.

If #20040 is accepted I can work on the new SettingsCollector that will either try to load a settings module or package.

Should the new default be a package or a module? If it's a package it means we should also change the project_template.

This is the working prototype that I wrote before I thought about creating a new SettingsCollector class.

To test it just call monkey_patch() in your manage.py.

We might want to allow multiple modules and packages to be loaded.

This will help us actually use the settings_local.py in a standard way.

What do you guys think? Am I clear enoguh? Should I explain in more detail what am I thinking here?

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Russell Keith-Magee]

On Thu, Mar 14, 2013 at 1:08 AM, Alex Ogier <alex....@gmail.com> wrote:

I find the value of separate settings modules is not splitting them by topic, but overriding them in specific contexts, like staging, production and development. Your implementation (and, I think, any solution that compiles multiple settings modules independently) don't have a way to specify orderings in a non-magical way. That's why I prefer explicitly importing some common settings into one module and overriding them piecemeal. The Zen of Python says "Explicit is better than implicit" and I think that is the case here. Packages provide settings and reasonable defaults. If you want to modularize them, you are free to do so yourself. I think composing settings internally is just added complexity for little benefit.

I agree with Alex. 

Splitting settings files in the way you've described in your patch doesn't strike me as something that solves a problem that actually exists in practice. Settings files aren't *that* long, and to the extent that splits are needed, they're based on roles, not content -- you need "production" settings, not "template" settings.

Even if you did want to break out settings into multiple files, a master file with "from template_settings import *" strikes me as a lot better approach than a settings gathering process -- and that requires no extra code in Django.

Yours,

Russ Magee %-)

[Val Neekman]

Yeah, split by role is what I have done.

settings/default.py (absolute necessities)
settings/deploy.py (inherits default.py & adds production specifics)
settings/debug.py (inherits deploy.py & overwrites debug stuff)
settings/test.py (inherits deploy.py & overwrites test specific)
settings/external.py (holds aws, google, and other social keys)

The above is coupled with:

bin/debug.py (= manage.py on development server)
bin/deploy.py (= manage.py on production server)
bin/test.py (= manage.py on integration server)

This has been working very well for me.

No loose files, everything has a directory to live in.
Very easy to manage.

Val

[Alex Ogier]

I think the important point is that all of these techniques for constructing settings are done without the help of Django (and in fact *should* be done without the help of Django, because some parts of Django expect to import a settings module that isn't under construction). I don't know if there is a good way to document these practices -- it's not an easily searchable topic, and more than a single settings.py is just added confusion for little benefit until you need to care about deploying to multiple contexts (it's not good tutorial material). Maybe somewhere in the how-tos on deployment we could write up some good practices for managing settings in multiple contexts -- that would be more welcome than code in Django core, I think. Midterms end today for me, so maybe I will try writing up some of the practices that make managing settings easier for me later tonight.

Best,

Alex Ogier

[Aymeric Augustin]

On 14 mars 2013, at 15:43, Alex Ogier <alex....@gmail.com> wrote:

> I don't know if there is a good way to document these practices -- it's not an easily searchable topic, and more than a single settings.py is just added confusion for little benefit until you need to care about deploying to multiple contexts (it's not good tutorial material). Maybe somewhere in the how-tos on deployment we could write up some good practices for managing settings in multiple contexts -- that would be more welcome than code in Django core, I think. Midterms end today for me, so maybe I will try writing up some of the practices that make managing settings easier for me later tonight.

Django leaves settings organization up to each developer because requirements vary widely from one project to another and because it's a very easy problem. In other words, it's the perfect topic for bikeshedding.

This wiki page lists a few common patterns: https://code.djangoproject.com/wiki/SplitSettings

Jannis wrote https://github.com/jezdez/django-configurations and it's a solid technique, even though using classes for settings offends my aesthetic sense ;-)

Of course, my own take on this matter is a hundred times more offensive: http://www.youtube.com/watch?v=tXyenP18iUg#t=1830s

If you want to work on a patch, I would suggest to extend docs/topics/settings.txt as follows:
1 - explain that different environments or sites need different settings;
2 - present 2 or 3 well-known techniques, probably chosen among these:
- from .local_settings import *
- readthedocs.org's or djangoproject.com's technique — it's the same
- an advanced modularization technique, like django-configurations, Transifex' technique, or my talk
- configure everything through environment variables
(I'm not aware of a standard best practice, if there's one and it isn't in this list, include it!)
3 - remind the reader to keep dev & prod settings as similar as possible

We should keep this short, and take care not to be prescriptive, because the right solution really depends on the context of each project.

--
Aymeric.

[Val Neekman]

Yeah, settings is more about the environments than the framework.
There is not a single solutions that can make everyone happy. I'd say
whatever works for you is the best solution for you.

With that said, I also agree that a few good examples would be very
beneficial to people new to Django.

Val

[Omer Katz]

You haven't referred to the pull request.

 The pull request itself currently just refactors the settings module in order to have better extensiblity hooks and to clarify the code.

The pull request introduced the SettingsCollector class which allows you to costumize how exactly the settings are collected.

The LazySettings class now accepts a constructor parameter that allows you to switch the settings class. The Settings class allows you to replace the SettingsCollector class through a constructor parameter as well.

The first question is if this refactoring is needed and welcome. In my opinion it is even if you don't need it.

The simplest example is when you want an environment variable to always override a setting. You might want it because your database server is down and you want to  at point to a backup server without changing the code and thus releasing a new temporary patch version or load all settings from an *,ini file, from the database and probably other scenarios I can't come up with right now (I'm after 3 glasses of wine :):).

 Django could supply multiple SettingsCollectors to choose from and keep the current beahvior for backwards compatibility until 2.0 if needed or until forever if the default behavior is desired.

It enables much more room for flexability for developers who have a special need for different form of settings files.

If I were to extend the SettingsCollector to support loading packages as well we might get a standard way to load settings for different environments.

You can easily develop a common settings package or module for that matter and than override it with a specific package or module for each deployment using another SettingsCollector class.

I see much benefit in this refactoring and I hope that it will be accepted.

I hope I made my intentions clearer.

בתאריך יום רביעי, 13 במרץ 2013 19:27:02 UTC+3, מאת Omer Katz:

[Omer Katz]

Also don't you guys think that the SettingsCollector class will simplify the implementation of django-configurations for example?

[Alex Ogier]

All of the changes you describe are what I would call "magic". Magic has a downside: it's hard to understand and debug. As it currently stands, Django has exactly one form of settings magic: there is a DJANGO_SETTINGS_MODULE environment variable that adjusts where the settings are found. This is easy to document and point to. "My settings aren't working, what is going on?" "Check DJANGO_SETTINGS_MODULE." "Why does this setting have this wonky value that's not the default?" "Read the source code to your DJANGO_SETTINGS_MODULE."

Everything you describe is possible to do yourself explicitly if you just want it for your own app. Want to override from environment variables? os.environ.get("VARIABLE", "default value"). Reading from ini files? ConfigParser.SafeConfigParser("my.ini"). The database is tricky because you can't use the ORM without a chicken-and-egg problem, but it would be even more confusing if you *have to* import from django.conf (for BaseSettings) but you *aren't allowed to* import from django.db.

django-configurations sits entirely in front of Django, which is the right place for it to sit. Forcing everything through the narrow bottleneck of a single settings module is *a good thing*. It makes things explicit and debuggable. Other python frameworks have similar bottlenecks. Pylons has a master config file (ini format) that you specify on the commandline, and several python files in expected locations. Flask is less obvious, it has no magic files at all, but at some point you make a project-wide instance of flask.Flask and pass it to WSGI or a ./manage.py you write yourself and you set config on it manually, for example app.config.from_object("myapp.MyConfig").

The only reason for having these SettingsCollectors in core is to allow redistributable apps like django-configurations to do magic. That is, so they can say, "Install django-configurations, and suddenly every setting can be overridden from the environment!" This seems like a bad idea, versus "Install django-configurations, and now you have a new tool in your toolbox to put in settings.py that imports from the environment!"

Best,

Alex Ogier

--

[Omer Katz]

Why would you call them magic?

Why does allowing extensibility for those who need it is a bad idea?

You will be doing it explicitly anyway by providing a SettingsCollector class to the Settings class' constructor.

If are doing it, you should know what you are doing.

. Is my code harder to debug or understand than the current code? I strongly disagree here. The current Settings class clearly violates SRP. It holds the settings, validates them and collect them. What's not easy to understand about SettingsCollector? You can understand by it's name what exactly it does.

"The only reason for having these SettingsCollectors in core is to allow redistributable apps like django-configurations to do magic. That is, so they can say, "Install django-configurations, and suddenly every setting can be overridden from the environment!" This seems like a bad idea, versus "Install django-configurations, and now you have a new tool in your toolbox to put in settings.py that imports from the environment!""

Why exactly this is a bad idea? Since when extensibility is a bad idea in software development?

You just say you are against it but you don't provide a good reason other than it's easier to explain to others what's wrong if something is wrong.

 The current behavior will stay as the default behavior at least until Django 2.0 if not forever so "it's easier to explain to others what's wrong" is not a valid argument in my opinion.

If you are writing your own SettingsCollector you probably know what you are doing.

If we'll introduce other types of SettingsCollectors in Django then we won't introduce them in the Getting Started documentation until a very late stage so that newcomers can understand the default behavior but that's another issue for later on.

For now I want to focus on the pull request itself.

Does the refactoring makes the code clearer in it's intension?

Does it allow extensibility when it is required?

Does it maintain the default behavior for most Django developers?

I believe that the answer to all of those questions is yes it does.

בתאריך יום חמישי, 14 במרץ 2013 23:56:31 UTC+3, מאת Alex Ogier:

[Russell Keith-Magee]

On Fri, Mar 15, 2013 at 3:36 PM, Omer Katz <omer...@gmail.com> wrote:

Why would you call them magic?

Why does allowing extensibility for those who need it is a bad idea?

You will be doing it explicitly anyway by providing a SettingsCollector class to the Settings class' constructor.

If are doing it, you should know what you are doing.

. Is my code harder to debug or understand than the current code? I strongly disagree here. The current Settings class clearly violates SRP. It holds the settings, validates them and collect them. What's not easy to understand about SettingsCollector? You can understand by it's name what exactly it does.

"The only reason for having these SettingsCollectors in core is to allow redistributable apps like django-configurations to do magic. That is, so they can say, "Install django-configurations, and suddenly every setting can be overridden from the environment!" This seems like a bad idea, versus "Install django-configurations, and now you have a new tool in your toolbox to put in settings.py that imports from the environment!""

Why exactly this is a bad idea? Since when extensibility is a bad idea in software development?

You just say you are against it but you don't provide a good reason other than it's easier to explain to others what's wrong if something is wrong.

 The current behavior will stay as the default behavior at least until Django 2.0 if not forever so "it's easier to explain to others what's wrong" is not a valid argument in my opinion.

If you are writing your own SettingsCollector you probably know what you are doing.

If we'll introduce other types of SettingsCollectors in Django then we won't introduce them in the Getting Started documentation until a very late stage so that newcomers can understand the default behavior but that's another issue for later on.

For now I want to focus on the pull request itself.

Does the refactoring makes the code clearer in it's intension?

Does it allow extensibility when it is required?

Does it maintain the default behavior for most Django developers?

I believe that the answer to all of those questions is yes it does.

I'd definitely argue the first point - that it makes the code clearer. 

If you break up your settings file using normal Python imports, the order of evaluation of a settings file is clear, and can be followed by anyone that understands the normal Python import process.

In order to use your patch, you need to understand how settings modules will be combined, and in what order, and with what precedence. It relies upon implicit knowledge of the mode of operation of the collector, rather than the explicit behaviour of a built-in language feature.

As for the second point -- As I've said previously, I'd argue that yes, it allows extensibility, but not on any axis that, in my experience, requires it. I've seen a range of problems related to the structure of settings files, but none of them require the sort of structure that you're making easy to achieve here. If you want to propose a way to make it easy to separate production from development settings, or a way to keep private settings (like passwords and SECRET_KEY) out of repositories, I'm all ears -- but I suspect these problems are more an issue of documenting conventions than of adding features.

The last point is moot - backwards compatibility is definitely required for any feature going into core, but that doesn't mean that just because a feature is backwards compatible it will be added to core.

Yours,

Russ Magee %-)

[Omer Katz]

Doesn't the fact that the patch makes the code clearer is a reason enough for a merge (providing that there will be tests attached to it and documentation)?

I guess that for a complex project like Django it might not but I still see value in this patch for that reason alone.

Well, if you are writing a SettingsCollector you do need to understand those things but it's an advanced topic to begin with.

This patch enables to seperate development settings from production settings or keep private settings outside by writing a SettingsCollector. Why do you think that this patch won't solve these problems? I can easily demonstrate how if you wish.

I am suggesting that at some point Django will provide other SettingsCollectors and the documentation will help you decide which one you need.

Why is it an issue of only documenting conventions if there is a feature that will help you achieve your desired settings structure much more easily?

בתאריך יום שישי, 15 במרץ 2013 11:14:31 UTC+3, מאת Russell Keith-Magee:

[Omer Katz]

By the way,

The fact that I mentioned that this patch can be extended to load packages as well doesn't mean it's the only use case.

I asked you guys if you think that this is needed.

It will enable you to split settings as packages per deployment (development package, staging package, production package) and split the settings themselves by topic through different modules if you want to.

בתאריך יום שישי, 15 במרץ 2013 11:14:31 UTC+3, מאת Russell Keith-Magee:

[Shai Berger]

Omer,

To convince the core committers that the patch is valuable, you need to show
how it improves things. Build examples for the use-cases mentioned in the
thread, show how they would be done without your patch, and how the presence
of your patch allows improvements. The main argument against the patch, as far
as I've seen, is that it doesn't really help with the common use-cases; the
onus is on you to refute this claim by examples.

(also, using paragraphs will make your mails easier to read).

HTH,
Shai.

On Friday 15 March 2013, Omer Katz wrote:
> By the way,
> The fact that I mentioned that this patch can be extended to load packages
> as well doesn't mean it's the only use case.
> I asked you guys if you think that this is needed.
> It will enable you to split settings as packages per deployment
> (development package, staging package, production package) and split the
> settings themselves by topic through different modules if you want to.
>
> בתאריך יום שישי, 15 במרץ 2013 11:14:31 UTC+3, מאת Russell Keith-Magee:
> > On Fri, Mar 15, 2013 at 3:36 PM, Omer Katz

> > <omer...@gmail.com<javascript:>

> >
> > > wrote:
> >> Why would you call them magic?
> >> Why does allowing extensibility for those who need it is a bad idea?
> >> You will be doing it explicitly anyway by providing a SettingsCollector
> >> class to the Settings class' constructor.
> >> If are doing it, you should know what you are doing.
> >> . Is my code harder to debug or understand than the current code? I
> >> strongly disagree here. The current Settings class clearly violates

> >> SRP<http://en.wikipedia.org/wiki/Single_responsibility_principle>. It

> >> holds the settings, validates them and collect them. What's not easy to
> >> understand about SettingsCollector? You can understand by it's name
> >> what exactly it does.
> >>
> >> "The only reason for having these SettingsCollectors in core is to allow
> >> redistributable apps like django-configurations to do magic. That is, so
> >> they can say, "Install django-configurations, and suddenly every setting
> >> can be overridden from the environment!" This seems like a bad idea,
> >> versus "Install django-configurations, and now you have a new tool in
> >> your toolbox to put in settings.py that imports from the environment!""
> >> Why exactly this is a bad idea? Since when extensibility is a bad idea
> >> in software development?
> >> You just say you are against it but you don't provide a good reason
> >> other than it's easier to explain to others what's wrong if something
> >> is wrong.
> >>

> >> The current behavior will stay as the default behavior *at least *until

[Aymeric Augustin]

On 15 mars 2013, at 09:22, Omer Katz <omer...@gmail.com> wrote:

> Doesn't the fact that the patch makes the code clearer is a reason enough for a merge (providing that there will be tests attached to it and documentation)?

Hi Omer,

This patch isn't only a refactoring; it adds a new feature. Otherwise you wouldn't be talking about documentation.

Each feature added to Django creates a burden:
- for users of Django, who must learn to use it;
- for the core team, who must maintain it for the foreseeable future.

To be accepted, a new feature must:
(a) provide benefits that clearly outweigh these costs
(b) not get in the way of future improvements — as much as can be foreseen.

Unfortunately (a) the benefits of your PR still aren't clear and (b) judging by the discussion, your abstraction doesn't match very well the needs of most users, and I suspect it could hinder further efforts to make per-environment settings (the actual problem) easier to define.

--
Aymeric.

[Omer Katz]

Shai,

The google groups editor is kinda annoying. I'll be using GMail from now on because it removes formatting on random basis (I don't really know why)

Also, I can rewrite django-configuartion in a couple of hours using the changes in this patch. Heck, I'll even make a pull request out of it.

Aymeric,

I can revert this patch to be a refactoring if we decide that we don't see any value in the extensibility part of this patch.

I hope we all agree that this patch does make the code much clearer and understandable.

I'll get back to you guys when I'll have more progress.

2013/3/15 Aymeric Augustin <aymeric....@polytechnique.org>

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/1M5nfnpba0M/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.

[Omer Katz]

Well, I was wrong. It took me exactly 5 minutes to get the basics of django-configurations right: https://gist.github.com/thedrow/5180120

This gist loads settings from a class instead of a module and shows how to load the settings from a module and override them by using a class.

All tests run correctly.

Are you guys still not convinced?

2013/3/16 Omer Katz <omer...@gmail.com>

[Val Neekman]

If you build a geom with a bounding box that contains IDL (International Date Line) and run a point__within query on it, then the result is unpredictable and incorrect. 

Does anyone know of a native GeoDjango solution to this?

If not, would there be a lot of resistance against a patch that incorporates this functionality in GeoDjango?

Val

Sent from my mobile device. 

You received this message because you are subscribed to the Google Groups "Django developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

[Omer Katz]

Sorry, how is this related?

בתאריך יום ראשון, 17 במרץ 2013 19:34:53 UTC+3, מאת Val Neekman:

[Val Neekman]

I must have hit reply on my iPhone instead of new email. 

Not meant to be a response to your email. 

Sorry about the confusion. 

Sent from my iPad

[Omer Katz]

So is my example good enough? Have you tried using it?

בתאריך יום שישי, 15 במרץ 2013 13:17:11 UTC+3, מאת Aymeric Augustin:

[Russell Keith-Magee]

Hi Omer,

I've looked at your code, and while you continue to call it a "refactoring", it's a refactoring with a specific purpose in mind -- to add a new feature. 

I disagree that your new code is "clearer" and more "understandable" than the existing code. For starters, it's got more moving parts (pluggable Collector and Loader interfaces, for example), and while it's not too hard to work out how the parts fit together, it isn't intuitively obvious (which is what would be needed to genuinely have a simplification here).

On top of all that, what you still haven't done is express is *why* this new feature is required. You clearly want the ability to load settings from a class in some way. It isn't obvious to met at all why this is beneficial or helpful.

Lastly, the settings module is one of the warts of Django that most of the core team would like to fix. In this case, "Fixing" it doesn't mean adding more complexity and/or flexibility. It means altering altogether the way settings are handled in Django. There hasn't really been any work in this area beyond some high level talks at DjangoCon etc [1], but adding additional complexity to Django's already complex settings system isn't something we're racing to do without a *really* good reason. 

[1] http://www.youtube.com/watch?v=0FD510Oz2e4

Yours,

Russ Magee %-)

[Omer Katz]

You are contradicting yourself. At first you said that it does make the code clearer. Now you say it doesn't.

Ok, you guys are right. I'm adding a new feature.

If you think my new design isn't good enough do tell me why. I'll improve it/change it completely.

It now doesn't violate SRP, it allows flexability for those who need it and maintains backward competitability. What else is missing here?

I feel a bit like a broken record here because I did specify over and over what's the purpose of this change.

I don't want to load settings from a class. I'd like to provide extensibiloty points in order to help implemet other workflows that people use.

The purpose of this patch is to:

• Allow developers to implement different workflows for loading settings for different environments (class based settings, different modules, different packages).
• Allow developers to load settings from other sources like django-debian does.
• Allow developers to change settings as they are collected (for example django-harness does multiple things like this).

There are usecases, some people actually implemented those outside of the framework and I don't see a reason why they should. We're perfectionists with deadlines right? There's no reason to write django-configurations if in five minutes you can get the same result out of the box.

Do you see value in django-configurations, django-debian or django-harness? Then you must see the value in this patch. You can implement all of those in an hour tops including documentation and tests with it.

If you don't, just keep using the default behavior. No harm done.

What exactly is the maintainance burden here?

Instead of saying "Check the DJANGO_SETTINGS_MODULE" you now just need to ask "Have you written a new collector or loader? No? Check the DJANGO_SETTINGS_MODULE".

You want to change how settings are being loaded by default? How does that contradict the use of my patch?

In fact it makes it much easier because now you can just rewrite the collector and loader and you have a new default way of loading settings.

I know there were talks about changing how settings behave and that's why I did something about it.

Am I clear enough now?

2013/3/24 Russell Keith-Magee <rus...@keith-magee.com>

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/1M5nfnpba0M/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.

[Russell Keith-Magee]

On Sun, Mar 24, 2013 at 6:28 PM, Omer Katz <omer...@gmail.com> wrote:

You are contradicting yourself. At first you said that it does make the code clearer. Now you say it doesn't.

My apologies -- I've apparently used an English idiom that doesn't have an obvious meaning. 

When I said "I'd argue the point that it makes the code clearer", that means "You said it was clearer, I disagree, and I'd be willing to defend my position."

I do not believe that your proposed patch makes the code clearer.

Ok, you guys are right. I'm adding a new feature.

If you think my new design isn't good enough do tell me why. I'll improve it/change it completely.

It now doesn't violate SRP, it allows flexability for those who need it and maintains backward competitability. What else is missing here?

I feel a bit like a broken record here because I did specify over and over what's the purpose of this change.

I don't want to load settings from a class. I'd like to provide extensibiloty points in order to help implemet other workflows that people use.

And at this point, I'm also feeling like a broken record. The point that I, and several other people have made repeatedly is that we fail to see how these extensibility points would be used in practice. You haven't demonstrated a use case where what you describe would be helpful.

Saying "It makes it easier to implement django-configurations" isn't a use case. A use case here might be "it makes it easier to define the development/production settings split". 

The purpose of this patch is to:

• Allow developers to implement different workflows for loading settings for different environments (class based settings, different modules, different packages).
• Allow developers to load settings from other sources like django-debian does.
• Allow developers to change settings as they are collected (for example django-harness does multiple things like this).

There are usecases, some people actually implemented those outside of the framework and I don't see a reason why they should. We're perfectionists with deadlines right? There's no reason to write django-configurations if in five minutes you can get the same result out of the box.

Do you see value in django-configurations, django-debian or django-harness?

In short? No. 

Adding an extensibility point is only beneficial if you actually want that particular axis to be extensible. It's good that Django has an extensibility point for database backends -- that allows someone outside core to develop a backend for DB2 or MSSQL. 

However, I don't see the benefit in having an extensible mechanism for defining settings. Settings are settings. There should be *one* way to define them, and that's it. There's *benefit* in there only being one way to define settings, because that means that someone coming to a project for doesn't need to discover, and then learn, about a whole other way of defining the fundamental feature the settings for a Django project.

Are the problems with Django's settings framework? Yes. Are there areas that would benefit from more documented "best practice" conventions? Absolutely. Does any of this require a flexible framework for altering the way Django loads settings? Not as far as I can make out.

Then you must see the value in this patch. You can implement all of those in an hour tops including documentation and tests with it.

If you don't, just keep using the default behavior. No harm done.

What exactly is the maintainance burden here?

There is code that can break. There is a publicly documented interface that, once added, we must support into the future. If we ever want to make a change to the way settings are handled (and we do, if you watch the video I linked), we would also need to support this interface going forward. Once a new feature is added, someone may ask for additional tweaks to that feature. Someone may attempt to use that interface in an unconventional way, forcing us to work out whether that usage is supported or not.

And on top of all of that, we're fracturing the community position on "how do you define settings". Instead of being able to tell someone, "go look in settings.py", we have to say "Go find out what settings loading mechanism the project has, and then consult the documentation on that".

*That* is the maintenance burden.

 

Instead of saying "Check the DJANGO_SETTINGS_MODULE" you now just need to ask "Have you written a new collector or loader? No? Check the DJANGO_SETTINGS_MODULE".

You want to change how settings are being loaded by default? How does that contradict the use of my patch?

In fact it makes it much easier because now you can just rewrite the collector and loader and you have a new default way of loading settings.

I know there were talks about changing how settings behave and that's why I did something about it.

Am I clear enough now? 

What is clear to me is that you would like to use django-configurations (or an analog of it). 

My question to you -- Why do you want to use django-configurations in the first place? What problem is it solving? Then lets focus on solving *that* problem. I'd be very much surprised if the solution involves introducing a pluggable mechanism for loading settings.

Yours,

Russ Magee %-)

[Albert O'Connor]

Hey Omar,

Though it may be clear now from this experience, I feel your expectations about how code becomes a part of Django were incorrect, leading to some frustration.

The onus is on the contributor not the core committers for nearly all aspects of making a contribution, which is likely a difference versus a smaller open source project. The onus is on the contributor to write the patch, the tests and the documentation. Not having enough time isn't really a reason not too, no one is paid to develop Django and no one has much time. Just sharing code that might be useful is a nice gesture, but one should not expect it to be picked up automatically.

The onus is also on the contributor to convince people that the change is worth while. It isn't the job of the core team to argue why a change is not worth making, though they often attempt to do so. The default reaction to change is a conservative one, change must prove why it is useful, not the other way around.

Like any open source project Django relies heavily on all the people who contribute, but in order to make it stable and useful to the huge community who uses, the process of getting change into it is intentionally somewhat of an up hill battle where the onus is on the contributor at every stage, not the project maintainers.

Albert O'Connor

--

You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

><><><>< Albert O'Connor - amjo...@gmail.com
albertoconnor.ca | watpy.ca | go-opendata.ca

[Omer Katz]

2013/3/24 Russell Keith-Magee <rus...@keith-magee.com>

On Sun, Mar 24, 2013 at 6:28 PM, Omer Katz <omer...@gmail.com> wrote:

You are contradicting yourself. At first you said that it does make the code clearer. Now you say it doesn't.

My apologies -- I've apparently used an English idiom that doesn't have an obvious meaning. 

When I said "I'd argue the point that it makes the code clearer", that means "You said it was clearer, I disagree, and I'd be willing to defend my position."

I do not believe that your proposed patch makes the code clearer.

Ok, you guys are right. I'm adding a new feature.

If you think my new design isn't good enough do tell me why. I'll improve it/change it completely.

It now doesn't violate SRP, it allows flexability for those who need it and maintains backward competitability. What else is missing here?

You ignored this point completely. From the design perspective alone, is there something wrong with this patch?

 

I feel a bit like a broken record here because I did specify over and over what's the purpose of this change.

I don't want to load settings from a class. I'd like to provide extensibiloty points in order to help implemet other workflows that people use.

And at this point, I'm also feeling like a broken record. The point that I, and several other people have made repeatedly is that we fail to see how these extensibility points would be used in practice. You haven't demonstrated a use case where what you describe would be helpful.

Saying "It makes it easier to implement django-configurations" isn't a use case. A use case here might be "it makes it easier to define the development/production settings split". 

If the purpose of django-configurations is (also) make it easier to define the development/production settings split then you can easily deduct that if anyone can implement it in 5 minutes it will also be easier to split development/production settings.

django-configurations take the class based approach but you can just as easily load settings from the development package or the production package.

If I want to package a django package for debian it is now very easy to add a source that loads the database settings from dbconfig instead of hacking your way around django.

If I want to get the database configuration from an environment variable or parse it from a database url (like how dj-database-config does) it also make it much easier.

What kind of example do you want? django-configurations, django-debian & django-harness are examples of what I'd like to be able to get almost out of the box.

I want to be able to split development settings from production settings in any way I feel that is contributing to my project.

I want to be able to package it for deb, rpm and other repositories.

I want to be able to load the installed apps from the apps folder by packages and not only by specifying them.

These are just examples of things that people already implemented in a hackish way.

What example do you want exactly?

The purpose of this patch is to:

• Allow developers to implement different workflows for loading settings for different environments (class based settings, different modules, different packages).
• Allow developers to load settings from other sources like django-debian does.
• Allow developers to change settings as they are collected (for example django-harness does multiple things like this).

There are usecases, some people actually implemented those outside of the framework and I don't see a reason why they should. We're perfectionists with deadlines right? There's no reason to write django-configurations if in five minutes you can get the same result out of the box.

Do you see value in django-configurations, django-debian or django-harness?

In short? No. 

And now give me the long version. Why each of these examples are not valid workflows that people want to use?

 

Adding an extensibility point is only beneficial if you actually want that particular axis to be extensible. It's good that Django has an extensibility point for database backends -- that allows someone outside core to develop a backend for DB2 or MSSQL. 

However, I don't see the benefit in having an extensible mechanism for defining settings. Settings are settings. There should be *one* way to define them, and that's it. There's *benefit* in there only being one way to define settings, because that means that someone coming to a project for doesn't need to discover, and then learn, about a whole other way of defining the fundamental feature the settings for a Django project.

The default behavior is still there. Go ahead and use it.

But those who created these projects disagree with you. They want a different way to load settings becuase they have different needs. Why make it hard on them? Newcomers to a project will still need to learn these workflows and how are they different from the normal django behavior.

I have never used feature X of Django but I'm still glad it's there if I need it or because it saves other people some time.

Are the problems with Django's settings framework? Yes. Are there areas that would benefit from more documented "best practice" conventions? Absolutely. Does any of this require a flexible framework for altering the way Django loads settings? Not as far as I can make out.

So you are claiming that these projects exist because they do not know the one true way to handle settings?

There is none. Everyone has different needs. It's like forcing all djangoers to use the same database or the django ORM for that matter.

 

Then you must see the value in this patch. You can implement all of those in an hour tops including documentation and tests with it.

If you don't, just keep using the default behavior. No harm done.

What exactly is the maintainance burden here?

There is code that can break. There is a publicly documented interface that, once added, we must support into the future. If we ever want to make a change to the way settings are handled (and we do, if you watch the video I linked), we would also need to support this interface going forward. Once a new feature is added, someone may ask for additional tweaks to that feature. Someone may attempt to use that interface in an unconventional way, forcing us to work out whether that usage is supported or not.

And on top of all of that, we're fracturing the community position on "how do you define settings". Instead of being able to tell someone, "go look in settings.py", we have to say "Go find out what settings loading mechanism the project has, and then consult the documentation on that".

*That* is the maintenance burden.

 

Instead of saying "Check the DJANGO_SETTINGS_MODULE" you now just need to ask "Have you written a new collector or loader? No? Check the DJANGO_SETTINGS_MODULE".

You want to change how settings are being loaded by default? How does that contradict the use of my patch?

In fact it makes it much easier because now you can just rewrite the collector and loader and you have a new default way of loading settings.

I know there were talks about changing how settings behave and that's why I did something about it.

Am I clear enough now? 

What is clear to me is that you would like to use django-configurations (or an analog of it). 

My question to you -- Why do you want to use django-configurations in the first place? What problem is it solving? Then lets focus on solving *that* problem. I'd be very much surprised if the solution involves introducing a pluggable mechanism for loading settings.

See above.

 

Yours,

Russ Magee %-)

[Omer Katz]

My name is Omer and got it. :)

2013/3/24 Albert O'Connor <amjo...@gmail.com>

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/1M5nfnpba0M/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.

[Jacob Kaplan-Moss]

Hi Omer -

OK, I think it's time for you to drop this. Thanks for your
suggestions, but we're not going to be adding this to Django.

Jacob

> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an