Configuration

[Jeff Lindsay]

In talking to Bulat over the weekend, I've become distressed on figuring out the ideal configuration input. 

What we have now is simple: A simple python file.

However, Bulat is strongly pushing for what he considers best practice (and is in fact described in 12-factor apps [1]) which is to just use environment variables. 

I mostly agree with him, but I have these issues:

1) It will introduce string parsing. No more native types, it's all strings. Any Setting that's not a string will have to provide a means to parse it.

2) Configuration is no longer dynamic. No loading configuration from other configuration files, querying servers, or anything like that. 

3) Environment for lots of configuration? Most of our apps to date have a fair amount of configuration. This means you'll be creating a bash script file anyway to set up your configuration.

It's also now a single flat namespace, no programming constructs to bring to configuration. Python files are also the favorite (not most popular) form of configuration in the Python world, perhaps we are all already biased. Django, Gunicorn, etc.

Okay, thoughts? Bulat, anything to add in favor of env vars?

[1] http://www.12factor.net/

--
Jeff Lindsay
http://progrium.com

[Bulat Shakirzyanov]

12 factor states some good reasons, I just find it natural, since, all configuration already comes from strings, be it a path to config file that populates some configuration object or command line arguments.

--
Bulat Shakirzyanov | Software Alchemist

a: about.me/avalanche123

e: malll...@gmail.com

[Chad Etzel]

¿porqué no las dos?

i've never really been a fan of using env vars for config... it seems mystical and creates more work (like writing those bash scripts) when deploying stuff in an automated setting. so, if we're going to have to write a file to auto-config the service/app when it boots, why not have it in the native language?

but, maybe you can have both. have the .cfg.py file read in env vars for each setting, and fall back to a default if they don't exist... that way you could override settings with env vars if you really wanted.

dunno, relying solely on env vars for config feels old and busted, but perhaps letting them override file-based settings feels alright.

-chad

[Sean McQuillan]

I'm generally not a fan with env based configurations, with the exception of overrides.  The base configuration is almost a collection of constants, and it makes sense to put it in a same-language configuration file.  Also, in the situations where you need to be clever, you can in a language you're comfortable with.  With env variables you're being clever in bash.

A good discussion is if our override procedure should change.  Currently we allow -X"python code here" to force an override of an env variable.  Implicit overrides seem like a bad idea, especially since we're using a lot of config options like "port" that would need to be namespaced somehow.  Explicit overrides maybe make sense?

Sean

[Jeff Lindsay]

A good discussion is if our override procedure should change.  Currently we allow -X"python code here" to force an override of an env variable.  Implicit overrides seem like a bad idea, especially since we're using a lot of config options like "port" that would need to be namespaced somehow.  Explicit overrides maybe make sense?

What do you mean by explicit overrides? 

Also, "port" can always be namespaced in name: "http_port", or "(some function)_port". 

-jeff

[Chad Selph]

I really like the python file based configuration for the case of having a dev config override certain parts of the base config.  I'd much rather write this code in Python than in Bash; especially if I'm doing things like dicts, lists, etc.

Maybe we could add some "syntactic sugar" for having env variables in the .conf.py files.

# c2c.conf.py

from gservice.config import env

MY_CONFIG = env("MY_CONFIG")

which maybe isn't that much better than simply

# c2c.conf.py

import os

MY_CONFIG = os.environ.get('MY_CONFIG')

finally, we could try to do something like what chetzel wants, with the "Fallback" behavior with a gservice.core.Service subclass that falls back to an env variable:

from gservice.core import EnvSetting

class MyService(Service):

    myvar = EnvSetting("MY_CONFIG")

--
Chad Selph

Software Engineer

Twilio - http://www.twilio.com

[Bulat Shakirzyanov]

Hey guys,

I was telling this to Jeff during our Portland trip, but won't hurt to reiterate here.

I think gservice library would standout if solved harder problems than making development fast and easy (although it should do it too).

Namely, configuration management and deployment and also service discovery and building distributed applications as a bonus feature.

Something like virtualenv which I'm sure you all are more familiar with than myself makes it very easy to interchange environment

configuration just like regular configuration files - http://www.virtualenv.org/en/latest/index.html#environment-variables-and-configuration-files

Additionally, something Ruby's RVM does is let you have project specific settings that get auto-sourced if you put them in .rbenv or something like that

and have that file in your project's root. Which this article describes how to do for python - http://www.redslider.net/2011/2011-11-22-auto-source-virtualenv-settings.html

The way environment works is quite well defined and all rules for overrides are known. File is an external resource, involves filesystem,

just like a database connection.

Cheers!

On Mon, Feb 27, 2012 at 11:59 AM, Sean McQuillan <se...@twilio.com> wrote:

[Bulat Shakirzyanov]

Pardon, filename RVM uses is .rvmrc, example here - https://github.com/imathis/octopress/blob/master/.rvmrc

Above example forces ruby version 1.9.2 to be used.

[Bulat Shakirzyanov]

I wasn't clear that .rvmrc is just a shell script.

Cheers!