Include dotenv in Phoenix

[Michael Schaefermeyer]

Hey all,

following the 12 factor app we are using environment variables for all configurations (tokens, hosts, ports, db, ...). For dev setups we rely heavily on dotenv.

In our phoenix apps I am using https://github.com/avdi/dotenv_elixir. The problem is that it can't be used in (mix) configurations due to the order in which applications are being loaded (I believe). It also can't be used in the router, etc.

Is there a chance to include something like dotenv in phoenix core? That would make configuring phoenix apps in the dev environment a lot easier.

I'd be happy to work on this, just wanted to gauge first wether this has relevance for anyone else and a chance of making it into phoenix core.

Thanks,

Michael

[José Valim]

The new router documentation makes it a bit clearer. There is compile time and runtime configuration and it should work if you have the variables set since compile time (and read them in the config file).

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Michael Schaefermeyer]

José,

thanks for the reply. By the way: Posted this in the wrong group, I apologise!

Regarding the topic: The new documentation is indeed very helpful, thank you. Unfortunately dotenv sets variables during runtime. 

I was wondering if dotenv "baked into" phoenix would make sense as I can see others facing the same issues.

Sorry again for picking the wrong group for this.

Thanks

Am Montag, 10. November 2014 12:41:07 UTC schrieb José Valim:

The new router documentation makes it a bit clearer. There is compile time and runtime configuration and it should work if you have the variables set since compile time (and read them in the config file).

On Monday, November 10, 2014, Michael Schaefermeyer <michael.sc...@gmail.com> wrote:

Hey all,

following the 12 factor app we are using environment variables for all configurations (tokens, hosts, ports, db, ...). For dev setups we rely heavily on dotenv.

In our phoenix apps I am using https://github.com/avdi/dotenv_elixir. The problem is that it can't be used in (mix) configurations due to the order in which applications are being loaded (I believe). It also can't be used in the router, etc.

Is there a chance to include something like dotenv in phoenix core? That would make configuring phoenix apps in the dev environment a lot easier.

I'd be happy to work on this, just wanted to gauge first wether this has relevance for anyone else and a chance of making it into phoenix core.

Thanks,

Michael

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[José Valim]

I don't think it makes sense to include dotenv in Mix because Mix already provides a configuration layer which is meant to work with hot code swapping and in general with how we configure applications in OTP.

That said, I don't see the benefit of using dotenv nowadays with the configuration layer in Mix. You can either use System.get_env("MY_VAR") || "value_for_development" in your config/config.exs or define the configuration per environment with config/dev.exs, config/prod.exs and so on.

What am I missing? :)

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

[Paul Schoenfelder]

As an aside, I think we should shy away from recommending the use of System.get_env in config.exs. It doesn't work with releases, and it catches people when they decide to start using them. Granted, if you are not ever going to use releases with your project it doesn't matter, but in general the recommendation is making an assumption about their production environment that probably shouldn't be made.

The best alternative (in my opinion) is to do something like `System.get_env("MY_VAR") || Application.get_env(:myapp, :my_var) || some_default` in the code which requires the configuration. You can provide an environment variable override if desired, but it will use the value in config.exs first, and potentially some fallback value if all else fails, and works whether you use releases or not.

Sorry for derailing the topic!

-- 
Paul

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Schaefermeyer]

José,

it's probably me who's missing something here. :)

I wasn't suggesting including dotenv in Mix. Agreed: Mix provides everything we need for normal OTP configuration.

My suggestion was to include dotenv in phoenix and in the dev environment only. This would give us means to have developer specific configurations in .env files.

Background:

While I like the System.get_env(...) || "default" approach it doesn't work with all configuration scenarios I've encountered. For example credentials, database connections (most noticeably the username in postgres as installed by homebrew) and many other things are developer-specific. We've had good experience having all those configurations in one place: The .env file. It's never checked in but easy to set up (given there is a template). Also the way variables are accessed mimics the way configuration is provided in production environments (System.get_env/1).

The reason for this whole thread is that this .env based development setup doesn't work very well with dotenv_elixir. Even with the config/2 approach system variables injected via the .env file are not accessible using config([path], default).

Maybe I can tackle this writing my own dotenv lib or there is a way to properly use dotenv_elixir. I just had a feeling that others might encounter the same challenges and therefore wanted to raise it here.

Regarding what you wrote, Paul:

Thoughts are always welcome and I don't think you're derailing the discussion. Given that this discussion is about phoenix (which is why I apologised for misfiling it) I do think System.get_env/1 is the way to go. At Bleacher Report tried several different configuration management systems and the one recommended by the 12 factor app is the only one we found to be maintainable. Especially when working with docker environment variables are IMHO the way to go.

Releases are not an issue for us, as new releases always entail new containers. This new container has an isolated environment therefore configuration updates are not an issue.

Maybe I misunderstood you, in which case I am sorry.

I hope this helps to clarify things, please feel free to tell me if I am not making any sense!

Thanks,

Michael

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Eric Meadows-Jönsson]

You can add export to the variables in the env file and depending on your shell load it with just $ . my.env. Now your environment is loaded in the shell and if you have System.get_env("MY_VAR") in your mix config it will work.

​

Eric Meadows-Jönsson

[Paul Schoenfelder]

Releases are not an issue for us, as new releases always entail new containers. This new container has an isolated environment therefore configuration updates are not an issue.

Maybe I misunderstood you, in which case I am sorry.

Hey Michael,

The problem I'm talking about can be illustrated like so: If your local dev machine's config.exs has some config value, say :port, set to the PORT environment variable (via System.get_env), which is 4000 locally but 80 in production, then when you build the release on your machine and deploy to production, the port will be 4000, not 80, as you expect it to be. This is because when you build the release locally, config.exs is evaluated, :port is set to 4000 as per the environment variable, and then sys.config is written with :port set to 4000. sys.config is what releases use for configuration, and can only contain terms, no function calls. This isn't apparent until you run into the issue in production.

This is part of the reason why I built the conform project, since obviously production machines will need configuration that you don't necessarily want (or even can) set via config.prod.exs, and sys.config is not the most ideal configuration file format (great for machines, poor for humans).

-- 
Paul

[Michael Schaefermeyer]

Paul,

Thanks for explaining and sorry it took me so long to understand that you were talking about "packaged apps".

We build our apps in Docker and in the production environment and have them within the Elixir environment. Therefore this is not an issue for us.

As I don't have any experiences with releases I cannot really speak on the matter. However I think your "usual" way to deploy phoenix apps will be Docker or an app server rather then releases. Therefore defaulting to System.get_env/1 still sounds like a good middle ground to me. Even though I don't have any data I would assume that a release-based deployment strategy is a fairly rare case.

Eric thanks for pointing out using a bash script for environment variables. While this works pretty well its yet another thing to remember. We are working with quite a few engineers and in my experience the way to handle environment variables you suggested is pretty error-prone (as it is easy to forget running . .env). Another way of doing this would be to have "start-up scripts" like the following:

script/server

source .env
mix phoenix.start

or

script/console

source .env
iex -S mix

The disadvantage of this approach is (a) I have to remember running the scripts instead of the commands and (b) I need a script for every command (like etco.migrate, etc.).

Ideally we have an automated way of configuring environment variables that is invoked with every command I execute in that directory.

I still hope I am making sense and this is a big enough issue to warrant this discussion. If not please let me know!

Thanks

[Paul Schoenfelder]

Michael,

I wasn't necessarily making this recommendation specifically for your use case, I was more responding to José's recommendation to use System.get_env in your config.exs.

I think you'd be surprised at the number of people using releases, including with Phoenix. I know I've had a number of people already either mention it or report issues that have been addressed. Regardless, how you approach configuration is always highly dependent upon your particular deployment process and the project itself, I just want to make sure that in general we aren't making recommendations without knowing those details, otherwise people will run into issues and not understand why.

Releases have a couple of nice bits that you don't get for free with your Docker setup, which is why people may choose to use them instead, but that's not really relevant to your question, so I'll leave that for another topic/conversation. In any case, it sounds like you have what you need :)

-- 
Paul