Proposal: make Application.get_env being able to resolve {:system, var} tuple out of the box

[Ivan Yurov]

While playing with deployment I found out that some libraries provide this feature that you can put {:system, var} in configuration and then it's resolved at runtime. However if it's in my code, I'd have to implement it on my own. Wouldn't it be nice if it was supported by Application module by default? Something like:

def get_env(app, key, default \\ nil) do
  case :application.get_env(app, key, default) do
    {:system, var} ->
      System.get_env(var) || default
    rest ->
      rest
  end
end

Would it make sense?

[OvermindDL1]

But what if I want to read it from a data store, or the database, or a variety of other places.

A whole configuration handling overhaul has been discussed on the forums not long ago:

https://elixirforum.com/t/proposal-moving-towards-discoverable-config-files/14302

For now, https://hex.pm/packages/confex is a decent'ish shim that works well enough for build-time configurations though.  :-)

But overall, things like that seem more suited for a configuration library (that should be included in Elixir once it is sufficiently generic enough and staged enough), though environment variables are common for the configs, that doesn't mean that someone really doesn't just want `{:system, var}` returned straight though, as can be the case..

:-)

[Ivan Yurov]

I agree that in general this is not something that should be defined in the language core or the standard lib.

However, configuration (application env) is already managed by elixir, and this pattern to access system env var is a widely used by the libraries, so this might appear confusing for someone, that they set {:system, var} and it doesn't magically get resolved as it does with many libraries. :)
Also, there's only one case when it really breaks, — when someone really wants to return {:system, something} on Application.get_env call, as you have pointed out. But why would they?

Thanks, I'll take a closer look at Confex it seems to be doing exactly what I want. Even though there's a bit more of a boilerplate.

[Allen Madsen]

I believe the community is moving away from the {:system, var} pattern, though some libraries will continue to support it. I believe the short term solution being worked on is to allow your config to run at runtime. Which would mean you could just do System.get_env/1. The longer term push is likely to encourage less configuration in config and move more of it into the supervisor inits.

Allen Madsen
http://www.allenmadsen.com

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/b4a347ae-baa1-4d79-b906-4108a414b33c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Paul Schoenfelder]

In my opinion we absolutely should not support this. It is an old convention that was created to patch over problems with configuration in releases, but it has been considered a bad pattern for quite some time now. Not only are there better patterns available (see Phoenix and Ecto and their init/1 callbacks as one example), but this is something that is being addressed in the release tooling itself - currently with Distillery's config provider framework, and perhaps some variation of that, or a better alternative if one presents itself, when the tooling moves to core.

If you have dependencies which still use `{:system, "VAR"}`, you should open issues to encourage maintainers to shift to the `init/1` callback approach, or better, supporting configuration by parameter when possible. The application env should be reserved for global config only.

Just my two cents,

Paul

On Mon, Aug 6, 2018 at 5:33 PM Ivan Yurov <ivan.y...@gmail.com> wrote:

--

[Ivan Yurov]

So the idea is to individually resolve these settings in init callback, am I getting it right?

https://github.com/elixir-ecto/ecto/blob/master/lib/ecto/repo.ex#L69-L75

Well, if I do that, I'll have to use the ENV variable in dev environment as well, which I wouldn't want to do for obvious reasons: these old configs are just code, when env variables are volatile.
And what is the rationale, why {:system, var} is so bad if what happens in init callback is essentially the same? Am I missing something here?

[Paul Schoenfelder]

> So the idea is to individually resolve these settings in init callback, am I getting it right?

https://github.com/elixir-ecto/ecto/blob/master/lib/ecto/repo.ex#L69-L75

> Well, if I do that, I'll have to use the ENV variable in dev environment as well, which I wouldn't want to do for obvious reasons: these old configs are just code, when env variables are volatile.

I think you are missing the fact that the resolved environment variable can be set conditionally, i.e. only when there is _no_ value found in the application environment. You can do whatever you want in that callback. But yes, the idea is that if there is no value found at that point, you can attempt to resolve configuration from the system environment or elsewhere - it becomes part of your application code and lives alongside the thing being configured.

> And what is the rationale, why {:system, var} is so bad if what happens in init callback is essentially the same? Am I missing something here?

It isn't the same - one is implicit, the other explicit, amongst other things - in addition, the behavior of `{:system, var}` is not well defined, it depends on the implementation; there is no way to handle converting values to appropriate internal types (i.e. an atom or an integer) or provide defaults. There is no way of knowing if that tuple is used for other purposes in some application or library somewhere or not - the convention was created by Elixir libraries early on, but there are far more Erlang libraries and Elixir libraries which do not use it than those which do, automatically transforming that tuple to the value of the system environment variable is "magical", and is something we in general try to avoid, particularly in cases where it may cause issues with existing applications.

By putting configuration handling code in the `init` callback, you can handle falling back to multiple sources, perform type conversions, validation, and more, is explicit, and becomes maintained with your other application code. If you are looking for something that would do that sort of conversion for you, you can simply write a thin Config module which checks for that tuple, and resolves the environment variable, and just invoke that module from the init callback instead.

Paul

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/cd78f661-96db-451a-a6e4-ed222c53a379%40googlegroups.com.

[Ivan Yurov]

This is a great point that it would be pretty hard to pass anything but strings. And validation too. What still bothers me is that it might be pretty hard to keep track of settings if they are spread over config.exs and init simultaneously. I'll probably go with this Config module idea, thanks. At least until there's a common solution.

[Paul Schoenfelder]

Personally I look at it the other way around: things are clearer to me when I can see the point at which all configuration values are assembled and then passed down from parent to child, the same you would do with a purely functional program that can only operate on it's inputs. All I generally want to use config.exs for is global, static config (i.e. things which are only likely to change at compile-time, not runtime). Default values should be part of the application code, and expect to be overridden as necessary, rather than using config.exs/application env for that kind of thing. This keeps configuration closer to the code that needs it, and encourages a more functional approach to your design.

Because of that functional approach, it becomes considerably easier to test an Elixir program when you can start a supervision tree, a branch of it, or a leaf (a single process) by simply passing in the configuration it needs, which is going to be part of it's interface. Taking it a step further and allowing one to configure the names of things allows you to start many instances of what would normally be a singleton process, and test many scenarios in parallel - this is particularly handy with stateful things where you need to test different conditions which might conflict with other in-progress tests.

The more libraries that adopt this approach to configuration, the easier it gets to configure and test your whole system. The config providers framework I've made for Distillery is intended to be a way to deal with legacy applications, as well as providing a mechanism for applications which use other mechanisms for configuration (i.e. init-style configs, etcd/Consul, and so on) - but it is not intended to make it easy to put everything in config.exs/application env; I still strongly believe that we need to move away from that as much as possible, not out of ideological purity (which I hope is refuted by the config providers work), but out of knowing that if we can move away from it as a community, things will get much better for all of us as a whole.

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/ed34aba1-e22f-4591-a019-6b11dbc19cea%40googlegroups.com.

[Juan Jose Comellas]

Ivan, as several other people have mentioned already, the Elixir community is slowly moving away from this idiom.

If you still want to use it, though, I wrote a small library that will easily deal with this issue. It's called app_config and you can find the hex package here. The documentation should be self-explanatory. The macros provided by the library create several functions that allow you to retrieve values from the application's environment or from OS environment variables using the same syntax.

Enjoy,

Juanjo

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/cf09772c-0540-49ef-998b-80747436f7bb%40googlegroups.com.

[boris kotov]

Guys, what do you think about the following

Let the config be resolvable by a module

config :app, { :resolve_by, App.Config, [
  key1: "default",
  key2: :other_default
]}

or even just a function

config :app,
  key: fn -> "value" end

For performance reasons, `Applicaiton.get_env` should cache the resolved values at some point, maybe it would be a good idea, to decide when to cache by the returning value from the resolver, like: 

config :app,
  key1: fn -> "dynamic" end, # will be called each time `get_env` is called
  key2: {:ok, "will be called once"} # will be called once by get_env, then it will be cached

Let me know what you think