[Proposal] Conveniences for casting environment variables

[dave.lu...@gmail.com]

In Elixir applications, config/runtime.exs is often used to parse environment variables from System.get_env2/ and friends, and convert into Application configuration.

Since environment variables can only be strings, it is often necessary to cast from string values into booleans and integer values. E.g.

config :my_app, :bool_value, System.get_env("BOOL_VALUE") == "true"

config :my_app, :int_value, String.to_string(System.get_env("INT_VALUE", "5"))

While there are entire libraries that deal with advanced runtime configuration, like Vapor and elixir-specify, I believe there is an opportunity to simplify common patterns and duplicative code in Elixir configuration that often leads to confusion and edge case issues.

My proposal is to add parsing for both booleans and integers to environment variable parsing. As an example:

config :my_app, :bool_value, System.get_env_as!("BOOL_VALUE", "false", as: :boolean)

config :my_app, :int_value, System.get_env_as!("INT_VALUE", "5", as: :integer)

The trickiness would be in the acceptable range of edge case values. For example, "1" and "0" are often used for boolean values, in addition to "TRUE" and "FALSE". I believe there could be a reasonable compromise and appropriate documentation for addressing these issues.

More complex parsing, such as floats and custom data types would not be supported.

[dave.lu...@gmail.com]

A solution to edge case values could be to allow the user to specify the truthy values, e.g.

config :my_app, :bool_value, System.get_env_as!("BOOL_VALUE", "false", as: :boolean, truthy_values: ["on", "true", "TRUE", "FIRE ME UP"])

I could also see separate functions for booleans and integers, but either works

[Cocoa Xu]

This would be quite helpful to deal with environment variables that are expected to be boolean or integers. In my experience, I always have to copy-paste a helper function to do the type cast:

defp to_boolean(nil), do: false

defp to_boolean(var) when is_boolean(var) do
  var
end

defp to_boolean(var) do
  String.downcase(to_string(var)) in ["1", "true", "on", "yes", "y"]
end

[Zach Daniel]

I think a simpler alternative to an `as` option might be:

`System.get_boolean_env/1` and `System.get_integer_env/1` that return things like:

`{:invalid, value}` when the value can’t be parsed, but provide ! versions that raise errors, i.e

```elixir

config :something, port: System.get_integer_env!(“SERVER_PORT”)

```

Raising something like

```elixir

Expected SERVER_PORT to be an integer, got <something>

```

-- 
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/elixir-lang-core/562c5949-fde0-4f3d-9357-d57aa1f9d4b3n%40googlegroups.com.

[Wojtek Mach]

FWIW in config/runtime.exs we can use dependencies and if we happen to have Ecto we can use:

    config :my_app, :bool_value, Ecto.Type.cast!(:boolean, System.get_env(“”, “false”))
    config :my_app, :maybe_int_value, (n = System.get_env("N")) && Ecto.Type.cast!(:integer, n)

What I like about this is instead of having System.get_env AND System.get_env_as, we compose System.get_env with another function that can be used for other things.

That being said this obviously requires Ecto. (Though looking at Ecto.Type, Saul, Norm, and friends, I’m rooting for lightweight data casting/validation library in core one day.) `TRUE`/`FALSE` values are not currently supported. So there’s an argument to be made for something more shell env vars specific so to speak.

-- 
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/elixir-lang-core/bd3ee17d-28fc-4944-89b5-204eab7e6bfcn%40googlegroups.com.

[dave.lu...@gmail.com]

> I think a simpler alternative to an `as` option might be:

> `System.get_boolean_env/1` and `System.get_integer_env/1` that return things like:

> `{:invalid, value}` when the value can’t be parsed, but provide ! versions that raise errors, i.e

I do aree with type specific functions, but I don't agree with adding non-raising variants as this is intended to be an ergonomic improvement to existing usage patterns. If the value is not parseable, it should raise. If you have a more complex usage pattern, use the existing `System.get_env/2` and do your own parsing

[Zach Daniel]

Hm…that makes sense. I’d maintain that it should end in a `!` for clarity though, even if there is no non-raising variant.

To view this discussion visit https://groups.google.com/d/msgid/elixir-lang-core/f3bdb53b-d44b-4559-8cc6-9cdcb203c513n%40googlegroups.com.

[Austin Ziegler]

I recently moved to a new position where we use Dotenvy and `System.get_env!` etc. I developed https://hexdocs.pm/enviable/Enviable.html on my own time and have released it. It's fairly comprehensive—and by fairly I mean extremely—and if there's something missing, I'm open to both requests and pull requests.

It supports:

• boolean
• integer
• float
• atom (and safe atom) also with constraints
• module (and safe module) also with constraints
• Elixir and Erlang code evaluation
• PEM decoding
• baseX decoding (and subsequent casting if required)
• list splitting (and subsequent casting)

I'm sure I’m missing something.

We aren't using it at work yet, but the team is looking forward to when we have a bit of breathing room to make this happen.

-a

To view this discussion visit https://groups.google.com/d/msgid/elixir-lang-core/A281A8D1-A3EB-43B7-AB76-9B8863DD32D6%40gmail.com.

--

Austin Ziegler • halos...@gmail.com • aus...@halostatue.ca
http://www.halostatue.ca/ • http://twitter.com/halostatue

[Austin Ziegler]

I also posted about it on Elixir Forum: https://elixirforum.com/t/enviable-robust-environment-variable-conversion/69360

[dave.lu...@gmail.com]

Thanks Austin, this is quite comprehensive, but way beyond the scope of what I am proposing, which is to handle the two most common cases. For anything sufficiently complex, this should be the responsibility of the application/library.

[Jon Rowe]

Having developed my own library to do this for internal projects I'd like to +1 this proposal for the simple cases, although thanks to this discussion I've now learnt about enviable and I'll be using that for any future complex cases!

Cheers

Jon

To view this discussion visit https://groups.google.com/d/msgid/elixir-lang-core/ea6e6238-edac-4dff-83bf-3ca60e30d1d1n%40googlegroups.com.

[Tomasz Kowal]

We've run into the same problem at Bluecode and decided to provide an escape hatch with generic parser.

```

def get_env(key, default, parser \\ &string/2, parser_opts \\ []) do

  if val = System.get_env(key) do

    case parser.(value, opts) do

      {:ok, parsed} -> parsed
      {:error, reason} -> default

  else

    default

  end

end

```

where parser is a fun `parser(String.t() :: {:ok, term()} | {:error, term()}`

We've found it super powerful because:

a) it is easy to define custom parsers

```

def get_boolean(key, default) when default in [true, false] do

  get_env(key, default, &boolean/2, [])

end

def boolean(value, _) do

  value = String.to_atom(value)

  if value in [true, false] do

    {:ok, value}

  else

    {:error, "not a boolean"}

end

```

The cool part is that the decision how to parse the variable is completely up to implementer of the use case.

You have "0" and "1" in the config? Just match that in the parser.

We are using the final `opts` to pass to parser, e.g. for atom parser, we add `allow: [:value1, :value2]` and then we can reject unknown.

`get_atom(key, default, allow: [:value1, :value2])`

Actually, the library we have is not general purpose. E.g. we do some logging in those get_envs to identify if variable is missing or parsing went wrong and with what error.

We also have a bunch of opinionated things in the library.

E.g. one of the things that bothered us with configs was that it was easy to add something in test to make tests pass and then forget in dev.

We decided to move everything to `runtime.exs` and added a convention that variable `FOO=bar` will be used in both dev and test, unless you override it for test with `TEST_FOO=baz`. Then in dev FOO=bar and in test FOO=baz.

Anyway, my point is: I wouldn't add specific cases like boolean and integer to standard library. This raises a question: why not atoms or modules or pids or all other basic and complex cases. Adding optional parser and parsing options would be very generic and accommodate for any usecase anyone can imagine without burdening standard library with decisions such as "should string "0" be false or not be a boolean at all".