[Proposal] Config.config variant that does *not* deep-merge keyword lists

22 views
Skip to first unread message

Hubert Łępicki

unread,
Dec 9, 2021, 9:09:58 AM12/9/21
to elixir-lang-core
I am stumbling at random intervals, but quite predictably every several months or so, on the issue that someone in some project created a bug, because they didn't realize at the moment that changes made in runtime.exs will be deep-merged with config.exs or prod.exs respectively.

The most recent example for me is when someone wanted to remove global_limit from our Oban engine this way:

# config/config.exs
config :core, :background_jbos, queues: [my_queue: [global_limit: 1, local_limit: 1]]

we had this in runtime.exs:

# config/runtime.exs
config :core, :background_jbos, queues: [my_queue: [global_limit: 100, local_limit: 10]]

The person changed it to:
# config/runtime.exs
config :core, :background_jbos, queues: [my_queue: [local_limit: 10]]

i.e. they removed global_limit configuration option, expecting it to lift the limit completely, instead because configs are deep-merged, it used global_lmit of 1.

Obviously everything locked up as the server is quite busy and we had tens of thousands jobs accumulating in every minute O_o.

I am raising the issue because I have seen this enough times that I am thinking a variant of config/2 function that does *not* deep merge with the existing config key, but replaces it completely could be warranted. 

Obviously naming it is hard but something like Config.put_config/2 could work for me I think.

Let me know if that's useful thing to have for other people, maybe it's a singular issue that only my teams stumble upon xD

Jon Rowe

unread,
Dec 9, 2021, 9:15:23 AM12/9/21
to Damir
If we were to add something here, I'd suggest `replace_config` or `replace_all_config` as a name, `put_config` is too similar to what we have already, given that if you were to `put` just one key in a map it would ignore other keys.

Cheers
Jon
--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

Hubert Łępicki

unread,
Dec 9, 2021, 9:17:57 AM12/9/21
to elixir-lang-core
I like replace_all_config a lot. I'll wait for some more votes before I start getting my hands dirty implementing it because I'm not entirely convinced it'll be a welcome change. There's always a trade-off between functionality and maintainability that's hard for me to judge.

José Valim

unread,
Dec 9, 2021, 9:33:52 AM12/9/21
to elixir-lang-core
This is tricky because at what level do you stop deep merging? What if you had other queues? Do you want to delete them? What if you stored it at:

config :core, queues: [my_queue: [global_limit: 1, local_limit: 1]]

Maybe the best would be "delete_config", so you explicitly delete up to a certain nesting? For example:

delete_config [:core, :queues, :my_queue]

If this sound sensible, a PR is welcome.

Hubert Łępicki

unread,
Dec 9, 2021, 9:35:40 AM12/9/21
to elixir-l...@googlegroups.com
Yes, I would very much like delete_config too. It'd do the same for me.



--
Pozdrawiam,
Hubert Łępicki
 -----------------------------------------------
[ http://hubertlepicki.com ]

Jason Axelson

unread,
Dec 9, 2021, 2:02:00 PM12/9/21
to elixir-l...@googlegroups.com
+1 for delete_config for me too

I've used a snippet like this before as a workaround:

        # We need to delete all the existing env because `Application.put_all_env` does a deep merge
        for {key, _val} <- Application.get_all_env(app), do: Application.delete_env(app, key)
        Application.put_all_env([{app, original_env}])

Also an earlier version of that code was incorrect because I forgot that config was always deep merged, which makes `Application.put_all_env([{app, []}]` a no-op which perhaps should be a warning instead.

-Jason

Reply all
Reply to author
Forward
Message has been deleted
0 new messages