[Proposal] Project specific iex configuration

[Chris Miller]

Currently we use the `dot-iex` file to configure an iex shell.  The `dot-iex` file that gets loaded is the first of these three things that is found
• --dot-iex PATH command line argument supplied to the iex command
• '.iex.exs' file in the directory that iex is run from
• '.iex.exs' file in directory found in the env var "IEX_HOME" OR the users home directory 

The issue I am facing currently is that project level shell configuration is hard to manage in a way that achieves these goals
• will apply configuration when running a shell in the context of a particular (mix) project
• will allow for a developer to apply their own particular customization
• does not require any additional scripts / arguments to start the project

Existing partial solutions for a project level iex configuration

• create and commit a `.iex.exs` file for the project
-- PROS:
   • the file will be loaded when `iex -S mix` is run
   • configuration can be specific to the project as it is part of the source code written for the project and tracked through whatever svc is used
-- CONS:
    • does not allow for an individual developer to include their own configuration as expected (you could add an `import_if_avaiable(".dev.iex.exs")` line to the project level `.iex.exs` file to allow for this extension, but it makes the file name arbitrary and could cause some confusion)
    • will be used when `iex` is run from that directory not in the context of that mix project

• create a project specific configuration file and use the --dot-iex command line arg
-- PROS:
    • does not interfere with running `iex` outside of the context of the mix project
    • can load additional configuration files by include `import_if_avaiable` statements
-- CONS:
    • Requires including the --dot-iex arg when running the `iex -S mix` command, which is prone to being forgotten, this could be wrapped in a very simple start script, but you would still need to remember to run that (I work with a largish number of elixir services and having individual start scripts or args per project can be cumbersome to remember)

I think an ideal solution would be a way to configure a mix project to load a particular configuration file that will be loaded when the IEx.Evaluator starts IN ADDITION to the existing `dot-iex` file options.  I believe this would allow for maintainers of a project to normalize some shell configuration while still allowing developers the full ability to add their own configuration while also keeping the workflow of starting the shell more standardized across projects.

A secondary goal might be that this could be incorporated into releases as well so that the `./bin/project remote` and similar commands could also load some particular configuration

Thanks in advance for any thoughts you had, and if I missed any existing options for this type of configuration, let me know!

[José Valim]

Hi Chris, thanks for writing.

> • will be used when `iex` is run from that directory not in the context of that mix project

The reason this happens is exactly because IEx starts before Mix, so we can't  use Mix to configure IEx. And I think that will get in the way of your proposal too. I hope this helps narrow down a bit the paths to explore.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/a956613f-7ef1-435c-8aaf-ab3af8058d5dn%40googlegroups.com.

[Chris Miller]

Thanks for the reply Jose!  The dependency order makes sense as an issue with my initial thought - one other approach is that a project could define its own method of configuring IEx if there were a hook to allow that - looking at the IEx.Evaluator I was able to make a pretty small patch to allow for this type of configuration by adding a new public function and extending the `loop` function

```elixir
  @spec load_dot_iex(pid, pid, String.t()) :: :ok | :error
  def load_dot_iex(evaluator, server, path) do
    ref = make_ref()
    send(evaluator, {:load_dot_iex, server, ref, self(), path})

    receive do
      {^ref, result} -> result
    after
      5000 -> :error
    end
  end

 ...

  defp loop(%{server: server, ref: ref} = state) do
    receive do
      {:eval, ^server, code, counter, parser_state} ->
        {status, parser_state, state} = parse_eval_inspect(code, counter, parser_state, state)
        send(server, {:evaled, self(), status, parser_state})
        loop(state)

      {:fields_from_env, ^server, ref, receiver, fields} ->
        send(receiver, {ref, Map.take(state.env, fields)})
        loop(state)

      {:value_from_binding, ^server, ref, receiver, var_name, map_key_path} ->
        value = traverse_binding(state.binding, var_name, map_key_path)
        send(receiver, {ref, value})
        loop(state)

      {:variables_from_binding, ^server, ref, receiver, var_prefix} ->
        value = find_matched_variables(state.binding, var_prefix)
        send(receiver, {ref, value})
        loop(state)

      # NEW RECEIVE CASE TO LOAD A DOT IEX FILE PROGRAMMATICALY  
      {:load_dot_iex, ^server, ref, receiver, path} ->
        next_state = load_dot_iex(state, path)
        send(receiver, {ref, :ok})
        loop(next_state)

      {:done, ^server, next?} ->
        {:ok, next?}

      {:done, ^ref, next?} ->
        {:ok, next?}
    end
  end
...

I think that combining this overriding the default mix task would allow for the type of configuration that I was hoping to achieve - or if there is desire for this functionality a small change could be incorporated into mix to do this as a feature using `project.cli()[:iex_configuration_file]` or something of the sort.

If you don't think any of this is necessary in Elixir proper I can move my work towards something at the project level, but wanted to see if there was any interest in upstreaming this concept

[Chris Miller]

Actually - I am a little confused by the startup dependencies between mix and iex as it seems like the Evaluator is starting after mix (or at least that how it appears in my tests), this patch to the IEx.Evaluator seems to accomplish what I was hoping to achieve (but it does make IEx depend on Mix which may have been something you wanted to avoid, but perhaps there is some more abstract way of getting this information from the mix project to the evaluator) 

```elixir
  defp load_dot_iex(state, path) do
    candidates =
      if path do
        [path]
      else
        # Do not assume there is a $HOME
        for dir <- [".", System.get_env("IEX_HOME") || System.user_home()],
            dir != nil,
            do: dir |> Path.join(".iex.exs") |> Path.expand()
      end

    mix_config_file = List.wrap(Mix.Project.get().cli()[:iex_configuration_file])

    candidates
    |> Enum.filter(&File.regular?/1)
    |> Enum.take(1)
    |> Enum.concat(mix_config_file)
    |> Enum.reduce(state, fn path, state ->
      eval_dot_iex(state, path)
    end)
  end
```
Once again - sorry for the noise if this in not a feature you are interested in introducing!

[José Valim]

I wonder if you could call something like "IEx.configure(...)" from the top of your mix.exs and that would be enough to configure its location. Or maybe it would only require a small tweak to make it work.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/085f777f-3b9c-4ef1-8787-d1396c13fc88n%40googlegroups.com.

[Chris Miller]

I hadn't considered the config, but thats an interesting thought - not entirely sure if I totally get your though on the matter, but I was able to extend the IEx.Config to track a new key (:configuration_files) and then extended the IEx.Evaluator to pull that value from the config and load the configured files plus the `.iex.exs` file.  This allows the feature to be used either through a call to `IEx.configure/1` before the evaluator starts, or the config can be added into any config file, which would allow you to easily swap config files per env if desired 

[José Valim]

Yeah, we can add a IEx.configure(dot_iex: "..."), which takes precedence over the other ones if set. And we should read it in the same place we read everything else.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/a01b0032-13ee-45a1-b59b-5320a70e3322n%40googlegroups.com.

[Chris Miller]

I think that would get us pretty close to the behavior I was hoping for - but I was generally hoping that instead of taking precedence over an existing dot-iex configured file it could be used in addition to the existing dot-iex file.  This might be hyper specific to my individual workflow, but I was hoping that we could add a feature to allow for application level configuration while also still using the existing dot-iex files to provide local configuration

An example would be that we would use the new application level configuration to import some helper function and cat out some introduction / instruction to the prompt, while the local would be some functions or data that I as a developer have around for development / debugging but are very specific to my development.

What are your thoughts on this multi-file configuration?

You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-core" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-core/qphuwdQxcsc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JX6KYe1En8WsCQeLhHF8SXnMB7jqps-qcwYKQwNWRimw%40mail.gmail.com.

[José Valim]

That's tricky, because nowhere in IEx ends-up loading multiple dot_iex files, so it would be a departure. But you can always emulate multiple files with import_if_available.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAOZBLdORTXogFXez02uX1NCU1dskL2opqFB9HQRpdGAFNEM8NA%40mail.gmail.com.

[Chris Miller]

> That's tricky, because nowhere in IEx ends-up loading multiple dot_iex files, so it would be a departure.

Very true - while it is technically trivial to load multiple files it is a departure workflow wise and you _could_ end up with some conflicts between the files - that being said, that why I had originally though a different key name would provide a different context for the two things - there could be the existing dot-iex concept that would work the same way as it currently does with the addition of an IEx.Config.configuration_file(s?) that would be an additional file(s) that are loaded either before or after the dot-iex file

> But you can always emulate multiple files with import_if_available.

This is also very true! I think the only potential downside I could think of for this is that it would be slightly more restrictive than the multiple file approach.  If you had a `dot-iex` file configured in your project and then ran `iex --dot-iex other_file.exs -S mix` you may be confused by the fact that your file didn't get loaded as expected

That being said - I defer to your judgement on the best approach here and if this is valuable!
As always, really appreciate your time and all of your work!

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4Kd6NeDoZQ%2Bx6i%3DwNj4yHK0pmoby5ay_CGkQqDcMCiCRg%40mail.gmail.com.

[José Valim]

Let's try starting with a IEx.configure(dot_iex: ...) for now. We could always support making it a list in the future for multiple paths.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAOZBLdOBYmuwaURE0uXNfJa-%2B4hyjbiDjYnkMs5XowXGasNp5g%40mail.gmail.com.

[Chris Miller]

Sounds great to me - and my concern that you couldn't emulate the correct loading a file provided via the command line has be assuaged by realizing you can find the `--dot-iex` arg in `:init.get_plain_arguments` - I'll open up a pr to continue the discussion around implementation there

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2B3mtzv73tnqWo9cg%2BwEVoDgd%2ByC%3DA8RmhN%3Dj%3DVENmk4w%40mail.gmail.com.

[Chris Miller]

PR here https://github.com/elixir-lang/elixir/pull/13647