Dependency/Library as part of umbrella?

[Brandon Gillespie]

I'm in the process of breaking our larger/monolithic app into separate repos so we can bring the pieces in as dependencies on other apps.

However, a key problem we are running into is ultimately around the database models.

We're working on a separate thing around most of that, but at the heart the problem comes in that we are one level too far removed, so a LOT of warnings appear during compile, even though they aren't relevant at runtime — notably stuff like below on the dependency compile.

warning: Core.Repo.transaction/2 is undefined (module Core.Repo is not available or is yet to be defined)
  lib/rivet/ecto/collection/general.ex:21: Rivet.Ident.Action.full_table_scan/2

I've looked at how oban handles it, and they setup an inefficient layer of indirection (obfuscation) to hide the warning.

The problem I have is this is what I'd call a developer vanity change. It serves NO purpose other than to hide a compiler warning, but the side-effect is less efficient code during runtime.

The resulting code with a warning includes the atom for the repo module, and everything works fine. The warnings are legitimate in the myopic context of compiling ONLY the dependency, but are not correct when considering the complete application.

I could perhaps add something like `[xref: [exclude: [Core.Repo]]]` to the dependent library's project data/mix.exs, and that might be fine for a closed-world where we don't share the library. But we are considering opening the library to the public, so this isn't a valid solution.

I feel like this same problem is resolved with umbrella applications.

Which makes me wonder, perhaps there could be a way to treat an external dependency like a member of the umbrella, and inject stuff like the xref in at the deps() array level, rather than from within that project?

While I'm not a fan of premature optimization, but I do care about keeping in mind the things that will have a potential of high-frequency calls, and avoiding doing anything obviously dangerous with those, especially over just a vanity thing.

In this case the warning isn't a real problem, it's just something the compiler thinks could be a problem.

Is there some other way to do this short of putting in a less efficient abstraction layer just to make it so the compiler can't detect what's really happening?

I'm just throwing mud on the wall now but something like:

{:included_thing, "~> 1.2.3", xref: [exclude: [Core.Repo]]},

Ultimately, the problem I have with all of the hereto offered solutions is that they effectively boil down to "obfuscate your code enough that the compiler won't throw a warning."

But that's seriously misaligned in my opinion.

The goal of coding should be (a) creating simple but efficient code to execute in production that is (b) readable and maintainable.  These obfuscation/vanity recommendations violate both of these points.

I don't know what the right solution is. I just feel like there should be something better than what's there now.

-Brandon

[José Valim]

Can you please provide a small app that reproduces the issue?

Even if the module comes from a transitive dependency, Elixir should be able to see it and avoid the warning.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/4b127f8b-ddcb-b25e-0e7c-12af3af68252%40cold.org.

[Austin Ziegler]

1. This is from code that last ran on Elixir 1.10 (it is currently offline with no plan to bring it back online).
2. The dependencies include Phoenix (1.5.4) and phoenix_html (2.14.2). It’s referring not to MyApp.Repo, but to Phoenix.View.
3. Phoenix and phoenix_html were compiled after bamboo.
4. The warning that I think is being discussed is a compile warning for the library dependency, not for the hosting application, which matches what I have seen.

```

==> bamboo
Compiling 26 files (.ex)
warning: Phoenix.View.render_to_string/3 is undefined (module Phoenix.View is not available or is yet to be defined)
Found at 2 locations:
  lib/bamboo/phoenix.ex:313: Bamboo.Phoenix.render_html/2
  lib/bamboo/phoenix.ex:323: Bamboo.Phoenix.render_text/2

```

In older releases — maybe in an earlier version of Oban, possibly something in Sentry  — I have seen a warning like this (module MyApp.Repo is not available or is yet to be defined). It’s also not clear to me that if this app were up to date with bamboo (which now has bamboo_phoenix), I would receive a similar warning.

It doesn’t matter much because `mix deps.compile` doesn’t have a `--warnings-as-errors` flag, but I understand those who wish their deps.compile were warning-free.

-a

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4LPM3tniuop5VivZZTEoFCMZuM-vimv1U9yF4tvDy2-NA%40mail.gmail.com.

--

Austin Ziegler • halos...@gmail.com • aus...@halostatue.ca
http://www.halostatue.ca/ • http://twitter.com/halostatue

[Brandon Gillespie]

On 3/10/23 3:33 PM, Austin Ziegler wrote:

1. The warning that I think is being discussed is a compile warning for the library dependency, not for the hosting application, which matches what I have seen.

This.

I'm working on creating a focused way of reproducing it, but need some time to get it all dialed in.

Essentially though, you are right.

What we are working on is a way to put Ecto models into a library dependency that can be brought into other applications. But for this to work, we have to abstract the definition of Ecto.Repo and defer it to runtime. However, because dependencies are pre-compiled, you then get these warnings.

This is a use case that's very compelling, and yet it goes against how things work right now with the way dependencies are, but that doesn't remove the value of having something like this.

oban is a great example.

-Brandon

[Zach Daniel]

This does actually strike a chord for me and I'm realizing this is likely a related question or perhaps the same question in another form. Is there some way to define a module that a dependency can use at compile time or at runtime without these warnings? Like a module that you want compiled before  a specific dependency/dependencies?

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/194c5d9c-6ba3-6599-0436-caae01bec86c%40cold.org.

[Brandon Gillespie]

That's where I was suggesting something on the deps line that could be inserted into the dependency's spec in mix.exs, like:

{:included_thing, "~> 1.2.3", xref: [exclude: [Core.Repo]]},

Where the xref is already a thing in a projects mix.exs for this very problem.

To better explain.

Project-A defines module "MyProjectNarf", and sets an application config `config :narf, external_module: MyProjectNarf`

Project-A brings in Dependency-B, which has an Application.compile_env(:narf, :external_module)

Right now when Dependency-B is compiled, it throws this warning.

If I add to Dependency-B's project spec `xref: [exclude: [MyProjectNarf]]` then the warning doesn't happen.

But this isn't workable because MyProjectNarf is meant to be the variable that can change depending on the outer project importing the dependency.

Yet if I could inject it from Project-A's spec where it imports the dependency...

-Brandon

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/lf3c11ws.8e05e338-daf8-44e1-aca4-5e4d26f470d0%40we.are.superhuman.com.

[José Valim]

Hi Austin and Zach,

For the Bamboo Phoenix case, they should declare phoenix_view as an optional dependency. Then either conditionally compile code or use "@compile {:no_warn_undefined, Phoenix.View}" in the relevant module. Note Elixir v1.14 introduce a --no-optional-deps flag to help library authors test that their projects work without optional dependencies.

Zach, if a module is configured and seen only at runtime, then no warning is emitted. If a module is configured at compile time, then you can also include it as part of xref. For example, imagine you have this:

config :some_dep, compile_time_module: MyApp.SomeModule

This means :some_dep can do this in its "mix.exs":

xref: [exclude: List.wrap(Application.get_env(:some_dep, :compile_time_module))]

Or, perhaps even better, you can do this in the file that uses the compile time attribute:

module = Application.compile_env(:some_dep, :compile_time_module)

@compile {:no_warn_undefined, module}

On Sat, Mar 11, 2023 at 3:16 AM Zach Daniel <zachary....@gmail.com> wrote:

This does actually strike a chord for me and I'm realizing this is likely a related question or perhaps the same question in another form. Is there some way to define a module that a dependency can use at compile time or at runtime without these warnings? Like a module that you want compiled before  a specific dependency/dependencies?

On Fri, Mar 10, 2023 at 9:11 PM, Brandon Gillespie <bra...@cold.org> wrote:

On 3/10/23 3:33 PM, Austin Ziegler wrote:

1. The warning that I think is being discussed is a compile warning for the library dependency, not for the hosting application, which matches what I have seen.

This.

I'm working on creating a focused way of reproducing it, but need some time to get it all dialed in.

Essentially though, you are right.

What we are working on is a way to put Ecto models into a library dependency that can be brought into other applications. But for this to work, we have to abstract the definition of Ecto.Repo and defer it to runtime. However, because dependencies are pre-compiled, you then get these warnings.

This is a use case that's very compelling, and yet it goes against how things work right now with the way dependencies are, but that doesn't remove the value of having something like this.

oban is a great example.

-Brandon

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/194c5d9c-6ba3-6599-0436-caae01bec86c%40cold.org.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/lf3c11ws.8e05e338-daf8-44e1-aca4-5e4d26f470d0%40we.are.superhuman.com.