Provide a way to remove runtime dependencies, similar to `expand_literal/2`
95 views
Skip to first unread message
Zach Daniel
Sep 11, 2022, 1:55:51 PM9/11/22
to elixir-lang-core
In Ash Framework, we have declarative ways to construct runtime behavior using behaviors. So an Ash resource might look like this:
```elixir
defmodule MyApp.User do
 use Ash.Resource
 alias MyApp.User.Changes.HashPassword
 attributes do
  uuid_primary_key :id
  ....
 end
 actions do
  create :register do
   change HashPassword
  end
 end
end
```
However, by default, this would incur a compile time dependency. This compile time dependency is unnecessary, as we won't call any functions on this module or use it in any way until runtime.
That optimization is well and good, but due to transitive compile time dependencies, we see some interesting behavior. Something you'd often see in a change module is things like pattern matching on other resources, or the resource in question in function heads. Resources are meant to be introspectable at compile time, and so this runtime dependency on a change, with a compile time dependency on a resource, incurs a transitive compile time dependency. This problem multiplies over time, and causes users to have to do things solely to optimize compile times, like only use map pattern matches instead of struct pattern matches.
So what we do is we actually disable the lexical tracker when accepting certain parts of the DSL. This prevents *any* dependency. Naturally, at compile time you are no longer safe to call a resource's change module as changes in that module won't cause recompiles, but that was never a thing you should have done in the first place so I'm not worried about that.
This leads us to the primary issue: disabling the lexical tracker when expanding aliases also causes warnings about unused aliases, even though they *are* used. I believe I've brought this issue up before and we were hoping that the feature introduced in 1.14 for `defimpl` would help, but that only helps prevent compile time issues, which is something I had already solved for in the same way it was solved for 1.14. I've laid it all out to help clarify exactly *why* I need it so perhaps someone can point me in a better direction.
defmodule MyApp.User do
 use Ash.Resource
 alias MyApp.User.Changes.HashPassword
 attributes do
  uuid_primary_key :id
  ....
 end
 actions do
  create :register do
   change HashPassword
  end
 end
end
```
However, by default, this would incur a compile time dependency. This compile time dependency is unnecessary, as we won't call any functions on this module or use it in any way until runtime.
That optimization is well and good, but due to transitive compile time dependencies, we see some interesting behavior. Something you'd often see in a change module is things like pattern matching on other resources, or the resource in question in function heads. Resources are meant to be introspectable at compile time, and so this runtime dependency on a change, with a compile time dependency on a resource, incurs a transitive compile time dependency. This problem multiplies over time, and causes users to have to do things solely to optimize compile times, like only use map pattern matches instead of struct pattern matches.
So what we do is we actually disable the lexical tracker when accepting certain parts of the DSL. This prevents *any* dependency. Naturally, at compile time you are no longer safe to call a resource's change module as changes in that module won't cause recompiles, but that was never a thing you should have done in the first place so I'm not worried about that.
This leads us to the primary issue: disabling the lexical tracker when expanding aliases also causes warnings about unused aliases, even though they *are* used. I believe I've brought this issue up before and we were hoping that the feature introduced in 1.14 for `defimpl` would help, but that only helps prevent compile time issues, which is something I had already solved for in the same way it was solved for 1.14. I've laid it all out to help clarify exactly *why* I need it so perhaps someone can point me in a better direction.
The simplest thing that could help:
A way to tell the lexical tracker that an alias has just been referenced without inducing any kind of compile or runtime dependency. The idea is to just prevent it from warning about the alias.
I'm open to other solutions as well.
A way to tell the lexical tracker that an alias has just been referenced without inducing any kind of compile or runtime dependency. The idea is to just prevent it from warning about the alias.
I'm open to other solutions as well.
Zach Daniel
Sep 11, 2022, 1:57:21 PM9/11/22
to elixir-lang-core
For clarity, the dependency I'm talking about there is the dependency from `MyApp.User` to `MyApp.User.Changes.HashPassword`.
José Valim
Sep 11, 2022, 2:31:42 PM9/11/22
to elixir-lang-core
Sorry, I don't understand the proposal. You mentioned expand_literal, which already removes the compile-time dependency but keeps the remaining functionality such as warnings. Can you please expand?
--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/54627973-7b74-47d7-9e35-4270621e6c91n%40googlegroups.com.
Zach Daniel
Sep 11, 2022, 2:52:01 PM9/11/22
to elixir-lang-core
`expand_literal` removes the compile time dependency, but leaves a runtime dependency when used inside of a module.
What I'm trying to do is remove both the compile time dependency *and* the runtime dependency, without requiring the use of `warn: false` on aliases.
What I'm trying to do is remove both the compile time dependency *and* the runtime dependency, without requiring the use of `warn: false` on aliases.
José Valim
Sep 11, 2022, 2:55:54 PM9/11/22
to elixir-lang-core
Why do you want to remove the runtime dependency when, per your description:
> In Ash Framework, we have declarative ways to construct runtime behavior
using behaviors.
Emphasis mine. If, at any moment, you call any function from that module at runtime, you must not remove the compile time dependency.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/22887a7e-a1b6-4ccc-98bf-69a5ad3551a5n%40googlegroups.com.
José Valim
Sep 11, 2022, 2:56:24 PM9/11/22
to elixir-lang-core
Sorry, correction: If, at any moment, you call any function from that module at runtime, you must not remove the runtime time dependency.
Zach Daniel
Sep 11, 2022, 2:59:59 PM9/11/22
to elixir-l...@googlegroups.com
So all we we do is hold onto the module, and then at runtime we go through the list of modules that we should call and call a specific function on them. Requiring a runtime dependency for that is causing really slow compile times because of transitive dependencies. Maybe there is some consequence I don't see, but I removed the runtime dependencies by disabling the lexical tracker when expanding the alias, and its been that way for months w/o anyone reporting any issues with that implementation. Aside from having to use `warn: false` if they use aliases.
To me, its the same as if they gave us, instead of a module, an `atom` that referred to application configuration, i.e the adapter pattern. That would work without a runtime dependency, so why couldn't this?
To me, its the same as if they gave us, instead of a module, an `atom` that referred to application configuration, i.e the adapter pattern. That would work without a runtime dependency, so why couldn't this?
On Sun, Sep 11, 2022 at 2:56 PM, José Valim <jose....@dashbit.co> wrote:
Sorry, correction: If, at any moment, you call any function from that module at runtime, you must not remove the runtime time dependency.
On Sun, Sep 11, 2022 at 8:55 PM José Valim <jose.valim@dashbit.co> wrote:
Why do you want to remove the runtime dependency when, per your description:> In Ash Framework, we have declarative ways to construct runtime behavior using behaviors.Emphasis mine. If, at any moment, you call any function from that module at runtime, you must not remove the compile time dependency.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/22887a7e-a1b6-4ccc-98bf-69a5ad3551a5n%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2BVkL%2B1V7LTDVyzhCqcNWrmHFPoWx2Fp916Ur%2ByL%2BiVBA%40mail.gmail.com.
José Valim
Sep 11, 2022, 3:05:18 PM9/11/22
to elixir-lang-core
The issue is in the transitive compile time dependencies, not the runtime dependencies.
I don't think we should be encouraging removing the runtime dependencies when they are explicitly listed in the code as above. When done via configuration, at least you are not literally listing it.
On Sun, Sep 11, 2022 at 8:59 PM Zach Daniel <zachary....@gmail.com> wrote:
So all we we do is hold onto the module, and then at runtime we go through the list of modules that we should call and call a specific function on them. Requiring a runtime dependency for that is causing really slow compile times because of transitive dependencies. Maybe there is some consequence I don't see, but I removed the runtime dependencies by disabling the lexical tracker when expanding the alias, and its been that way for months w/o anyone reporting any issues with that implementation. Aside from having to use `warn: false` if they use aliases.
To me, its the same as if they gave us, instead of a module, an `atom` that referred to application configuration, i.e the adapter pattern. That would work without a runtime dependency, so why couldn't this?
On Sun, Sep 11, 2022 at 2:56 PM, José Valim <jose....@dashbit.co> wrote:
Sorry, correction: If, at any moment, you call any function from that module at runtime, you must not remove the runtime time dependency.
On Sun, Sep 11, 2022 at 8:55 PM José Valim <jose....@dashbit.co> wrote:
Why do you want to remove the runtime dependency when, per your description:> In Ash Framework, we have declarative ways to construct runtime behavior using behaviors.Emphasis mine. If, at any moment, you call any function from that module at runtime, you must not remove the compile time dependency.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/22887a7e-a1b6-4ccc-98bf-69a5ad3551a5n%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4%2BVkL%2B1V7LTDVyzhCqcNWrmHFPoWx2Fp916Ur%2ByL%2BiVBA%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/l7xp6eut.444ff913-f528-4b88-805a-cac120cdb4d6%40we.are.superhuman.com.
Zach Daniel
Sep 11, 2022, 3:08:34 PM9/11/22
to elixir-lang-core
Hm...yeah, that makes sense. Are there other things a runtime dependency is meant to do aside from ensure that transitive compile time dependencies are properly tracked? If so, is there a way to solve for the transitive dependencies without removing the runtime dependency? Because ultimately the idea here is that this module is essentially just a big validated/extensible configuration. The resource itself doesn't do anything. So requiring things that refer to a resource to recompile when any of the modules it refers to in a `change` like that is unnecessary. However we can express that reality is totally fine with me.
Zach Daniel
Sep 13, 2022, 6:49:18 PM9/13/22
to elixir-l...@googlegroups.com
Although the solution I originally proposed may not be correct (totally fine with me 
), I think the problem statement is still valid. Can we agree that there are some cases where you may want to reference a module without creating transitive compile time dependencies, i.e in the case of a DSL like what Ash provides?
), I think the problem statement is still valid. Can we agree that there are some cases where you may want to reference a module without creating transitive compile time dependencies, i.e in the case of a DSL like what Ash provides?
On Sun, Sep 11, 2022 at 3:08 PM, Zach Daniel <zachary....@gmail.com> wrote:
Hm...yeah, that makes sense. Are there other things a runtime dependency is meant to do aside from ensure that transitive compile time dependencies are properly tracked? If so, is there a way to solve for the transitive dependencies without removing the runtime dependency? Because ultimately the idea here is that this module is essentially just a big validated/extensible configuration. The resource itself doesn't do anything. So requiring things that refer to a resource to recompile when any of the modules it refers to in a `change` like that is unnecessary. However we can express that reality is totally fine with me.
On Sunday, September 11, 2022 at 3:05:18 PM UTC-4 José Valim wrote:
The issue is in the transitive compile time dependencies, not the runtime dependencies.I don't think we should be encouraging removing the runtime dependencies when they are explicitly listed in the code as above. When done via configuration, at least you are not literally listing it.On Sun, Sep 11, 2022 at 8:59 PM Zach Daniel <zachary....@gmail.com> wrote:So all we we do is hold onto the module, and then at runtime we go through the list of modules that we should call and call a specific function on them. Requiring a runtime dependency for that is causing really slow compile times because of transitive dependencies. Maybe there is some consequence I don't see, but I removed the runtime dependencies by disabling the lexical tracker when expanding the alias, and its been that way for months w/o anyone reporting any issues with that implementation. Aside from having to use `warn: false` if they use aliases.
To me, its the same as if they gave us, instead of a module, an `atom` that referred to application configuration, i.e the adapter pattern. That would work without a runtime dependency, so why couldn't this?
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/072bb99d-09a0-4567-a934-f8893015dd91n%40googlegroups.com.
José Valim
Sep 13, 2022, 6:51:37 PM9/13/22
to elixir-lang-core
As I mentioned, the issue on transitive compile-time dependencies are the compile-time deps, not the runtime ones. So I would focus on how to eliminate those. Otherwise I am not sure we will agree on the problem statement. :)
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/072bb99d-09a0-4567-a934-f8893015dd91n%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/l80s9s37.13dfe243-83b3-4133-8aad-7a86910f56fa%40we.are.superhuman.com.
Zach Daniel
Sep 13, 2022, 7:37:24 PM9/13/22
to elixir-l...@googlegroups.com
Well, its runtime dependencies that create transitive compile time dependencies, isn't it? So if I have a function like
defmodule Foo do
 # callers are aware that they can't use the module at compile time
 def returns_a_module() do
   SomeSpecificModule
 end
 def something_used_at_compile_time() do
  10
 end
end
# and then some other module
defmodule Bar do
 @something Foo.something_used_at_compile_time()
end
 def something_used_at_compile_time() do
  10
 end
end
# and then some other module
defmodule Bar do
 @something Foo.something_used_at_compile_time()
end
This induces a transitive compile time dependency from Bar to SomeSpecificModule.
In the case of Ash DSLs, for example, this happens because we store the resulting configuration in a module attribute. Users are aware that you can't call these pluggable modules at compile time. Only the framework code calls those modules. The above example is contrived, of course, I'm not suggesting that we need a feature to make *that* work, just trying to draw some kind of parallel.
I don't see a reasonable way to handle this without essentially removing the ability to use modules in the way that we are. It would be a pretty unreasonable amount of work/change for users to change the way that we plug behavior in Ash.
To me, another kind of module dependency, like `runtime-only`, would solve for this, and it would have to be explicitly requested, i.e `expand_literal(.., runtime_only: true)`. Then when determining transitive compile time dependencies, the compiler would not use those modules.
In the case of Ash DSLs, for example, this happens because we store the resulting configuration in a module attribute. Users are aware that you can't call these pluggable modules at compile time. Only the framework code calls those modules. The above example is contrived, of course, I'm not suggesting that we need a feature to make *that* work, just trying to draw some kind of parallel.
I don't see a reasonable way to handle this without essentially removing the ability to use modules in the way that we are. It would be a pretty unreasonable amount of work/change for users to change the way that we plug behavior in Ash.
To me, another kind of module dependency, like `runtime-only`, would solve for this, and it would have to be explicitly requested, i.e `expand_literal(.., runtime_only: true)`. Then when determining transitive compile time dependencies, the compiler would not use those modules.
On Tue, Sep 13, 2022 at 6:51 PM, José Valim <jose....@dashbit.co> wrote:
As I mentioned, the issue on transitive compile-time dependencies are the compile-time deps, not the runtime ones. So I would focus on how to eliminate those. Otherwise I am not sure we will agree on the problem statement. :)
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/072bb99d-09a0-4567-a934-f8893015dd91n%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/l80s9s37.13dfe243-83b3-4133-8aad-7a86910f56fa%40we.are.superhuman.com.
--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4J8qxM%3DKxMUUW6X%2Bt%2BrD9rKn0yDHbDY1fDgopxRvtNn5A%40mail.gmail.com.
Marlus Saraiva
Sep 16, 2022, 8:43:28 AM9/16/22
to elixir-lang-core
Hi Zach!
The only way you can minimize the transitive dependency problem is by removing the compile-time deps. I struggled with this for years with Surface, and as you did, I tried the approach of removing the runtime deps to avoid them from turning into transitive compile-time deps. Eventually, I realized that avoiding compile-time deps would be the only way to have a permanent solution and started removing any implementation that required information at compile-time. As soon as LV v0.8 is out, we'll wrap up that work, so we don't expect further issues after that. In the meantime, to solve the problem, I added a fix that drastically cut down those dependencies, which was to convert the `compile` deps into `export` deps, by using `import` instead of `require`. For this to work properly, I have to generate and automatically rename a signature function to force the recompilation of direct callers whenever the component's metadata (props, slots, etc) changes. Otherwise, it would only recompile them if you added or removed another function.
It seems to me that Elixir could provide a way to hook into the compiler allowing DSL developers to inform a criteria that could be used to trigger callers' recompilation. Similar to `__mix_recompile__?`. Maybe something like `__export_changed__?` or anything else. With this approach, DSL authors could replace the `compile` deps with `export` deps as long as the changes in that dependency don't need to be propagated to other modules other than the callers. This is the case for Surface and many other libs, like Commanded, for instance. I'm not sure the proposed solution would solve your issue with Ash but I thought it was worth bringing it here since it may solve issues on other libs.
José, is something like that feasible?
On Tuesday, September 13, 2022 at 8:37:24 PM UTC-3 zachary....@gmail.com wrote:
Well, its runtime dependencies that create transitive compile time dependencies, isn't it? So if I have a function like
defmodule Foo do
 # callers are aware that they can't use the module at compile time def returns_a_module() do   SomeSpecificModule end
 def something_used_at_compile_time() do
  10
 end
end
# and then some other module
defmodule Bar do
 @something Foo.something_used_at_compile_time()
endThis induces a transitive compile time dependency from Bar to SomeSpecificModule.
In the case of Ash DSLs, for example, this happens because we store the resulting configuration in a module attribute. Users are aware that you can't call these pluggable modules at compile time. Only the framework code calls those modules. The above example is contrived, of course, I'm not suggesting that we need a feature to make *that* work, just trying to draw some kind of parallel.
I don't see a reasonable way to handle this without essentially removing the ability to use modules in the way that we are. It would be a pretty unreasonable amount of work/change for users to change the way that we plug behavior in Ash.
To me, another kind of module dependency, like `runtime-only`, would solve for this, and it would have to be explicitly requested, i.e `expand_literal(.., runtime_only: true)`. Then when determining transitive compile time dependencies, the compiler would not use those modules.On Tue, Sep 13, 2022 at 6:51 PM, José Valim <jose....@dashbit.co> wrote:As I mentioned, the issue on transitive compile-time dependencies are the compile-time deps, not the runtime ones. So I would focus on how to eliminate those. Otherwise I am not sure we will agree on the problem statement. :)
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/072bb99d-09a0-4567-a934-f8893015dd91n%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/l80s9s37.13dfe243-83b3-4133-8aad-7a86910f56fa%40we.are.superhuman.com.
--You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4J8qxM%3DKxMUUW6X%2Bt%2BrD9rKn0yDHbDY1fDgopxRvtNn5A%40mail.gmail.com.
José Valim
Sep 16, 2022, 9:43:35 AM9/16/22
to elixir-l...@googlegroups.com
Yes, the functionality you ask is the @after_verify callback in Elixir v1.14. It is used by LiveView to provide functionality similar to Surface and also the new Phoenix.VerifiedRoutes. :)
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/325ea272-674a-4a5a-9228-6af11579bd05n%40googlegroups.com.
Marlus Saraiva
Sep 19, 2022, 5:18:15 PM9/19/22
to elixir-lang-core
Ok, but as far as I remember, @after_verify can be used for validations but not for triggering recompilation of caller modules. Does the compiler validate the result of that callback? I tried to return different values than :ok but it does not seem to make any difference.
José Valim
Sep 19, 2022, 5:29:50 PM9/19/22
to elixir-lang-core
What I am saying is that you need to rethink the approach altogether. :)
As you said, the issue is with compile-time dependencies. Rewriting compile-time dependencies to export dependencies and changing signatures for recompilation is a brittle work-around the compiler.
With @after_verify, you should be able to treat all of them as runtime dependencies, and let the @after_verify callback be invoked whenever you are supposed to verify them again. :) This means you no longer need to track compile, exports, etc. The compiler is doing all the job for you!
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/73a308e4-59e9-4f23-bb92-1417855cc996n%40googlegroups.com.
Zach Daniel
Sep 19, 2022, 6:05:14 PM9/19/22
to elixir-l...@googlegroups.com
I guess what I'm ultimately confused about is what *absolutely necessitates* a runtime dependency becoming a transitive compile time dependency. To me, that sounds like excellent *default* behavior.
In the case of Ash resources, they are introspectable configurations. Some modules that you get back when inspecting that configuration make sense to "do something with" at compile time. Others absolutely don't. In Ash these things are modeled as behaviours because that is the way that I can define a functional contract. Not being able to refer to modules in this way essentially forces us to bypass the compiler. The way that I'm doing it currently works, incurs no runtime or compile time bugs, and the only drawback is that nothing tells the compiler that you've used any corresponding alias. Anyone doing something like the following at compile time:
```elixir
# in MyApp.Accounts.User
create :register_user do
 change HashPassword #makes your app slower for no benefit
 end
# MyApp.Accounts.User
# |> Ash.Resource.Info.actions()
# |> do_some_metaprogramming_with_that()
```
is fully aware that you can't do anything with the list of changes that are configured for the action, with the exception of know what the module name is.
In the case of Ash resources, they are introspectable configurations. Some modules that you get back when inspecting that configuration make sense to "do something with" at compile time. Others absolutely don't. In Ash these things are modeled as behaviours because that is the way that I can define a functional contract. Not being able to refer to modules in this way essentially forces us to bypass the compiler. The way that I'm doing it currently works, incurs no runtime or compile time bugs, and the only drawback is that nothing tells the compiler that you've used any corresponding alias. Anyone doing something like the following at compile time:
```elixir
# in MyApp.Accounts.User
create :register_user do
 change HashPassword #makes your app slower for no benefit
 end
# MyApp.Accounts.User
# |> Ash.Resource.Info.actions()
# |> do_some_metaprogramming_with_that()
```
is fully aware that you can't do anything with the list of changes that are configured for the action, with the exception of know what the module name is.
I really don't see any problem with this design. The way that this differs from surface is that users build compile time tools that refer to their resources, and pointing at one of these modules can cause large compile time issues that are just not necessary. Nothing about the resource will ever change because something in the `change` module changed. You're not allowed to call them at compile time, so nothing downstream will change.
i.e if `HashPassword` looks like this:
```elixir
defmodule HashPassword do
 use Ash.Resource.Change
 @impl true
 def change(user = %MyApp.Accounts.User{profile: MyApp.Accounts.Profile{}}) do
  ...
 end
 end
end
```
Now anything that depends on `MyApp.Accounts.User` at compile time also depends on `MyApp.Accounts.Profile`, but Ash has all the information to do whatever is required to make that not true (which is currently to disable the lexical tracker). The `HashPassword` module should recompile on changes to `MyApp.Accounts.User` or `MyApp.Accounts.Profile`, but the aforementioned transitive compile time dependency is, to me, entirely unnecessary.
I don't want to be a hassle or be confrontational, but this design pattern is very ingrained into Ash, and has served its users (who have shipped multiple production apps) well for some time now, so if I'm going to make them change it, or accept the fact that `alias` currently requires supplying `warn: false` (or accept significantly longer compile times on their behalf), I want to make completely sure that I understand the reason, and that Ash users are getting a benefit aside from not needing to use `warn: false` on their aliases.
I'm also more than happy to take you through what a simple usage of Ash looks like, why these modules look the way they do, or anything else that might provide context. I know its entirely possible that you could shine some light onto either 1. why this is necessary or 2. a better way to do things that wouldn't involve massive change for Ash users
I don't want to be a hassle or be confrontational, but this design pattern is very ingrained into Ash, and has served its users (who have shipped multiple production apps) well for some time now, so if I'm going to make them change it, or accept the fact that `alias` currently requires supplying `warn: false` (or accept significantly longer compile times on their behalf), I want to make completely sure that I understand the reason, and that Ash users are getting a benefit aside from not needing to use `warn: false` on their aliases.
I'm also more than happy to take you through what a simple usage of Ash looks like, why these modules look the way they do, or anything else that might provide context. I know its entirely possible that you could shine some light onto either 1. why this is necessary or 2. a better way to do things that wouldn't involve massive change for Ash users
On Mon, Sep 19, 2022 at 5:29 PM, José Valim <jose....@dashbit.co> wrote:
What I am saying is that you need to rethink the approach altogether. :)As you said, the issue is with compile-time dependencies. Rewriting compile-time dependencies to export dependencies and changing signatures for recompilation is a brittle work-around the compiler.With @after_verify, you should be able to treat all of them as runtime dependencies, and let the @after_verify callback be invoked whenever you are supposed to verify them again. :) This means you no longer need to track compile, exports, etc. The compiler is doing all the job for you!
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/73a308e4-59e9-4f23-bb92-1417855cc996n%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4KrE3xy3PnQecUP0HM%2BMNx6dboS0Ar50p8cBW9qAAwP-w%40mail.gmail.com.
José Valim
Sep 20, 2022, 3:24:32 AM9/20/22
to elixir-lang-core
I guess what I'm ultimately confused about is what *absolutely necessitates* a runtime dependency becoming a transitive compile time dependency.
defmodule A do
 def const, do: 13
end
defmodule B do
 def const, do: A.const()
end
defmodule C do
 @const B.const()
end
 def const, do: 13
end
defmodule B do
 def const, do: A.const()
end
defmodule C do
 @const B.const()
end
C depends on B at compile-time. B depends on A at runtime. Still, if A changes, C must be recompiled.
There is zero chance we will allow developers to bypass this behaviour because, if you get this wrong (now or in the future when the project evolves), then your users will increasingly run into situations where an interactive build emits different results than a full build. And those will be impossible to understand why in large projects. In turn this will ultimately harm developers confidence on projects, who will then proceed to perform full builds, "just to be sure", and harm productivity.
Once again, the runtime dependency is not the root cause. You must remove the compile-time dependencies OR isolate runtime deps from compile-time deps. I understand users are familiar with how Ash works today but, in my opinion, this discussion is indicative of design issues. I also understand change is frustrating, but sometimes it is necessary, especially as we recognize patterns that can be harmful once projects grow. Elixir changed how configuration works because it was unproductive for large projects. Phoenix moved helpers from from imports to aliases, due to similar runtime-compile issues in large projects. Etc.
Â
In the case of Ash resources, they are introspectable configurations. Some modules that you get back when inspecting that configuration make sense to "do something with" at compile time. Others absolutely don't. In Ash these things are modeled as behaviours because that is the way that I can define a functional contract. Not being able to refer to modules in this way essentially forces us to bypass the compiler.
As mentioned before, the fact a dependency can be compile-time or runtime can be problematic. You should have distinct entry-points for those. Ideally, remove most compile-time configuration OR encapsulate it in a single place instead of throughout the system.
I will bow out of the discussion for now because I believe I have given all insight I have on the topic. Once again, I want to say bypassing the compiler can be really harmful and I do not encourage this pattern. We make the compiler smarter, whenever we can, but sometimes we have to make our abstractions simpler too.
---
Marlus, feel free to start a separate thread if you need help with moving to the @after_verify callback. That's what we are using for declarative assigns in LiveView and I assume it is similar in capabilities to Surface. If the assumption is not true, please let me know.
Â
José Valim
Sep 20, 2022, 4:05:22 AM9/20/22
to elixir-lang-core
Oh, one additional point. As we add more static verification to Elixir, this verification will be done based on runtime dependencies. For example, if A depends on B at runtime, and B changes, we need to verify the types of A in relation to B are still valid. Elixir v1.15 already moved behaviours from compile time deps to runtime deps, which is a win. Therefore, hiding runtime dependencies is bound to have more and more downsides.
Reply all
Reply to author
Forward
0 new messages