Proposal: after_changes for Ecto.Changeset

[Steve Pallen]

I use Ecto.Changeset.prepare_changes for several of my packages I. What I like most about this function is that it allows me to keep side effect free changesets for testing since the fun is not called until the Repo action. However, I have come across several use cases where I would like to use something similar, but have access to the inserted/updated model in the same transaction.

Yes, I'm aware of Ecto.Multi and have been using it. However, this approach requires that I make changes in the controller to use multi instead of of the standard Repo actions with a changeset.

So, I wanted to get feedback on the idea of adding an API (called after_changes??) that:

* provides the inserted/updated/deleted model

* is run in the same transaction as the insert/update/delete

* is executed on the Repo action (like prepare_changes)

* provides support for failure errors and rollback

One such use case would be:

defmodule MyProject.Post do
 
  schema "..." do
  end

  def changeset(struct, params \\ %{}) do
    struct
    |> cast(params, [:title])
    |> valdate_required([:title])
    |> after_changes(&insert_version/1)
  end

  def insert_version(changeset) do
    Version.changeset(%Version{}, version_params(changeset.data, changeset.action))
   |> Repo.insert
   |> case do
     {:ok, _} -> changeset
     {:error, cs} -> add_error(changeset, ....)
  end
end

[José Valim]

Hi Steve, I believe we won't accept the proposal above.

The whole point of changesets was to focus on the data and abstract away transactions, constraints, etc. Now if changesets start having callbacks that were designed for transaction purposes, what will happen when I insert/update/delete a changeset just became a lot less clear.

I am also afraid providing a quick escape route will lead people to rely on callbacks while they can still solve the same problems with "pure" changesets. For example, there is nothing in the example you showed that seems to require a callback?

I am aware there is prepare_changes today but it was designed for preparing changes and not repository side-effects. That's exactly when multi should be used.

If the issue is the mismatch between the Repo API for Multi and Changeset, I would rather try to find a way to unify them (i.e. make multi work like a changeset or vice-versa) then add transaction-aware functionality to changesets.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

--
You received this message because you are subscribed to the Google Groups "elixir-ecto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/7733ee23-ee2f-42f7-8e61-ce697e5e6db6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Steve Pallen]

José,

I can see your point about the changesets and understand why this proposal would not be accepted. 

With that said, as a package developer, there are times when I would like to integrate into an existing application, providing a way to insert into the database layer (for lack of a better phrase) without requiring application changes. This is a pattern I've come across on a number of occasions on both my open source packages and a couple of my commercial applications. Furthermore, I'd be really surprised if I'm the only one with this expectation.

At this point, I can't see any options due to the following:

• No repo call backs
• No changeset side effects
• Don't override Repo actions

I think the whatwasit design is a good example to illustrate the issue. To recap, my first attempt was to override Repo.update/delete. You pointed me to prepare_changes which I used (with side effects). And finally, in your comment above, you indicate that prepare_changes is not meant to handle side effects. :)

I guess my first question is if my expectation of being able to insert side effect based behaviour into an existing project with minimal changes is a correct one. If not, then perhaps the whole discussion is moot and I'll just move on. But if you agree that the Phoenix/Ecto community would benefit, then we should continue to discuss.

Respectively, Steve

[Steve Pallen]

José, BTW, I have just pushed a branch with some WIP for insert support in Whatwasit. This new mode tracks the post insert/update model version instead of the previous state of the model. To support this, the user must modify each of their controllers' create/update/delete actions to call a <action>_with_insert/2 api on their Repo. Whereas the previous design required adding a use and 1 pipe to the changeset.

[José Valim]

I think the whatwasit design is a good example to illustrate the issue. To recap, my first attempt was to override Repo.update/delete. You pointed me to prepare_changes which I used (with side effects). And finally, in your comment above, you indicate that prepare_changes is not meant to handle side effects. :)

I remember. :) I believe you were migrating from callbacks and prepare_changes was the closest thing to emulate them?

 

I guess my first question is if my expectation of being able to insert side effect based behaviour into an existing project with minimal changes is a correct one.

I believe this is a valid issue/concern, I just do not agree with the proposed solution so far. Let's keep the discussion alive.

 

To support this, the user must modify each of their controllers' create/update/delete actions to call a <action>_with_insert/2 api on their Repo. Whereas the previous design required adding a use and 1 pipe to the changeset.

If you are introducing this new API, then does the user need to call a function in their changeset in the first place?

[Steve Pallen]

José,

I'm encouraged with your reply.

I believe this is a valid issue/concern, I just do not agree with the proposed solution so far. Let's keep the discussion alive.

Excellent :) 

If you are introducing this new API, then does the user need to call a function in their changeset in the first place?

No the function in the changeset is not required and won't work if it is there. At this point, both approaches are supported (its just an experiment right now) and converting from the old approach to the new approach requires removing the changeset call.

Continuing the discussion...

First, I like the multi api, with one exception; the error tuple. I can understand why it returns more than two elements in the tuple and sometimes that information may be required. But I find myself adding a conversion case after the |> Repo.transaction to convert to {:error, changeset} to fit into the existing controller.

Second, what I believe is missing is being able to inject into the Repo actions functionality before and after the repo calls. 

Another approach I just though. I could override the Repo alias at the top of the controller and provide a module where the create/update/etc functions are implemented with Ecto.Multi. I think I could implement version tracking pretty transparently with this approach. However, it  seems a little hackie when all I'm trying to do is override the repo functions. 

Thoughts?

[José Valim]

I prefer the one with extended repository functions insert_with_xyz. It does have an issue of composing poorly though, what if another package provides a similar API and you want to use both? But I believe that can be fixed by exposing a more low level function when necessary.

I guess at the end of the day it will always be a balance between implicit and explicit.

--


You received this message because you are subscribed to the Google Groups "elixir-ecto" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto...@googlegroups.com.


To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/b2faac43-a40a-4645-be67-18c8a8ae91d3%40googlegroups.com.

[Steve Pallen]

Composibility a pretty significant concern. This is where Plug adds significant value for web requests. I'm going to try prototyping some ideas.

[Michał Muskała]

My ideal solution to something like this would leverage Ecto.Multi

    Multi.new

    |> Multi.insert(:foo, foo_changeset)

    |> Whatwasit.track(:foo)

    |> Multi.update(:bar, bar_changeset)

    |> Whatwasit.track(:bar)

This does require changes in the controller, but this should become less of an issue with the recommended structure of phoenix 1.3, where each action is wrapped in a function and repo is entirely abstracted away from controllers.

I could easily imagine function used like:

    multi

    |> Repo.transaction

    |> Whatwasit.unwrap(:foo)

That would extract foo from the multi result and generate a result similar to what you'd get form just calling Repo.insert with the changeset. That way you would be entirely possible to do that in the "action" function keeping the controller exactly the same. Such solution is also easily composable.

Michał.

On 13 December 2016 at 22:46:07, Steve Pallen (smpal...@gmail.com) wrote:

Composibility a pretty significant concern. This is where Plug adds significant value for web requests. I'm going to try prototyping some ideas.

--
You received this message because you are subscribed to the Google Groups "elixir-ecto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/c8407a5f-55e0-4994-89fa-0c6970f6b946%40googlegroups.com.