Add an association between two unassociated Ecto records

[Razvan Musetescu]

Let's say I have two tables, Category and Product, a Product belongs to a Category, a Category has many Products. Let's say I inserted all categories in the database and I also insert a product record. I want to add an association between the product and the category records after both were created. How can I achieve that in Ecto?

I tried using put_assoc but it doesn't seem to work.

I'm not even sure what is the order in which the records should be passed to put_assoc

I've tried both situations

    changeset = product

             |> Ecto.Changeset.put_assoc(:category, category)

    changeset = category 

             |> Ecto.Changeset.put_assoc(:products, product)

In both cases I get `** (MatchError) no match of right hand side value:`

In the first case I also get the Product record struct being returned, In the second case I get a Category record back

[Michał Muskała]

First: please, please, please, do not post your questions in multiple places. There are many people who observe most of them, and that is just a very annoying noise.

Most functions from Ecto.Changeset require the first argument to be a changeset. It looks like you're trying to pass a schema directly. You can use a function like Ecto.Changeset.change to turn a schema into a changeset.

Michał.

> --
> You received this message because you are subscribed to the Google Groups "elixir-ecto" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/390c2521-e220-4a0e-8f23-b20846543c3a%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Razvan Musetescu]

I was already using Ecto.Changeset  and it was still failing. The problem was that I was not using the action on_replace: :nilify in the model. I managed to find the fix by accident since the documentation of put_assoc/4 doesn't specify anything about setting on_replace for the relation. I also had to preload the category's :products association before turning it into a changeset. I believe it would be nice if this would be also be mentioned in the docs.

From what I've read I have to always pull out the existing products for a category in order to add a new product to that specific category. 

Now, I can have millions of products for a specific category,  so I have to pull all of them out of the database in order to update the category_id of a single product. Is this the best solution Ecto has to offer for my problem? I never had this kind of issues when using Active Record. 

Does the Repo.update method update all the products or does it do a diff between the existing products of the category and the list passed to the put_assoc method? 

From my personal observations it does a diff before sending the UPDATE command to the database but I would like someone to confirm me that this is the case since the documentation doesn't talk about it and it is a very important issue( you don't want to touch thousands or millions of rows that have no changes in order to update a single row!)

[José Valim]

I was already using Ecto.Changeset  and it was still failing. The problem was that I was not using the action on_replace: :nilify in the model. I managed to find the fix by accident since the documentation of put_assoc/4 doesn't specify anything about setting on_replace for the relation. I also had to preload the category's :products association before turning it into a changeset. I believe it would be nice if this would be also be mentioned in the docs.

Please send a pull request! :D The only way we can continue to improve the documentation is with the help of the community. It may have been even improved at this point, so please let us know!

  

From what I've read I have to always pull out the existing products for a category in order to add a new product to that specific category. 

You don't need to do that at all. An association has two sides, why are you working only on one side? If you want to update the category_id of a single product, you update the category of that single product:

%{product | category_id: category.id}

We could also add an operation that allows to add only one product to a category but I don't think it would be simpler than changing it from the product side. Doing so would have an unfortunate consequence that you have mixed associations in your data. For example, imagine that you could do this:

add_one_product(category_without_preloading_products, product)

 

Now your category has one associated product in-memory although it has millions of entries in the database! You can mistakenly traverse this data, thinking it is the complete set, while it isn't. IIRC, this precise scenario happens on Active Record.

Does the Repo.update method update all the products or does it do a diff between the existing products of the category and the list passed to the put_assoc method? 

Look at the changeset. The changeset tells you if each child record has changes and if so, which operation will be done to update the database. But, at the end of the day, it will be a diff, yes. And I think you should change the product, not the category as said above. :)

[Razvan Musetescu]

You mean I should do something like this? :

p = Product |> Repo.get_by(id: 107)
ch = Ecto.Changeset.change(p, category_id: 204)
Repo.update!(ch)

That is what I tried to avoid. I assumed that it's not a good idea to add the foreign key of an association manually. I was also confused by the fact that the changeset function of the model( in this case Product) is used only when doing inserts and not when doing updated. By default the boilerplate code in the model doesn't whitelist the foreign key in the Product.changeset function( i.e. you can't set the :category_id on insertion unless you whitelist it the model or unless you bypass the execution of the changeset method - by using build_assoc) . By analogy, I (wrongly) assumed that this is also true for the update of the Product Chansets unless whitelisted it in the model or unless I used put_assoc. Now I realize that you never run the Product.changeset method when updating records, you actually use the Ecto.Changeset.change method( so you already bypassed the changeset method, you don't need to whitelist anything!). The problem is not that I didn't use the right method, it was that for some reason I confused the two and their effects on the data.

TL;DR 

My point is that maybe changesets should be explained better. 

[José Valim]

> That is what I tried to avoid. I assumed that it's not a good idea to add the foreign key of an association manually.

Why? Where are those assumptions coming from? The changeset documentation? I am asking because I need to understand where the confusion comes from if we need to make things better.

>  I was also confused by the fact that the changeset function of the mode is used only when doing inserts and not when doing updated

Where does that happen? Are we talking about Phoenix? In Phoenix the changeset function is used on both operations by default.

Plus you can change it to whitelist to be anything. Phoenix does not include the foreign key nor the cast_assoc commands when you use an association. So why are you assuming one would be preferred over the other?

> Now I realize that you never run the Product.changeset method when updating records, you actually use the Ecto.Changeset.change 

That's not necessarily true. Product.changeset is one of many mechanisms for applying external parameters to your data, typically from from a form or an API. If you want to directly change your product because you are not receiving data, then you can create another function, that returns a new changeset that performs something else, like updating the category.

I believe you have reached the same conclusion though. :)

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

--

You received this message because you are subscribed to the Google Groups "elixir-ecto" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/4fa18fd7-7046-4de6-bfdf-4e5a459f88f3%40googlegroups.com.

[Razvan Musetescu]

Why? Where are those assumptions coming from? The changeset documentation? I am asking because I need to understand where the confusion comes from if we need to make things better. 

I'm comming from Rails, because of this I was biased to think that

1) Since in ActiveRecord|Rails and in other frameworks it's recommended to create association via create() and build()  I assumed that Ecto also encourages developers to avoid setting foreign key ids manually,( not that it forbids them from doing so, but it is considered bad practice).

2) When I've learned that when inserting data into Ecto you can't simply give a map/hash/dictionary/struct of parameters, but that I had to create a changeset first, it felt different from what I was used to. Basically now I have to do an extra step from what I used to in Rails. I'm not saying this is good or bad, but it still an extra step you need to do and also an extra concept you need to learn. And also there is nothing analogous to Changesets in Rails. When someone starts using Changesets for the first time he might say to himself: "Why do I need to do this step? Why were Changesets created in the first place? Do they relate in any way to anything else from Rails/Active Record/Ruby?"

3) Combining the knowledge from 1) and 2) and finding out that foreign keys are not whitelisted by default inside the cast params method of the Model.changeset method, I assumed that there is a reason for this, and that reason is to simply encourage people to update foreign keys using dedicated methods( put_assoc ?)

Where does that happen? Are we talking about Phoenix? In Phoenix the changeset function is used on both operations by default.

Yes I am using it in Phoenix, but how is the Product.changeset function used on both( insert and update) by default? Don't you have to explicitely run Product.changeset(struct, params) if you want that code to get executed?

Product.changeset is one of many mechanisms for applying external parameters to your data, typically from from a form or an API.

In my case I get the data from a CSV so yet another type of data source. But I think a more general way to phrase it is that you use Product.changeset when you want to execute validations and other constraints( defined in the changeset function from the model). You use Ecto.Changeset.change when you want to get a Product Changeset and directly insert/update the data without executing any of those validations.

 If you want to directly change your product because you are not receiving data, then you can create another function, that returns a new changeset that performs something else, like updating the category.

So basically the Product.changeset function is not special by itself, it's just the default name of the function that should be used for creating a Product Changeset after going thorugh the validation/data transformation pipeline. You can always create functions with other names like Product.special_changeset and use them in other contexts. Right?

Plus you can change it to whitelist to be anything. Phoenix does not include the foreign key nor the cast_assoc commands when you use an association. So why are you assuming one would be preferred over the other?

Prefer what over what? Foreign key whitelisting over cast_assoc and build_assoc? If that's the question, I gave you the answer at  number 3) above. If not, please rephrase the question so I'd be able to understand it.

I really hope I managed to be clear enough. If there is anything that doesn't make sense please ask me in a reply!

[José Valim]

Thanks for the replies! To answer your questions.

> Yes I am using it in Phoenix, but how is the Product.changeset function used on both( insert and update) by default? Don't you have to explicitely run Product.changeset(struct, params) if you want that code to get executed?

The controller generated by Phoenix calls it on both create and update. But you can change it to do whatever you want.

> You can always create functions with other names like Product.special_changeset and use them in other contexts. Right?

Yes.

Finally, I don't think setting the foreign keys manually is a bad idea. Most times I tend to treat associations as a read-only convenience anyway. :)

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

--

You received this message because you are subscribed to the Google Groups "elixir-ecto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/ffddd7f3-d565-4204-802a-776e1ae5ac19%40googlegroups.com.