Support storing embeds_many in :map column

[Arno Dirlam]

Lots of nice work has been done on embedded schemas (most recently inline embedded schemas, optional primary_key, etc.).
According to the docs, the column to store embeds_many schemas must be {:array, :map}.
However, MySQL (and probably also other DBs) doesn't have an array column type.

Would it be an option to able to store embeds_many schemas in a :map column?

All the user-facing syntax would basically be the same, just the storage would be different (as the whole list/array is part of one JSON-encoded string).


Cheers, Arno

[José Valim]

Maybe it would be better to support it as {:map, :map} because I am sure the abstraction is going to break in other places if we try to fake arrays.

[Carsten Kraus]

I was having problems to query an embeds_many relation {:array, :map} in Postgres. Posted here: https://elixirforum.com/t/querying-embeds-many-field/2453/3

In my case I could actually just use :jsonb(instead of {:array, :map}) in the migration and was able to query for things. Maybe have a shot just using :map?

However, my case was a small prototype only and maybe, as Jose mentions, stuff would have broken later on.

I'm curious: would at least for Postgres :jsonb just work for embeds_many?

[Arno Dirlam]

Please see my replies inline.

José Valim wrote:

Maybe it would be better to support it as {:map, :map} because I am sure the abstraction is going to break in other places if we try to fake arrays.

I'm not sure about the abstraction in other places, but imho the ideal way would be to keep the separation of concerns, i.e. the migration is only concerned with how data is stored in the database. It has been the case for {:array, :map}, as :array is a native data type in Postgres. However, as far as I understand, :map already represents a JSON-encoded data field, so it should not have to be further specified at the migration level. You wouldn't need a migration to change a column from :map to {:map, :map}, for instance, as the database doesn't care what's inside the JSON; this is handled by the application (the schema).

Potentially, if we had a column type that ecto uses to store arbitrary JSON-encoded data, the schema logic (including casting and validations) could even be extended further, e.g. to support nested inline embeds and the like, which could be extremely powerful.

Carsten Kraus wrote:

I was having problems to query an embeds_many relation {:array, :map} in Postgres. Posted here: https://elixirforum.com/t/querying-embeds-many-field/2453/3

In my case I could actually just use :jsonb(instead of {:array, :map}) in the migration and was able to query for things. Maybe have a shot just using :map?

However, my case was a small prototype only and maybe, as Jose mentions, stuff would have broken later on.

I'm curious: would at least for Postgres :jsonb just work for embeds_many?

That's definitely in line with what I thought. Being new to ecto (and elixir in general), I had a hard time figuring out what the column type (!) :map stands for. The only hint in the docs is actually a code comment

add :object,  :map  # Elixir type which is handled by the database

 

so I inferred that it means "a column to store arbitrary JSON-encoded data". I'm still not sure what exactly it represents. In MySQL it creates a TEXT column, maybe it creates an hstore in Postgres?

To wrap up:

Ideally imho, we'd have a column type (:map? :json??) to store arbitrary JSON-encoded data, that is managed on the schema level (embeds, casting, validation), and merely encoded/decoded at the db adapter level, if the column type is used in the migration.

Is this way off from what you had in mind for ecto, José? I'm willing to spend some time working on a PR, if that's welcome, but I'd first have to figure out how to approach this :)

[José Valim]

You could attempt to do that with a custom type today. We already have :json at the database level and you can have custom types with types :any, allowing you to encode/decode it to whatever you want.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

--
You received this message because you are subscribed to the Google Groups "elixir-ecto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/fadecc70-677b-4c57-b0ce-2f0678f8fe6a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.