Dynamic selects in queries

[Arno Dirlam]

Hello,

I'd like to suggest the ability to use dynamic in conjunction with select and select_merge in order to return data in a shape that's defined at runtime. Would that make sense?

For example:

    alias = :blog
    field = :title
    ref = dynamic(field(as(^alias), ^field))
    from(b in "blogs", as: :blog, select: %{title: ^ref})

It is already possible to build complex select statements; it's just not possible yet to do it dynamically.

It currently fails with an error:

    ** (ArgumentError) expected a list of fields in `select/2` inside `select`, got: `dynamic([], as(:blog).title)`

Any feedback is appreciated! Thanks a lot 🙏

[José Valim]

I don't see an issue with it in theory but in practice.all of dynamic functionality assumes the dynamic is only interpolated at the root. This is also for performance (so we don't have to traverse trees to find there are more trees inside). I assume select would be the easiest element to lift this restriction from but it may still be quite a lot of work.

--
You received this message because you are subscribed to the Google Groups "elixir-ecto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/554f146c-0056-457a-9cd7-43cabe4c153en%40googlegroups.com.

[José Valim]

That said, if you want to give it a try, then go for it!

[Arno Dirlam]

Great, thanks for the openness and hints, José!

I will give it a try and open a PR if and when I have come up with something useful :)

Would it make sense to implement it while keeping this restriction in place first, or does it only make sense when implementing non-root interpolation with it from the get-go?

[José Valim]

I would try removing the restriction only for select first.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/3f9e7594-fc7c-4b40-a60f-b9ab69d54f55n%40googlegroups.com.

[Zach Daniel]

Hey Arno,

Did you get anywhere with this? I'm refactoring AshPostgres to be a better citizen when it comes to its usage of Ecto, and I'm going to need to be able to do this as well. Let me know if I can help :D 

[Zach Daniel]

I'm looking into implementing this, and I'm not 100% sure where to start, but I think I can figure it out. What I'm actually curious of is the fact that it isn't possible to do something like this:

```
dynamic([row], %{foo: row.foo})
```

which is the main limitation between making selects work exactly like other things (allowing dynamics at the top level).

Would we rather make it possible to have map and merge as valid expressions in dynamics, and the require selects to be dynamics at the top level?

Or would it be better to traverse the select?

And in the case of traversal, are there any tips on when/where to do that? Should I do it in `Ecto.Query.Builder.Select.select!/5`?

Would love a nudge in the right direction.

[Zach Daniel]

I have a few pieces of information gathered thus far: 1. it does not look like we want select to work like the rest of the things that accept dynamic expressions, which is that they only work at the top level. E.g from row in "foo", select: %{a: ^dynamic} should work 2. this means that I don't think I can just do my work here: https://github.com/elixir-ecto/ecto/blob/master/lib/ecto/query/builder/select.ex#L150. Or at least I need some way to get the actual dynamic at that point because the select looks like this: {:^, [line: 12], [{:dynamic, [if_undefined: :apply, line: 12], nil}]} . (maybe its two step here? Just return the quoted expression and do the work later?) 3. I additionally don't think I can do it in the escape logic in that same file, primarily because I can't find a precedent for using escape in that way, and also there is no query at that point to do this on: Ecto.Query.Builder.Dynamic.fully_expand(query, dynamic) 4. It looks like it could potentially be done in the planner, but that seems like the wrong place for it.

[Arno Dirlam]

Sorry Zach, I somehow missed out on your continuing the conversation, got slightly distracted.

Motivated by ElixirConf EU last week, presenting my library, and talking to José about it (also in regard to set-theoretic types),

I spent quite some time this week on this and just opened a PR with the full feature set discussed here, plus support for subqueries in selects 😁

Please let me know if this also fits your use cases for AshPostgres.

Cheers, Arno