Can we improve how we work with prepared queries and how they are cached

[benjamin...@gmail.com]

Currently prepared queries are using references, and store a lot of data in ETS. In SQL we take a different approach to identify queries, we have globally unique ids, which are generated at compile time. I could imagine a world where the only information needed to be stored would be the conn pid and id to signify that the query has been prepared in the specific connection or needed to be prepared.

To be clear, I'm not asking anyone to do this work, I can do it myself. But I want to gauge the core team if this kind of improvement is of intrest to them.

[José Valim]

Because Ecto queries are composed at runtime, we can’t build compile time IDs. This is why our query builder is based on the runtime parts. If you have good ideas to do both composition and unique compile time IDs, then that will be appreciated. PRs welcome!

José Valim

https://dashbit.co/

On Sat, Oct 11, 2025 at 19:22 benjamin...@gmail.com <benjamin...@gmail.com> wrote:

Currently prepared queries are using references, and store a lot of data in ETS. In SQL we take a different approach to identify queries, we have globally unique ids, which are generated at compile time. I could imagine a world where the only information needed to be stored would be the conn pid and id to signify that the query has been prepared in the specific connection or needed to be prepared.

To be clear, I'm not asking anyone to do this work, I can do it myself. But I want to gauge the core team if this kind of improvement is of intrest to them.

--
You received this message because you are subscribed to the Google Groups "elixir-ecto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/elixir-ecto/73b679f1-49e3-411d-a9d5-43fcf71aec18n%40googlegroups.com.

[benjamin...@gmail.com]

Yeah, I was afraid that was the issue, or more specifically the DSL. In SQL it's trvial as we concatenate tokens before we parse and generate strings. I know it's a huge ask to deprecate the DSL in favour of SQL and I really wanted to reuse db_connection and all the drivers. Although I do believe there is a lot to gain by having a consistent structure to represent SQL across drivers and libraries.

This feels like being stuck between and rock and a hard place, as there is no straigth forward way to satisfy everything and everybody.