[PROPOSAL] Support runtime dynamic preload bindings in Ecto.Query

29 views
Skip to first unread message

Zach Allaun

unread,
Dec 5, 2022, 4:05:52 PM12/5/22
to elixi...@googlegroups.com
Hi all,

I’m wondering what the likelihood of acceptance would be for a PR that adds runtime dynamic support to Ecto.Query.preload.

## Context

Right now, preload can accept a keyword list referencing join bindings, allowing a single query to fetch the source resource and the preloaded resource.

from x in SomeSchema,
  join: y on assoc(x, :some_assoc),
  where: x.value > y.value,
  preload: [some_assoc: y]

Even more clever things can be done using lateral joins to, for instance, limit the number of preloaded resources per record. 

Additionally, these preloads can nest. 

from x in SomeSchema,
  join: y on assoc(x, :some_assoc),
  join: z on assoc(y, :other_assoc),
  where: …,
  preload: [some_assoc: {y, other_assoc: z}]

## Problem

It is not appear to be possible to dynamically specify the associations to be preloaded at runtime. To be clear, you cannot do:

from …,
  preload: ^some_preloads

where some_preloads references a binding established by an earlier join clause. This is because preload does not accept a dynamic that is able to reference arbitrary bindings.

This makes it difficult to create composable queries with arbitrary preloads without the use of macros, and even that has some limitations. 

## Proposal and Discussion 

I’d love to see some way to specify a fully dynamic argument to preload. I believe this could be accomplished using the existing dynamic. 

preloads = [
  foo: {
    dynamic([foo: f], f), 
    bar: dynamic([bar: b], b)
  }
]

from …,
  preload: ^preloads

I believe Ecto.Query.Builder.Update might be something of a template for this, as update allows for dynamics in this way. Consider this example from the docs:

updates = [
  set: [average: dynamic([p], p.sum / p.count)]
]

from query, update: ^updates

Update handles this by differentiating between compile and runtime values (
https://github.com/elixir-ecto/ecto/blob/v3.9.2/lib/ecto/query/builder/update.ex#L43), handling dynamic expansion at runtime (
https://github.com/elixir-ecto/ecto/blob/v3.9.2/lib/ecto/query/builder/update.ex#L172). I believe preload could do something similar, though I admit that I’m not at all familiar with the code beyond looking into it for the purpose of this proposal. 

I’d be happy to work towards contributing this if the maintainers feel that it would be appropriate and valuable!

Thanks,
Zach

José Valim

unread,
Dec 6, 2022, 5:31:31 AM12/6/22
to elixi...@googlegroups.com
HI Zack, I think this is a good proposal and we could definitely consider a PR!

--
You received this message because you are subscribed to the Google Groups "elixir-ecto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/CAPwr2_v1E55mGxkJQZGenhnvQ%2BACN526_V7epFJFdAYYa77B0Q%40mail.gmail.com.

Zach Allaun

unread,
Dec 6, 2022, 10:04:03 AM12/6/22
to elixi...@googlegroups.com
Thanks, José.

A quick question on style -- Ecto.Query.Builder.Preload, for instance, hasn't been touched in 5 years. My editor is configured to format on save, and naturally, `mix format` wants to format parts of the code differently. Would you prefer that I disable format-on-save and use the same "local style" present in the files I'm working in? Alternatively, I could format files I'm working on prior to making any actual changes and commit that separately.

Zach

You received this message because you are subscribed to a topic in the Google Groups "elixir-ecto" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-ecto/Z_tFUi6ONWU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elixir-ecto...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/CAGnRm4K3qggfBFYLd5J_kPuUMzZquemKoVYKtf-fFVR5T_eDhA%40mail.gmail.com.

José Valim

unread,
Dec 6, 2022, 10:09:01 AM12/6/22
to elixi...@googlegroups.com
Yes, please disable format on save as that makes it hard to review. We will eventually format the codebase. :)

Zach Allaun

unread,
Dec 6, 2022, 10:11:26 AM12/6/22
to elixi...@googlegroups.com
Will do!

I'll make a draft PR once I have something concrete and will cross-reference this thread.

Zach Allaun

unread,
Dec 6, 2022, 10:43:16 PM12/6/22
to elixi...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages