[Proposal] Add support for ParadeDB pg_search

[Philippe Noël]

Hi folks,

I am one of the maintainers of https://github.com/paradedb/paradedb. We've started getting users that use both ParadeDB and Elixir requesting support for pg_search in Ecto. You can see an example here:

> To exposition dump a bit: I'm an Elixir/Phoenix die-hard. I'd say a lot Elixir devs lean more into radical simplicity and the "just use postgres" movement. The main barrier I'm facing in my app right now is the lack of integration between Ecto and ParadeDB's expressions.
Right now to use ParadeDB, I'm foregoing Ecto's superb composability for more manual query building. I'm looking into what it'd take to build a package that'd make it feel first class though.

Lmk how we can help. This would be super cool to see come to life.

Philippe

[José Valim]

Hi Philippe, nice meeting you! Both Ecto and Postgrex (our bindings to PostgreSQL) are extensible, so you should be able to add your own types if necessary (which is usually the only work required to support a new extension). A good example to follow, for those interested, is the pg-vector extension:

https://github.com/pgvector/pgvector-elixir/

Keep in mind that pgvector introduces 3 different PostgreSQL columns, so there is a bit more code in there, but if pg_search requires only a new column type, it should be easier. :)

--
You received this message because you are subscribed to the Google Groups "elixir-ecto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/d55c39ec-cce4-4591-9f35-3aa3720afb4an%40googlegroups.com.

[Philippe Noël]

Appreciate it. I don't know if we have the bandwidth to handle this just yet, so would love help from the community :) Otherwise perhaps in a few weeks/months we can take a look!

[Greg Rychlewski]

I took a look at your extension and it wasn't immediately obvious to me what is missing from Ecto to support it. Do you perhaps have any specific examples of what your users would like to do that they can't do now?

[Cameron Duley]

I put together an API proposal based on my experience w/ ParadeDB thus far, with the intent of:

• Figuring out what an unambiguous and ergonomic interface might look like.
• Identifying what it'd take to implement it.
• Soliciting feedback/gathering ideas from others.

I looked at pgvector-elixir and timescale, but supporting pg_search seems more involved on account of its calling convention.

If anyone has suggestions or starting points on how a package could implement this, I'm happy to take them and dig deeper.

Meta: If this conversation's better off on elixirforum, I can start a thread there.

On Friday, August 23, 2024 at 4:18:06 PM UTC-4 ph...@paradedb.com wrote:

[Ruslan Doga]

👋 Cameron

I saw https://moosie.us/parade_db_ecto just now and I wonder if having something like "private" or adapter specific field in Ecto.Query struct would be helpful?

I think it might allow for better support of ParadeDB some other SQL-but-not-quite databases like ClickHouse (e.g. here, in my attempt to add input function https://clickhouse.com/docs/en/sql-reference/table-functions/input via custom API https://github.com/plausible/ecto_ch/blob/120a7b5ac19ba56f5b6378062cbb62b3284bfa16/lib/ecto/adapters/clickhouse/api.ex#L8-L50)

[Ruslan Doga]

I also think it might still be possible to express ParadeDB queries without forking Ecto and just relying on fragments: https://github.com/ruslandoga/parade :)

[Ruslan Doga]

These tests provide a quick view of what I tried to achieve: https://github.com/ruslandoga/parade/blob/a1b76a506d7fab04622b075d3fe65251a2b53a2b/test/parade_test.exs#L15-L49