Endpoints support in ecto URLs

[Nicolas Goy]

I see that postgrex now support specifying multiple endpoints (for failover).

https://github.com/elixir-ecto/postgrex/blob/fc129a9597e79ab7cb413c1d58615b85e81eb076/lib/postgrex.ex#L81

If I am not mistaken, this is not exposed through to URLs.

It would be nice to be able to configure it via URLs?

Something similar to https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING

So we could do something like:

config :myapp, MyRepo, url: "postgres://host1,host2/dbname"

And that would be translated to the `endpoints` options of postgrex.

[José Valim]

It is supported but you need to configure your Ecto repo using database + endpoints + password, instead of the url. The url is just a convenience for those forms anyway. :)

--
You received this message because you are subscribed to the Google Groups "elixir-ecto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/26bc3f63-5867-4b22-947d-13a9e435ce36n%40googlegroups.com.

[Nicolas Goy]

Yes, but that is exactly my point.

I'd like the convenience support with URL.

Actually it is not for me, but for the OPS guys who prefer to have a single thing to configure.

As I realize it is not trivial to implement I am wondering what is your stance on it.

I usually point people to the libpq docs for URL format, but as this is not supported it confuses them.

If you think this is unreasonable, it is one more argument in favor of splitting the configuration into multiple vars.

If you think it would be a good addition, I could open a PR.


======================
Nicolas Goy

Email: ku...@goyman.com
Matrix: @kuon:goyman.com

Programmer
https://www.kuon.ch

Goyman SA
https://www.goyman.com
======================

On 16.09.21 14:01, José Valim wrote:
> It is supported but you need to configure your Ecto repo using database + endpoints + password, instead of the url. The url is just a convenience for those forms anyway. :)
>

> On Thu, Sep 16, 2021 at 1:53 PM Nicolas Goy <ku...@goyman.com <mailto:ku...@goyman.com>> wrote:
>
>
> I see that postgrex now support specifying multiple endpoints (for failover).
>

> https://github.com/elixir-ecto/postgrex/blob/fc129a9597e79ab7cb413c1d58615b85e81eb076/lib/postgrex.ex#L81 <https://github.com/elixir-ecto/postgrex/blob/fc129a9597e79ab7cb413c1d58615b85e81eb076/lib/postgrex.ex#L81>

>
> If I am not mistaken, this is not exposed through to URLs.
>
> It would be nice to be able to configure it via URLs?
>

> Something similar to https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING <https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING>

>
> So we could do something like:
>
> config :myapp, MyRepo, url: "postgres://host1,host2/dbname"
>
> And that would be translated to the `endpoints` options of postgrex.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "elixir-ecto" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto...@googlegroups.com <mailto:elixir-ecto...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/26bc3f63-5867-4b22-947d-13a9e435ce36n%40googlegroups.com <https://groups.google.com/d/msgid/elixir-ecto/26bc3f63-5867-4b22-947d-13a9e435ce36n%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google Groups "elixir-ecto" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto...@googlegroups.com <mailto:elixir-ecto...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/CAGnRm4Kof1UGgeS7WU8KkaJp5syVyma_-SyAXzAGzpD%2BsnK-Gg%40mail.gmail.com <https://groups.google.com/d/msgid/elixir-ecto/CAGnRm4Kof1UGgeS7WU8KkaJp5syVyma_-SyAXzAGzpD%2BsnK-Gg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[José Valim]

Yes, sorry, your initial message was clear. I should have read it better. :)

I think this is tricky because the URL needs to be a valid URL. So we need to find a way to express it correctly. Maybe:

    postgres://user:password@host1:port1/database?fallback=host2:port2&fallback=host3:port3

?

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/b1978fd5-bb6e-d1c6-23a2-a739ad069132%40goyman.com.

[Austin Ziegler]

The libpq link provided in the first message gives this as the URL
format supported:

postgresql://host1:123,host2:456/somedb?target_session_attrs=any&application_name=myapp

That _does_ parse:

URI.parse("postgresql://host1:123,host2:456/somedb?target_session_attrs=any&application_name=myapp")
%URI{
authority: "host1:123,host2:456",
fragment: nil,
host: "host1",
path: "/somedb",
port: 123,
query: "target_session_attrs=any&application_name=myapp",
scheme: "postgresql",
userinfo: nil
}

However, it puts the whole thing into the `authority` instead of
`host`, so it would require some special handling if a comma is
detected in `authority`.

URI.parse("postgresql://user:password@host1:123,user:password@host2:456/somedb?target_session_attrs=any&application_name=myapp")
%URI{
authority: "user:password@host1:123,user:password@host2:456",
fragment: nil,
host: "host2",
path: "/somedb",
port: 456,
query: "target_session_attrs=any&application_name=myapp",
scheme: "postgresql",
userinfo: "user:password@host1:123,user:password"
}

I don’t have any particular use for this myself, but I was pleasantly
surprised to see that it parses.

-a

> To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-ecto/CAGnRm4LHktwEacYBnce7MPBb%3DjGuG%2BHob%2Bo9YsnKiBXyiC-CHA%40mail.gmail.com.



--
Austin Ziegler • halos...@gmail.com • aus...@halostatue.ca
http://www.halostatue.ca/ • http://twitter.com/halostatue

[Austin Ziegler]

My second example is for a case not supported by libpq. I _think_ that
this means that

URI.parse("postgresql://user:password@host1:123,:password@host2:456/somedb?target_session_attrs=any&application_name=myapp")
%URI{
authority: "user:password@host1:123,:password@host2:456",

fragment: nil,
host: "host2",
path: "/somedb",
port: 456,
query: "target_session_attrs=any&application_name=myapp",
scheme: "postgresql",

userinfo: "user:password@host1:123,:password"
}

would be legitimate, but I’m not sure. One alternative (not compatible
with libpq) would be to accept a url list instead of just a single url
string, such that an environment variable might contain

"postgresql://user:password@host1:123/somedb?target_session_attrs=any&application_name=myapp,postgresql://user:password@host2:456/somedb?target_session_attrs=any&application_name=myapp"

and then be turned into a list

["postgresql://user:password@host1:123/somedb?target_session_attrs=any&application_name=myapp",
"postgresql://user:password@host2:456/somedb?target_session_attrs=any&application_name=myapp"]

-a

Here’s the relevant section from the libpq documentation:

33.1.1.3. Specifying Multiple Hosts

It is possible to specify multiple hosts to connect to, so that they
are tried in the given order. In the Keyword/Value format, the host,
hostaddr, and port options accept comma-separated lists of values. The
same number of elements must be given in each option that is
specified, such that e.g., the first hostaddr corresponds to the first
host name, the second hostaddr corresponds to the second host name,
and so forth. As an exception, if only one port is specified, it
applies to all the hosts.

In the connection URI format, you can list multiple host:port pairs
separated by commas in the host component of the URI.

In either format, a single host name can translate to multiple network
addresses. A common example of this is a host that has both an IPv4
and an IPv6 address.

When multiple hosts are specified, or when a single host name is
translated to multiple addresses, all the hosts and addresses will be
tried in order, until one succeeds. If none of the hosts can be
reached, the connection fails. If a connection is established
successfully, but authentication fails, the remaining hosts in the
list are not tried.

If a password file is used, you can have different passwords for
different hosts. All the other connection options are the same for
every host in the list; it is not possible to e.g., specify different
usernames for different hosts.