Ecto Url Scheme Question

[Jason S]

Why? Why wouldn't we just stick to postgres:// or mysql:// or etc which seem to be the standard for other languages and packages. 

Makes it tricky to work with heroku and others which setup DATABASE_URL to be a postgres url. For example.

[José Valim]

That's a good question Jason.

The main issue I can see with this approach is that the adapter today is capable of injecting code in the repository and it wouldn't be able to if we move it to the URL (we'd have a chicken-egg issue: to get the url, we need to compile the repo, but if we compile the repo, we can no longer inject the adapter code).

However, I am not sure what changing the scheme would bring us. I know Active Record uses the schema you mentioned but I believe DATABASE_URL is configurable in Heroku.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

On Tue, Dec 10, 2013 at 3:17 AM, Jason S <stie...@gmail.com> wrote:

Why? Why wouldn't we just stick to postgres:// or mysql:// or etc which seem to be the standard for other languages and packages. 

Makes it tricky to work with heroku and others which setup DATABASE_URL to be a postgres url. For example.

--
You received this message because you are subscribed to the Google Groups "elixir-ecto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-ecto...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Jason Stiebs]

I am mostly just curious because it seemed out of place but it seems to have a good reason. Otherwise I would have opened a PR or something. :) 

I personally think Sequel http://sequel.jeremyevans.net/rdoc/files/doc/opening_databases_rdoc.html does a really good job of explaining the different connections and parameters (and a ton of other things as an ORM :)). 

Also Heroku allows you to configure it, but most of its commands assume the default database is DATABASE_URL=postgres://. Like the backups, followers, upgrading and etc. 

[Clint Shryock]

Heroku assumes the default, or more commonly considered "primary", database to be at DATABASE_URL. It's a convention that started at Heroku to aid in 12 factor apps (http://12factor.net) and I believe made it's way into Rails as a convention. I could be wrong there though...

PGBackups requires DATABASE_URL to be `postgres://` format, otherwise PGBackups won't work. That's about the only restriction/gotcha Heroku has regarding DATABASE_URL though. Heroku recommends you do not configure DATABASE_URL directly. Instead, you set it by adding a database add-on and using pg:promote to change DATABASE_URL. The idea of using environment variables for things like this is that Heroku can update that DATABASE_URL in extreme circumstances  (server crash, et. al) and your application only needs a restart and no code changes. Requiring users to manually set DATABASE_URL to `ecto://<string>` is going to make using this library on Heroku troublesome. 

I believe Rails ORM (and maybe Sequel) determines the driver based on the protocol (postgres://, mysql://), which I think is a good idea. The `ecto://` protocol threw me when I first saw this library, but I don't design many libraries, so I assumed it was made for a good reason.

--

Clint

[José Valim]

Thanks for the input! Two questions:

1. What if we ignore the scheme? We don't use it for anything right now anyway, then you can set it to postgres or whatever you choose;

2. We use the query string in DATABASE_URL to specify adapter options. Is that a bad idea? I.e. would Heroku try to use the query string for something else?

[Jason Stiebs]

Currently a standard heroku DATABASE_URL follows the standard postgres://username:password@domain/database conventions and does not include any other parameters. Sequel just takes on a seperate OPTS hash for connect to include parameters.

[José Valim]

Can someone please give a try and verify that Heroku won't freak out with query strings?

Well, even if it does, I don't think it would be an issue, as you can add the query string just in Ecto land.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[kwilliams]

Jason,

I don't think the Heroku DATABASE_URL should be used directly as an ecto repo url. The heroku docs provide an example of parsing the DATABASE_URL string to build a JDBC connection. The same approach should probably be used with ecto.

see https://devcenter.heroku.com/articles/heroku-postgresql#jdbc

[Clint Shryock]

Jos é – Heroku just uses DATABASE_URL as a convention. Your application is what reads it and behaves accordingly. You can add query strings to DATABASE_URL all you want, as long as the ORM you're using understands them :)

- HOWEVER -

Don't do that :P

Specifically for Heroku Postgres, part of making your database/app highly available is being able to update DATABASE_URL for you (server crash, change IPs of server, etc). Changing this env. var causes your application to restart, thus picking up the new DATABASE_URL and your application continues to work with no required code changes.

Heroku does not preserve the query string parameters you've manually added to DATABASE_URL :( 

For things like Rails, we recommend configuring things in initializers, example:

- https://devcenter.heroku.com/articles/concurrency-and-database-connections

--

Clint

[Jason Stiebs]

I would personally not touch the DATABASE_URL and just update in elixir land, so those changes are in source code. I'll run this test later tonight to see what happens with basic changes/commands.

Kurt I don't think its an issue see https://devcenter.heroku.com/articles/rack where they use it directly.

 

[José Valim]

Thanks Jason and Clint!

That said, someone using Heroku can simply do:

def url do

  base = System.get_env("DATABASE_URL") || raise "No DATABASE_URL"

  "#{base}?size=10"

end

I have opened an Ecto issue to guarantee we don't care about the scheme, in case anyone wants to send a pull request. :)

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Brendan Murphy]

A given solution to this by Philipp Küng -

http://philippkueng.ch/deploy-a-phoenix-application-on-heroku.html#parse-the-environment-variable

:-)