[Django] #32292: Allow postgresql database connections to use postgres services
Django
-------------------------------------+-------------------------------------
Reporter: levihb | Owner: nobody
Type: New | Status: new
feature |
Component: Database | Version: 3.1
layer (models, ORM) | Keywords: database postgresql
Severity: Normal | postgres service pg_service config
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Postgres offers a way to make database connections through the use of
services, which are basically equivalent to MySQL's options files.
Server, database, username, etc information is stored by default in
`~/.pg_service.confg` and takes a very similar format to MySQL cnf files:
```
[my_alias]
host=10.0.19.10
user=postgres
dbname=postgres
port=5432
```
And password can be stored in `~/.pgpass` under a different format.
I think being able to just add them to the DATABASES config would be
useful, similar to how you can add MySQL cnf files. psycopg2 supports it
just fine through the service argument/string `connect(service='my_alias')
connect('service=my_alias')`.
At the moment it can be added like this:
```
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'OPTIONS': {'service': 'my_alias'}
}
}
```
Which works, however it involves repeating the database name. I don't
think the database name should be repeated twice because it couples the
config and the service file together, and makes it harder to just move it
between different environments. I think ideally you would just specify the
service, either like this:
```
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'OPTIONS': {'service': 'my_alias'}
}
}
```
Or maybe a better way would be?:
```
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'SERVICE': 'my_alias
}
}
```
It seems like something that would be super easy to add. I don't mind
creating a pull request for it, but would like to know why it hasn't been
added, and how it would be recommended to add it.
--
Ticket URL: <https://code.djangoproject.com/ticket/32292>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Reporter: levihb | Owner: nobody
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: database postgresql | Triage Stage:
postgres service pg_service | Unreviewed
config |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
New description:
Postgres offers a way to make database connections through the use of
services, which are basically equivalent to MySQL's options files.
Server, database, username, etc information is stored by default in
`~/.pg_service.confg` and takes a very similar format to MySQL cnf files:
```
[my_alias]
host=10.0.19.10
user=postgres
dbname=postgres
port=5432
```
And password can be stored in `~/.pgpass` under a different format.
I think being able to just add them to the DATABASES config would be
useful, similar to how you can add MySQL cnf files. psycopg2 supports it
just fine through the service argument/string `connect(service='my_alias')
connect('service=my_alias')`.
At the moment it can be added like this:
{{{#!python
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'OPTIONS': {'service': 'my_alias'}
}
}
}}}
Which works, however it involves repeating the database name. I don't
think the database name should be repeated twice because it couples the
config and the service file together, and makes it harder to just move it
between different environments. I think ideally you would just specify the
service, either like this:
{{{#!python
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'OPTIONS': {'service': 'my_alias'}
}
}
}}}
Or maybe a better way would be?:
{{{#!python
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'SERVICE': 'my_alias
}
}
}}}
It seems like something that would be super easy to add. I don't mind
creating a pull request for it, but would like to know why it hasn't been
added, and how it would be recommended to add it.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:1>
Django
-------------------------------------+-------------------------------------
Reporter: levihb | Owner: nobody
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: database postgresql | Triage Stage:
postgres service pg_service | Unreviewed
config |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
> Postgres offers a way to make database connections through the use of
> services, which are basically equivalent to MySQL's options files.
>
> Server, database, username, etc information is stored by default in
> `~/.pg_service.confg` and takes a very similar format to MySQL cnf files:
>
> ```
> [my_alias]
> host=10.0.19.10
> user=postgres
> dbname=postgres
> port=5432
> ```
>
> And password can be stored in `~/.pgpass` under a different format.
>
> I think being able to just add them to the DATABASES config would be
> useful, similar to how you can add MySQL cnf files. psycopg2 supports it
> just fine through the service argument/string
> `connect(service='my_alias') connect('service=my_alias')`.
>
> At the moment it can be added like this:
>
> {{{#!python
> DATABASES = {
> 'default': {
> 'ENGINE': 'django.db.backends.postgresql',
> 'NAME': 'postgres',
> 'OPTIONS': {'service': 'my_alias'}
> }
> }
> }}}
>
> Which works, however it involves repeating the database name. I don't
> think the database name should be repeated twice because it couples the
> config and the service file together, and makes it harder to just move it
> between different environments. I think ideally you would just specify
> the service, either like this:
>
> {{{#!python
> DATABASES = {
> 'default': {
> 'ENGINE': 'django.db.backends.postgresql',
> 'OPTIONS': {'service': 'my_alias'}
> }
> }
> }}}
>
> Or maybe a better way would be?:
>
> {{{#!python
> DATABASES = {
> 'default': {
> 'ENGINE': 'django.db.backends.postgresql',
> 'SERVICE': 'my_alias
> }
> }
> }}}
>
> It seems like something that would be super easy to add. I don't mind
> creating a pull request for it, but would like to know why it hasn't
> been added, and how it would be recommended to add it.
New description:
Postgres offers a way to make database connections through the use of
services, which are basically equivalent to MySQL's options files.
Server, database, username, etc information is stored by default in
`~/.pg_service.confg` and takes a very similar format to MySQL cnf files:
{{{#!python
[my_alias]
host=10.0.19.10
user=postgres
dbname=postgres
port=5432
}}}
And password can be stored in `~/.pgpass` under a different format.
I think being able to just add them to the DATABASES config would be
useful, similar to how you can add MySQL cnf files. psycopg2 supports it
just fine through the service argument/string `connect(service='my_alias')
connect('service=my_alias')`.
At the moment it can be added like this:
{{{#!python
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'OPTIONS': {'service': 'my_alias'}
}
}
}}}
Which works, however it involves repeating the database name. I don't
think the database name should be repeated twice because it couples the
config and the service file together, and makes it harder to just move it
between different environments. I think ideally you would just specify the
service, either like this:
{{{#!python
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'OPTIONS': {'service': 'my_alias'}
}
}
}}}
Or maybe a better way would be?:
{{{#!python
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'SERVICE': 'my_alias
}
}
}}}
It seems like something that would be super easy to add. I don't mind
creating a pull request for it, but would like to know why it hasn't been
added, and how it would be recommended to add it.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:2>
Django
-------------------------------------+-------------------------------------
Reporter: levihb | Owner: nobody
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: database postgresql | Triage Stage:
postgres service pg_service | Unreviewed
config |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
> Postgres offers a way to make database connections through the use of
> services, which are basically equivalent to MySQL's options files.
>
> Server, database, username, etc information is stored by default in
> `~/.pg_service.confg` and takes a very similar format to MySQL cnf files:
>
> {{{#!python
> [my_alias]
> host=10.0.19.10
> user=postgres
> dbname=postgres
> port=5432
> }}}
>
> And password can be stored in `~/.pgpass` under a different format.
>
> I think being able to just add them to the DATABASES config would be
> useful, similar to how you can add MySQL cnf files. psycopg2 supports it
> just fine through the service argument/string
> `connect(service='my_alias') connect('service=my_alias')`.
>
> At the moment it can be added like this:
>
> {{{#!python
> DATABASES = {
> 'default': {
> 'ENGINE': 'django.db.backends.postgresql',
> 'NAME': 'postgres',
> 'OPTIONS': {'service': 'my_alias'}
> }
> }
> }}}
>
> Which works, however it involves repeating the database name. I don't
> think the database name should be repeated twice because it couples the
> config and the service file together, and makes it harder to just move it
> between different environments. I think ideally you would just specify
> the service, either like this:
>
> {{{#!python
> DATABASES = {
> 'default': {
> 'ENGINE': 'django.db.backends.postgresql',
> 'OPTIONS': {'service': 'my_alias'}
> }
> }
> }}}
>
> Or maybe a better way would be?:
>
> {{{#!python
> DATABASES = {
> 'default': {
> 'ENGINE': 'django.db.backends.postgresql',
> 'SERVICE': 'my_alias
> }
> }
> }}}
>
> It seems like something that would be super easy to add. I don't mind
> creating a pull request for it, but would like to know why it hasn't
> been added, and how it would be recommended to add it.
New description:
Postgres offers a way to make database connections through the use of
services, which are basically equivalent to MySQL's options files.
Server, database, username, etc information is stored by default in
`~/.pg_service.conf` and takes a very similar format to MySQL cnf files:
{{{#!python
[my_alias]
host=10.0.19.10
user=postgres
dbname=postgres
port=5432
}}}
And password can be stored in `~/.pgpass` under a different format.
I think being able to just add them to the DATABASES config would be
useful, similar to how you can add MySQL cnf files. psycopg2 supports it
just fine through the service argument/string `connect(service='my_alias')
connect('service=my_alias')`.
At the moment it can be added like this:
{{{#!python
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'OPTIONS': {'service': 'my_alias'}
}
}
}}}
Which works, however it involves repeating the database name. I don't
think the database name should be repeated twice because it couples the
config and the service file together, and makes it harder to just move it
between different environments. I think ideally you would just specify the
service, either like this:
{{{#!python
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'OPTIONS': {'service': 'my_alias'}
}
}
}}}
Or maybe a better way would be?:
{{{#!python
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'SERVICE': 'my_alias
}
}
}}}
It seems like something that would be super easy to add. I don't mind
creating a pull request for it, but would like to know why it hasn't been
added, and how it would be recommended to add it.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: levihb | Owner: nobody
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution:
postgres service pg_service |
config |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Unreviewed => Accepted
Comment:
Configuration without `NAME` already works for me, e.g.:
{{{
'default': {
'ENGINE': 'django.db.backends.postgresql',
'OPTIONS': {
'service': 'default_django_test'
}
},
}}}
so only setting `PGSERVICE` for
[https://github.com/django/django/blob/2a76f4313423a3b91caade4fce71790630ef9152/django/db/backends/postgresql/client.py#L10-L44
the underlying command-line client] and docs are missing.
> I don't mind creating a pull request for it, but would like to know why
it hasn't been added, ...
Because you're the first to suggest it.
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: levihb | Owner: nobody
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: database postgresql | Triage Stage: Accepted
postgres service pg_service |
config |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by levihb):
Replying to [comment:4 Mariusz Felisiak]:
> Configuration without `NAME` already works for me, e.g.:
> {{{
> 'default': {
> 'ENGINE': 'django.db.backends.postgresql',
> 'OPTIONS': {
> 'service': 'default_django_test'
> }
> },
> }}}
It doesn't work for me. E.g.:
{{{#!python
'default': {
'ENGINE': 'django.db.backends.postgresql',
'OPTIONS': {
'service': 'test_service'
}
},
}}}
Throws the following when it tries to access the database:
{{{
Traceback (most recent call last):
File "/redacted_path/postgres_services/venv/lib/python3.8/site-
packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/redacted_path/postgres_services/venv/lib/python3.8/site-
packages/django/core/handlers/base.py", line 179, in _get_response
response = wrapped_callback(request, *callback_args,
**callback_kwargs)
File "/redacted_path/postgres_services/mysite/testapp/views.py", line 9,
in index
c = db_conn.cursor()
File "/redacted_path/postgres_services/venv/lib/python3.8/site-
packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/redacted_path/postgres_services/venv/lib/python3.8/site-
packages/django/db/backends/base/base.py", line 259, in cursor
return self._cursor()
File "/redacted_path/postgres_services/venv/lib/python3.8/site-
packages/django/db/backends/base/base.py", line 235, in _cursor
self.ensure_connection()
File "/redacted_path/postgres_services/venv/lib/python3.8/site-
packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/redacted_path/postgres_services/venv/lib/python3.8/site-
packages/django/db/backends/base/base.py", line 219, in ensure_connection
self.connect()
File "/redacted_path/postgres_services/venv/lib/python3.8/site-
packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/redacted_path/postgres_services/venv/lib/python3.8/site-
packages/django/db/backends/base/base.py", line 199, in connect
conn_params = self.get_connection_params()
File "/redacted_path/postgres_services/venv/lib/python3.8/site-
packages/django/db/backends/postgresql/base.py", line 157, in
get_connection_params
raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: settings.DATABASES is
improperly configured. Please supply the NAME value.
}}}
While if I just add the NAME value it works just fine. This makes total
sense as if you check the postgres base file it checks:
{{{#!python
if settings_dict['NAME'] == '':
raise ImproperlyConfigured(
"settings.DATABASES is improperly configured. "
"Please supply the NAME value.")
if len(settings_dict['NAME'] or '') > self.ops.max_name_length():
raise ImproperlyConfigured(
"The database name '%s' (%d characters) is longer than "
"PostgreSQL's limit of %d characters. Supply a shorter NAME "
"in settings.DATABASES." % (
settings_dict['NAME'],
len(settings_dict['NAME']),
self.ops.max_name_length(),
)
)
}}}
The first if evaluates to true. `settings_dict['NAME']` must be getting
defaulted to an empty string, because I'm not setting it to empty, I'm
entirely leaving it out.
> so only setting `PGSERVICE` for
[https://github.com/django/django/blob/2a76f4313423a3b91caade4fce71790630ef9152/django/db/backends/postgresql/client.py#L10-L44
the underlying command-line client] and docs are missing.
I don't mind adding support for those either.
> > I don't mind creating a pull request for it, but would like to know
why it hasn't been added, ...
>
> Because you're the first to suggest it.
Oh I was sure I wouldn't have been. I've looked up how to use postgres
services for many projects, and nearly always find people asking, even
when it's for much much smaller projects. That's rather interesting.
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: levihb | Owner: nobody
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: database postgresql | Triage Stage: Accepted
postgres service pg_service |
config |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak):
> It doesn't work for me. E.g.:
You're right. I've only checked running tests which works fine.
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:6>
Django
-------------------------------------+-------------------------------------
| Ramezani
Type: New feature | Status: assigned
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: database postgresql | Triage Stage: Accepted
postgres service pg_service |
config |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: nobody => Hasan Ramezani
* status: new => assigned
* has_patch: 0 => 1
Comment:
[https://github.com/django/django/pull/13808 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:7>
Django
-------------------------------------+-------------------------------------
| Ramezani
Type: New feature | Status: assigned
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: database postgresql | Triage Stage: Accepted
postgres service pg_service |
config |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
* needs_docs: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:8>
Django
-------------------------------------+-------------------------------------
| Ramezani
Type: New feature | Status: assigned
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: database postgresql | Triage Stage: Accepted
postgres service pg_service |
config |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
* needs_docs: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:9>
Django
-------------------------------------+-------------------------------------
| Ramezani
Type: New feature | Status: assigned
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: database postgresql | Triage Stage: Accepted
postgres service pg_service |
config |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
* needs_docs: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:10>
Django
-------------------------------------+-------------------------------------
| Ramezani
Type: New feature | Status: assigned
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: database postgresql | Triage Stage: Accepted
postgres service pg_service |
config |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
* needs_docs: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:11>
Django
-------------------------------------+-------------------------------------
| Ramezani
Type: New feature | Status: assigned
Component: Database layer | Version: 3.1
(models, ORM) |
Severity: Normal | Resolution:
postgres service pg_service | checkin
config |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:12>
Django
-------------------------------------+-------------------------------------
| Ramezani
Component: Database layer | Version: 3.1
(models, ORM) |
Keywords: database postgresql | Triage Stage: Ready for
postgres service pg_service | checkin
config |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"dcb3ad3319cad5c270a1856fd5f355e37cf9d474" dcb3ad33]:
{{{
#!CommitTicketReference repository=""
revision="dcb3ad3319cad5c270a1856fd5f355e37cf9d474"
Fixed #32292 -- Added support for connection by service name to
PostgreSQL.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:13>
Django
-------------------------------------+-------------------------------------
| Ramezani
Type: New feature | Status: closed
(models, ORM) |
Severity: Release blocker | Resolution: fixed
Keywords: database postgresql | Triage Stage: Accepted
postgres service pg_service |
config |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* version: 3.1 => 4.0
* severity: Normal => Release blocker
* stage: Ready for checkin => Accepted
Comment:
I've noticed an issue when checking related ticket #32456. Using `.pgpass`
with [https://github.com/django/django/pull/14043 PR] doesn't work because
`dbshell` is trying to use the default `'postgres'` database when `NAME`
is not set and a service name is in use. It shouldn't pass `dbname` as in
the case of
[https://github.com/django/django/pull/13808#discussion_r559328449
connections].
[https://github.com/django/django/pull/14054 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:14>
Django
-------------------------------------+-------------------------------------
| Ramezani
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Release blocker | Resolution:
postgres service pg_service |
config |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: closed => new
* resolution: fixed =>
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:15>
Django
-------------------------------------+-------------------------------------
| Ramezani
Type: New feature | Status: new
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: database postgresql | Triage Stage: Accepted
postgres service pg_service |
config |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by GitHub <noreply@…>):
In [changeset:"8908846444156eeae88be70332f96759cfce7740" 8908846]:
{{{
#!CommitTicketReference repository=""
revision="8908846444156eeae88be70332f96759cfce7740"
Refs #32292 -- Made dbshell do not use 'postgres' database when service
name is set.
Regression in dcb3ad3319cad5c270a1856fd5f355e37cf9d474.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:16>
Django
-------------------------------------+-------------------------------------
| Ramezani
Component: Database layer | Version: 4.0
(models, ORM) |
Keywords: database postgresql | Triage Stage: Accepted
postgres service pg_service |
config |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => fixed
--
Ticket URL: <https://code.djangoproject.com/ticket/32292#comment:17>