https://github.com/django/django/blob/master/django/contrib/postgres/apps.py#L20
https://github.com/django/django/blob/stable/1.11.x/django/contrib/postgres/signals.py#L16
Behind the scenes `register_hstore` runs sql query to get `oid` of
`hstore` type:
https://github.com/psycopg/psycopg2/blob/master/lib/extras.py#L885
This happens on every(!) request (unless persistent connections enabled)
SELECT t.oid, %s
FROM pg_type t JOIN pg_namespace ns
ON typnamespace = ns.oid
WHERE typname = 'hstore';
This is just huge overhead, since `register_hstore` accepts `oid` argument
to avoid hitting database every call.
We have seen significant latency spikes because of this.
Possible solution would be have configurable `GET_HSTORE_OID_FUNC` in
`DATABASES->OPTIONS` setting which can be cached in any way.
This will also maintain backwards compatibility.
I can send patch for this if proposed solution is acceptable to be merged
in master.
--
Ticket URL: <https://code.djangoproject.com/ticket/28334>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Old description:
> On each newly created database connection `django.contrib.postgresql`
> tries to register hstore type for this connection via
> `connection_created` signal and `register_hstore` function.
>
> https://github.com/django/django/blob/master/django/contrib/postgres/apps.py#L20
> https://github.com/django/django/blob/stable/1.11.x/django/contrib/postgres/signals.py#L16
>
> Behind the scenes `register_hstore` runs sql query to get `oid` of
> `hstore` type:
> https://github.com/psycopg/psycopg2/blob/master/lib/extras.py#L885
>
> This happens on every(!) request (unless persistent connections enabled)
> SELECT t.oid, %s
> FROM pg_type t JOIN pg_namespace ns
> ON typnamespace = ns.oid
> WHERE typname = 'hstore';
>
> This is just huge overhead, since `register_hstore` accepts `oid`
> argument to avoid hitting database every call.
> We have seen significant latency spikes because of this.
>
> Possible solution would be have configurable `GET_HSTORE_OID_FUNC` in
> `DATABASES->OPTIONS` setting which can be cached in any way.
> This will also maintain backwards compatibility.
>
> I can send patch for this if proposed solution is acceptable to be merged
> in master.
New description:
On each newly created database connection `django.contrib.postgresql`
tries to register hstore type for this connection via `connection_created`
signal and `register_hstore` function.
https://github.com/django/django/blob/master/django/contrib/postgres/apps.py#L20
https://github.com/django/django/blob/stable/1.11.x/django/contrib/postgres/signals.py#L16
Behind the scenes `register_hstore` runs sql query to get `oid` of
`hstore` type:
https://github.com/psycopg/psycopg2/blob/master/lib/extras.py#L885
This happens on every(!) request (unless persistent connections enabled)
SELECT t.oid, %s
FROM pg_type t JOIN pg_namespace ns
ON typnamespace = ns.oid
WHERE typname = 'hstore';
This is just huge overhead, since `register_hstore` accepts `oid` argument
to avoid hitting database every call.
We have seen significant latency spikes because of this.
Possible solution would be to have configurable `GET_HSTORE_OID_FUNC` in
`DATABASES->OPTIONS` setting which can be cached in any way.
This will also maintain backwards compatibility.
I can send patch for this if proposed solution is acceptable to be merged
in master.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/28334#comment:1>
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/28334#comment:2>
* status: new => assigned
* owner: (none) => Igor Gumenyuk
--
Ticket URL: <https://code.djangoproject.com/ticket/28334#comment:3>
* has_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/28334#comment:4>
* needs_better_patch: 0 => 1
Comment:
There are test failures on the pull request.
--
Ticket URL: <https://code.djangoproject.com/ticket/28334#comment:5>
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/28334#comment:6>
Comment (by Igor Gumenyuk):
All issues resolved, there are no failed tests
--
Ticket URL: <https://code.djangoproject.com/ticket/28334#comment:7>
* needs_tests: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/28334#comment:8>
* needs_tests: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/28334#comment:9>
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/28334#comment:10>
* needs_better_patch: 1 => 0
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/28334#comment:11>
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"86a18dc46aae3a4a6410e3f3d864fa0ec4e5b2cd" 86a18dc]:
{{{
#!CommitTicketReference repository=""
revision="86a18dc46aae3a4a6410e3f3d864fa0ec4e5b2cd"
Fixed #28334 -- Added caching for hstore/citext OIDs.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28334#comment:12>
Comment (by Tim Graham <timograham@…>):
In [changeset:"f7b0532ec0cf703849ef50e513e3dc6b5483a6b0" f7b0532]:
{{{
#!CommitTicketReference repository=""
revision="f7b0532ec0cf703849ef50e513e3dc6b5483a6b0"
Refs #28334 -- Fixed crash in hstore/citext oid caching during test db
creation.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28334#comment:13>