Hi,
I found that django migrations don't designed for live updates on
large tables for postgres, eg. index creation doesn't use concurrently
or alter table operation can take ACCESS EXCLUSIVE lock for long time.
Before I mostly wrote custom migrations, but after one of downtime I
decided to avoid most common cases at all and wrote postgres backend
that helps me in it:
https://github.com/tbicr/django-pg-zero-downtime-migrations. When I wrote it I found that it's a pretty easy extend core postgres backend, but a few things required rewrite on my side:
-
https://github.com/django/django/blob/master/django/db/backends/base/schema.py#L152
used both for table columns creation and altering table add column, but
not null constraint and index constraints can't be changed without
whole method rewriting
- for one core sql statement I found more "safe" equivalent with two sql statements, but it's a bit hard to execute it, for example more save create invalid constraint and than validate it or create concurrently index and than create unique constraint
also I found that not all constraint names quoted in generated sql, for example add_unique_together got `ALTER TABLE "table_name" ADD CONSTRAINT table_name_f1_f2_hash_uniq UNIQUE ("f1", "f2")`, but expected `ALTER TABLE "table_name" ADD CONSTRAINT "table_name_f1_f2_hash_uniq" UNIQUE ("f1", "f2")`
So
I just curious is anybody interested to make django migrations more
reliable for large tables in prod or interested in some improvements highlighted above, that
can simplify writing backends for this purpose?
I also will be appreciated if somebody can share another solutions for this purpose.
Paveł