[Django] #30269: AlterField in database_operations of SeparateDatabaseAndState migration doesn't set NOT NULL constraint for PostgreSQL

[Django]

#30269: AlterField in database_operations of SeparateDatabaseAndState migration
doesn't set NOT NULL constraint for PostgreSQL
--------------------------------------+------------------------
Reporter: Dmitrii | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 2.1
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
--------------------------------------+------------------------
For changing a field, and not having some unpredictable behavior we employ
multi-stage deployments.

So for example we have a field:
{{{
name = models.CharField(max_length=255, null=True)
}}}

And we want to make it non-nullable. To make it so we perform two
migrations, firstly the state:

{{{
operations = [
# Plus whatever data migration is needed for the NULL values
migrations.SeparateDatabaseAndState(
state_operations=[
migrations.AlterField(
model_name='testmodel',
name='name',
field=models.CharField(max_length=255)
)
]
)
]
}}}

After all parts of the system have been updated with code that handles the
given field only in a non-nullable way, we can safely set it to NON NULL
in the database:

{{{
operations = [
migrations.SeparateDatabaseAndState(
database_operations=[
migrations.AlterField(
model_name='testmodel',
name='name',
field=models.CharField(max_length=255)
)
]
)
]
}}}

But the last migration doesn't actually perform any changes, the field
remains nullable in the database. This can be circumvented by using
migrations.RunSQL, but one can easily miss out the need to do that.

PostgreSQL in use: 9.6 official docker image.
Originally found for Django 1.8, reproduced for Django 2.1.7.

Didn't reproduce with SQLite.

--
Ticket URL: <https://code.djangoproject.com/ticket/30269>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#30269: AlterField in database_operations of SeparateDatabaseAndState migration
doesn't set NOT NULL constraint for PostgreSQL

----------------------------------+--------------------------------------
Reporter: Dmitrii Prihodco | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 2.1
Severity: Normal | Resolution: invalid

Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

----------------------------------+--------------------------------------
Changes (by Simon Charette):

* status: new => closed
* resolution: => invalid

Comment:

All the operations operate on the difference between the implicit previous
state and the one that applying the operation on it would generate and
`SeparateDatabaseAndState` behaves the same way.

What you are doing here by breaking the desired changes in two migrations
by creating an asymmetry between the project state and the database state.
When the second operation runs the state already has it's field altered by
the previous `SeparateDatabaseAndState` and thus the migration framework
considers your database operation as a noop.

In summary this is working as designed and performing the second operation
using `RunSQL` is the right approach if you want to effectively break this
in two migrations. A preferred way of ensuring no downtime for such field
alteration is usually to simply run such migrations once the code is
deployed instead.

--
Ticket URL: <https://code.djangoproject.com/ticket/30269#comment:1>