[Django] #33899: migrations.RemoveField causes OperationalError "no such column" upon migration
Django
migration
-------------------------------------+-------------------------------------
Reporter: cessor | Owner: nobody
Type: Bug | Status: new
Component: | Version: 4.1
Migrations | Keywords: Migration Sqlite
Severity: Normal | Docker
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
== Description
I encountered the following error with django 4.1 in my Gitlab CI/CD
Pipeline. When I bumped django versions from 4.0.7 to 4.1. my pipeline
broke during the testing stage; specifically during db migrations. I have
not changed the any other source code.
== Steps to reproduce
Minimal example attached. Run `make green` to see that it works with
4.0.7, run `make red` to see that it does not work with 4.1.
Manual steps:
0. Install django 4.1
1. Create a new project
2. Create an app
3. Install app in project
4. Create a model
5. Add field on model
6. Make migrations
7. Remove field
8. Make migrations
The migration should fail.
--
Ticket URL: <https://code.djangoproject.com/ticket/33899>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
migration
-------------------------------------+-------------------------------------
Reporter: cessor | Owner: nobody
Type: Bug | Status: new
Severity: Normal | Resolution:
Keywords: Migration Sqlite | Triage Stage:
Docker | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* Attachment "django-bug-33899.tgz" added.
Minimal Example
Django
migration
-------------------------------------+-------------------------------------
Reporter: cessor | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Keywords: Migration Sqlite | Triage Stage:
Docker | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
> == Description
>
> I encountered the following error with django 4.1 in my Gitlab CI/CD
> Pipeline. When I bumped django versions from 4.0.7 to 4.1. my pipeline
> broke during the testing stage; specifically during db migrations. I have
> not changed the any other source code.
>
> == Steps to reproduce
>
> Minimal example attached. Run `make green` to see that it works with
> 4.0.7, run `make red` to see that it does not work with 4.1.
>
> Manual steps:
>
> 0. Install django 4.1
> 1. Create a new project
> 2. Create an app
> 3. Install app in project
> 4. Create a model
> 5. Add field on model
> 6. Make migrations
> 7. Remove field
> 8. Make migrations
>
> The migration should fail.
New description:
== Description
I encountered the following error with django 4.1 in my Gitlab CI/CD
Pipeline. When I bumped django versions from 4.0.7 to 4.1. my pipeline
broke during the testing stage; specifically during db migrations. I have
not changed any other source code.
== Steps to reproduce
Minimal example attached. Run `make green` to see that it works with
4.0.7, run `make red` to see that it does not work with 4.1. It will build
and exercise a docker container which installs all dependencies in
isolation and sets up an example django app and run migrations.
Manual steps:
0. Install django 4.1
1. Create a new project
2. Create an app
3. Install app in project
4. Create a model
5. Add field on model
6. Make migrations
7. Remove field
8. Make migrations
The migration should fail.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/33899#comment:1>
Django
migration
-------------------------------------+-------------------------------------
Reporter: cessor | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Keywords: Migration Sqlite | Triage Stage:
Docker | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak):
I'm really puzzled. I cannot reproduce this issue manually, but `make red`
crashes.
--
Ticket URL: <https://code.djangoproject.com/ticket/33899#comment:2>
Django
migration
-------------------------------------+-------------------------------------
Reporter: cessor | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Keywords: Migration Sqlite | Triage Stage:
Docker | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by cessor):
Replying to [comment:2 Mariusz Felisiak]:
> I'm really puzzled. I cannot reproduce this issue manually, but `make
red` crashes.
Thank you for giving this a try. I investigated this some further and
realized that my manual description is not precise enough to reproduce the
error. The example I attached uses a SlugField, but the error does not
occur with other field types, such as CharFields. I noticed that
SlugFields set `db_index=True` by default (See Link 1), and could
reproduce the bug with other fields when setting `db_index=True`. I will
change the bug description accordingly.
Links:
- 1: SlugField:
[https://github.com/django/django/blob/main/django/db/models/fields/__init__.py#L2301]
--
Ticket URL: <https://code.djangoproject.com/ticket/33899#comment:3>
Django
migration
-------------------------------------+-------------------------------------
Reporter: cessor | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Keywords: Migration Sqlite | Triage Stage:
Docker | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* Attachment "migrations_33899.zip" added.
Regression test of the bug reported
--
Ticket URL: <https://code.djangoproject.com/ticket/33899>
Django
migration
-------------------------------------+-------------------------------------
Reporter: cessor | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 4.1
Severity: Normal | Resolution:
Docker |
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by fizaashraf37):
* has_patch: 0 => 1
* needs_tests: 0 => 1
* easy: 0 => 1
* stage: Unreviewed => Accepted
Comment:
The issue is valid. I have written the regression test of the exact test
case. And found the first bad commit as below:
{{{
3702819227fd0cdd9b581cd99e11d1561d51cbeb is the first bad commit
commit 3702819227fd0cdd9b581cd99e11d1561d51cbeb
Author: Mariusz Felisiak <felisiak...@gmail.com>
Date: Fri Feb 11 22:21:58 2022 +0100
Refs #32502 -- Avoided table rebuild when removing fields on SQLite
3.35.5+.
ALTER TABLE ... DROP COLUMN was introduced in SQLite 3.35+ however
a data corruption issue was fixed in SQLite 3.35.5.
django/db/backends/sqlite3/features.py | 2 ++
django/db/backends/sqlite3/schema.py | 10 ++++++++++
tests/schema/tests.py | 18 ++++++++++++++++++
3 files changed, 30 insertions(+)
bisect found first bad commit
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33899#comment:4>