[Django] #30033: SQLite schema editor should use the documented to emulate table alterations on SQlite3.
Django
on SQlite3.
-------------------------------------+-------------------------------------
Reporter: Simon | Owner: Simon Charette
Charette |
Type: | Status: assigned
Cleanup/optimization |
Component: | Version: master
Migrations |
Severity: Normal | Keywords:
Triage Stage: Accepted | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
In the light of #29182 SQLite's main author mentioned that we should
conform to the documented procedure to emulate table alterations on
SQlite3.
Right now the [https://www.sqlite.org/lang_altertable.html#caution
SQLite's schema editor follows a procedure documented as incorrect] and
prevents usages of the SQLite 3.26 improvements regarding table and column
renames while foreign key constraints are disabled.
--
Ticket URL: <https://code.djangoproject.com/ticket/30033>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
on SQlite3.
-------------------------------------+-------------------------------------
Type: | Charette
Cleanup/optimization | Status: assigned
Component: Migrations | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
> In the light of #29182 SQLite's main author mentioned that we should
> conform to the documented procedure to emulate table alterations on
> SQlite3.
>
> Right now the [https://www.sqlite.org/lang_altertable.html#caution
> SQLite's schema editor follows a procedure documented as incorrect] and
> prevents usages of the SQLite 3.26 improvements regarding table and
> column renames while foreign key constraints are disabled.
New description:
In the light of #29182
[https://github.com/django/django/pull/10733#issuecomment-445862681
SQLite's main author mentioned] that we should conform to the documented
procedure to emulate table alterations on SQlite3.
Right now the [https://www.sqlite.org/lang_altertable.html#caution
SQLite's schema editor follows a procedure documented as incorrect] and
prevents usages of the SQLite 3.26 improvements regarding table and column
renames while foreign key constraints are disabled.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/30033#comment:1>
Django
on SQlite3.
-------------------------------------+-------------------------------------
Type: | Charette
Cleanup/optimization | Status: assigned
Component: Migrations | Version: master
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* has_patch: 0 => 1
Old description:
> In the light of #29182
> [https://github.com/django/django/pull/10733#issuecomment-445862681
> SQLite's main author mentioned] that we should conform to the documented
> procedure to emulate table alterations on SQlite3.
>
> Right now the [https://www.sqlite.org/lang_altertable.html#caution
> SQLite's schema editor follows a procedure documented as incorrect] and
> prevents usages of the SQLite 3.26 improvements regarding table and
> column renames while foreign key constraints are disabled.
New description:
In the light of #29182
[https://github.com/django/django/pull/10733#issuecomment-445830242
SQLite's main author mentioned] that we should conform to the documented
procedure to emulate table alterations on SQlite3.
Right now the [https://www.sqlite.org/lang_altertable.html#caution
SQLite's schema editor follows a procedure documented as incorrect] and
prevents usages of the SQLite 3.26 improvements regarding table and column
renames while foreign key constraints are disabled.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/30033#comment:2>
Django
alterations on SQlite3.
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
Type: | Charette
Cleanup/optimization | Status: assigned
Component: Migrations | Version: master
Severity: Normal | Resolution:
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/30033#comment:3>
Django
alterations on SQlite3.
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
Type: | Charette
Component: Migrations | Version: master
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Carlton Gibson <carlton.gibson@…>):
In [changeset:"894cb13779e6d092974c873bd2cf1452554d2e06" 894cb13]:
{{{
#!CommitTicketReference repository=""
revision="894cb13779e6d092974c873bd2cf1452554d2e06"
Refs #29182 -- Stopped relying on legacy alter table semantic on SQLite
3.26+.
SQLite 3.26 changed the behavior of table and column renaming operations
to
repoint foreign key references even if foreign key checks are disabled.
This makes the workarounds in place to simulate this behavior unnecessary
on
SQLite 3.26+. Refs #30033.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30033#comment:5>
Django
alterations on SQlite3.
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
Type: | Charette
Cleanup/optimization | Status: closed
Component: Migrations | Version: master
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"7289874adceec46b5367ec3157cdd10c711253a0" 7289874a]:
{{{
#!CommitTicketReference repository=""
revision="7289874adceec46b5367ec3157cdd10c711253a0"
Fixed #30033 -- Conformed to the recommended table alterations procedure
on SQlite3.
Refs #29182.
The previous implementation was following a procedure explicitly
documented
as incorrect and was the origin of the breakage experienced on SQLite 3.26
release that were addressed by c8ffdbe514b55ff5c9a2b8cb8bbdf2d3978c188f.
Thanks to Richard Hipp for pointing out the usage of the incorrect
procedure.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30033#comment:4>
Django
alterations on SQlite3.
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
Type: | Charette
Cleanup/optimization | Status: closed
Component: Migrations | Version: master
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"790d108c97f0cf9caa02c72791f2bf158d308fcd" 790d108c]:
{{{
#!CommitTicketReference repository=""
revision="790d108c97f0cf9caa02c72791f2bf158d308fcd"
Refs #30033 -- Checked constraints before committing SQLite schema
changes.
This order of operations is more in line with SQLite's documented table
rebuild procedure and ensures that changes aren't committed if foreign key
integrity is broken.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30033#comment:6>
Django
alterations on SQlite3.
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
Type: | Charette
Cleanup/optimization | Status: closed
Component: Migrations | Version: master
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"ce8b65ac5e8e20912c36ab5c714ddf11de3b664c" ce8b65a]:
{{{
#!CommitTicketReference repository=""
revision="ce8b65ac5e8e20912c36ab5c714ddf11de3b664c"
Fixed #30054 -- Implemented cascaded flush on SQLite.
This is required to maintain foreign key integrity when using
TransactionTestCase.available_apps.
Refs #30033, #14204, #20483.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30033#comment:7>
Django
alterations on SQlite3.
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
Type: | Charette
Component: Migrations | Version: master
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: closed => new
* has_patch: 1 => 0
* resolution: fixed =>
* severity: Normal => Release blocker
* stage: Ready for checkin => Accepted
Comment:
On Ubuntu 16.04 which has SQLite 3.11, some problems appeared after
7289874adceec46b5367ec3157cdd10c711253a0:
{{{
======================================================================
ERROR: test_m2m_rename_field_in_target_model (schema.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/media/sf_django/django/db/backends/utils.py", line 82, in
_execute
return self.cursor.execute(sql)
File "/media/sf_django/django/db/backends/sqlite3/base.py", line 360, in
execute
return Database.Cursor.execute(self, query)
sqlite3.OperationalError: no such table: schema_tagm2mtest
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/media/sf_django/tests/schema/tests.py", line 70, in tearDown
self.delete_tables()
File "/media/sf_django/tests/schema/tests.py", line 97, in delete_tables
connection.enable_constraint_checking()
File "/media/sf_django/django/db/backends/sqlite3/schema.py", line 34,
in __exit__
self.connection.check_constraints()
File "/media/sf_django/django/db/backends/sqlite3/base.py", line 320, in
check_constraints
column_name, referenced_column_name,
File "/media/sf_django/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(sql, params, many=False,
executor=self._execute)
File "/media/sf_django/django/db/backends/utils.py", line 76, in
_execute_with_wrappers
return executor(sql, params, many, context)
File "/media/sf_django/django/db/backends/utils.py", line 84, in
_execute
return self.cursor.execute(sql, params)
File "/media/sf_django/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/media/sf_django/django/db/backends/utils.py", line 82, in
_execute
return self.cursor.execute(sql)
File "/media/sf_django/django/db/backends/sqlite3/base.py", line 360, in
execute
return Database.Cursor.execute(self, query)
django.db.utils.OperationalError: no such table: schema_tagm2mtest
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30033#comment:8>
Django
alterations on SQlite3.
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
Type: | Charette
Cleanup/optimization | Status: new
Component: Migrations | Version: master
Severity: Release blocker | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"0b54ab0675a5443dbda2fc48e98ab474c295ac0c" 0b54ab06]:
{{{
#!CommitTicketReference repository=""
revision="0b54ab0675a5443dbda2fc48e98ab474c295ac0c"
Refs #30033 -- Fixed schema's test_m2m_rename_field_in_target_model test
failure on SQLite < 3.20.
Mixing local test models with non-local models resulted in a referential
integrity error during tear down since the models are removed in separate
schema editor instances which each check constraints.
Failure appeared after 7289874adceec46b5367ec3157cdd10c711253a0.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30033#comment:9>
Django
alterations on SQlite3.
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
Type: | Charette
Component: Migrations | Version: master
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => fixed
* severity: Release blocker => Normal
--
Ticket URL: <https://code.djangoproject.com/ticket/30033#comment:10>
Django
alterations on SQlite3.
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
Type: | Charette
Cleanup/optimization | Status: closed
Component: Migrations | Version: master
Severity: Normal | Resolution: fixed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"e8cb0dcc19c4bcabee1d856b7fae1b54fa3e3380" e8cb0dcc]:
{{{
#!CommitTicketReference repository=""
revision="e8cb0dcc19c4bcabee1d856b7fae1b54fa3e3380"
[2.2.x] Refs #30033 -- Doc'd change regarding apps without migrations
depending on apps with migrations.
The addition of self.connection.check_constraints() in
7289874adceec46b5367ec3157cdd10c711253a0 is the cause.
Backport of fcfb7306586184f08c7f75c174f95b54a212320d from master.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30033#comment:12>
Django
alterations on SQlite3.
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Simon
Type: | Charette
Cleanup/optimization | Status: closed
Component: Migrations | Version: master
Severity: Normal | Resolution: fixed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"fcfb7306586184f08c7f75c174f95b54a212320d" fcfb7306]:
{{{
#!CommitTicketReference repository=""
revision="fcfb7306586184f08c7f75c174f95b54a212320d"
Refs #30033 -- Doc'd change regarding apps without migrations depending on
apps with migrations.
The addition of self.connection.check_constraints() in
7289874adceec46b5367ec3157cdd10c711253a0 is the cause.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/30033#comment:11>