[Django] #32502: Replace SQLite schema table rebuild by ALTER functions when supported

45 views
Skip to first unread message

Django

unread,
Mar 3, 2021, 9:05:54 AM3/3/21
to django-...@googlegroups.com
#32502: Replace SQLite schema table rebuild by ALTER functions when supported
------------------------------------------------+------------------------
Reporter: Simon Charette | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Migrations | Version: 3.1
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
------------------------------------------------+------------------------
Per https://www.sqlite.org/lang_altertable.html

- `ALTER TABLE RENAME COLUMN` is
[https://www.sqlite.org/search?s=c&q=RENAME+COLUMN supported since 3.25
but had issues with unique columns until 3.27]
- `ALTER TABLE ADD COLUMN` is
[https://www.sqlite.org/search?s=c&q=ADD+COLUMN supported since 3.2.0] not
sure why it wasn't used in the first place since 3.2 was released in 2005?
- `ALTER TABLE DROP COLUMN`
[https://www.sqlite.org/draft/releaselog/3_35_0.html will be supported by
3.35]

This will leave only `ALTER COLUMN` and `ADD/REMOVE CONSTRAINT` that must
be emulated on SQLite 3.35+

--
Ticket URL: <https://code.djangoproject.com/ticket/32502>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,
Mar 3, 2021, 2:43:29 PM3/3/21
to django-...@googlegroups.com
#32502: Replace SQLite schema table rebuild by ALTER functions when supported
-------------------------------------+-------------------------------------

Reporter: Simon Charette | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Someday/Maybe

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):

* version: 3.1 => master
* stage: Unreviewed => Someday/Maybe


Comment:

Thanks Simon!

Marking as "Someday/Maybe" because it can we fully implemented after
3.35.0 release (such as #32501).

--
Ticket URL: <https://code.djangoproject.com/ticket/32502#comment:1>

Django

unread,
Apr 6, 2021, 11:41:25 AM4/6/21
to django-...@googlegroups.com
#32502: Replace SQLite schema table rebuild by ALTER functions when supported
--------------------------------------+------------------------------------

Reporter: Simon Charette | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
Changes (by Mariusz Felisiak):

* stage: Someday/Maybe => Accepted


Comment:

https://sqlite.org/releaselog/3_35_0.html

--
Ticket URL: <https://code.djangoproject.com/ticket/32502#comment:2>

Django

unread,
Aug 25, 2021, 11:19:12 PM8/25/21
to django-...@googlegroups.com
#32502: Replace SQLite schema table rebuild by ALTER functions when supported
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Girish
Type: | Sontakke
Cleanup/optimization | Status: assigned

Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Girish Sontakke):

* owner: nobody => Girish Sontakke
* status: new => assigned


--
Ticket URL: <https://code.djangoproject.com/ticket/32502#comment:3>

Django

unread,
Dec 10, 2021, 6:06:42 AM12/10/21
to django-...@googlegroups.com
#32502: Replace SQLite schema table rebuild by ALTER functions when supported
--------------------------------------+------------------------------------

Reporter: Simon Charette | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------

Comment (by GitHub <noreply@…>):

In [changeset:"4e249d11a6e56ca8feb4b055b681cec457ef3a3d" 4e249d1]:
{{{
#!CommitTicketReference repository=""
revision="4e249d11a6e56ca8feb4b055b681cec457ef3a3d"
Refs #32502 -- Added SchemaEditor.prepare_default() on SQLite.
}}}

Django

unread,
Dec 10, 2021, 12:42:04 PM12/10/21
to django-...@googlegroups.com
#32502: Replace SQLite schema table rebuild by ALTER functions when supported
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Mariusz
Type: | Felisiak
Cleanup/optimization | Status: assigned

Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):

* owner: nobody => Mariusz Felisiak


* status: new => assigned

* has_patch: 0 => 1


Comment:

[https://github.com/django/django/pull/15175 PR]

--
Ticket URL: <https://code.djangoproject.com/ticket/32502#comment:4>

Django

unread,
Dec 11, 2021, 10:29:23 AM12/11/21
to django-...@googlegroups.com
#32502: Replace SQLite schema table rebuild by ALTER functions when supported
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Mariusz
Type: | Felisiak
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Mariusz Felisiak <felisiak.mariusz@…>):

In [changeset:"2f73e5406d54cb8945e187eff302a3a3373350be" 2f73e540]:
{{{
#!CommitTicketReference repository=""
revision="2f73e5406d54cb8945e187eff302a3a3373350be"
Refs #32502 -- Avoided table rebuild when adding fields with no default on
SQLite.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/32502#comment:5>

Django

unread,
Dec 11, 2021, 10:30:15 AM12/11/21
to django-...@googlegroups.com
#32502: Replace SQLite schema table rebuild by ALTER functions when supported
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Mariusz
Type: | Felisiak
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):

* has_patch: 1 => 0


Old description:

> Per https://www.sqlite.org/lang_altertable.html
>
> - `ALTER TABLE RENAME COLUMN` is
> [https://www.sqlite.org/search?s=c&q=RENAME+COLUMN supported since 3.25
> but had issues with unique columns until 3.27]
> - `ALTER TABLE ADD COLUMN` is
> [https://www.sqlite.org/search?s=c&q=ADD+COLUMN supported since 3.2.0]
> not sure why it wasn't used in the first place since 3.2 was released in
> 2005?
> - `ALTER TABLE DROP COLUMN`
> [https://www.sqlite.org/draft/releaselog/3_35_0.html will be supported by
> 3.35]
>
> This will leave only `ALTER COLUMN` and `ADD/REMOVE CONSTRAINT` that must
> be emulated on SQLite 3.35+

New description:

Per https://www.sqlite.org/lang_altertable.html

- `ALTER TABLE RENAME COLUMN` is
[https://www.sqlite.org/search?s=c&q=RENAME+COLUMN supported since 3.25
but had issues with unique columns until 3.27]
- `ALTER TABLE ADD COLUMN` is
[https://www.sqlite.org/search?s=c&q=ADD+COLUMN supported since 3.2.0] not
sure why it wasn't used in the first place since 3.2 was released in 2005?

(added in 2f73e5406d54cb8945e187eff302a3a3373350be)


- `ALTER TABLE DROP COLUMN`
[https://www.sqlite.org/draft/releaselog/3_35_0.html will be supported by
3.35]

This will leave only `ALTER COLUMN` and `ADD/REMOVE CONSTRAINT` that must
be emulated on SQLite 3.35+

--

--
Ticket URL: <https://code.djangoproject.com/ticket/32502#comment:6>

Django

unread,
Dec 30, 2021, 7:37:31 AM12/30/21
to django-...@googlegroups.com
#32502: Replace SQLite schema table rebuild by ALTER functions when supported
--------------------------------------+------------------------------------
Reporter: Simon Charette | Owner: (none)
Type: Cleanup/optimization | Status: new

Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
Changes (by Mariusz Felisiak):

* owner: Mariusz Felisiak => (none)
* status: assigned => new


--
Ticket URL: <https://code.djangoproject.com/ticket/32502#comment:7>

Django

unread,
Feb 11, 2022, 4:45:10 AM2/11/22
to django-...@googlegroups.com
#32502: Replace SQLite schema table rebuild by ALTER functions when supported
--------------------------------------+------------------------------------
Reporter: Simon Charette | Owner: (none)
Type: Cleanup/optimization | Status: new

Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
Description changed by Mariusz Felisiak:

Old description:

> Per https://www.sqlite.org/lang_altertable.html
>
> - `ALTER TABLE RENAME COLUMN` is
> [https://www.sqlite.org/search?s=c&q=RENAME+COLUMN supported since 3.25
> but had issues with unique columns until 3.27]
> - `ALTER TABLE ADD COLUMN` is
> [https://www.sqlite.org/search?s=c&q=ADD+COLUMN supported since 3.2.0]
> not sure why it wasn't used in the first place since 3.2 was released in

> 2005? (added in 2f73e5406d54cb8945e187eff302a3a3373350be)


> - `ALTER TABLE DROP COLUMN`
> [https://www.sqlite.org/draft/releaselog/3_35_0.html will be supported by
> 3.35]
>
> This will leave only `ALTER COLUMN` and `ADD/REMOVE CONSTRAINT` that must
> be emulated on SQLite 3.35+

New description:

Per https://www.sqlite.org/lang_altertable.html

- ~~`ALTER TABLE RENAME COLUMN` is

but had issues with unique columns until 3.27] ~~ (duplicate of #29763).


- `ALTER TABLE ADD COLUMN` is
[https://www.sqlite.org/search?s=c&q=ADD+COLUMN supported since 3.2.0] not
sure why it wasn't used in the first place since 3.2 was released in 2005?

(added in 2f73e5406d54cb8945e187eff302a3a3373350be)


- `ALTER TABLE DROP COLUMN`
[https://www.sqlite.org/draft/releaselog/3_35_0.html will be supported by
3.35]

This will leave only `ALTER COLUMN` and `ADD/REMOVE CONSTRAINT` that must
be emulated on SQLite 3.35+

--

--
Ticket URL: <https://code.djangoproject.com/ticket/32502#comment:8>

Django

unread,
Feb 11, 2022, 5:32:49 AM2/11/22
to django-...@googlegroups.com
#32502: Replace SQLite schema table rebuild by ALTER functions when supported
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Mariusz
Type: | Felisiak
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):

* owner: (none) => Mariusz Felisiak


* status: new => assigned
* has_patch: 0 => 1


Comment:

[https://github.com/django/django/pull/15417 PR]

--
Ticket URL: <https://code.djangoproject.com/ticket/32502#comment:9>

Django

unread,
Feb 11, 2022, 4:22:30 PM2/11/22
to django-...@googlegroups.com
#32502: Replace SQLite schema table rebuild by ALTER functions when supported
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Mariusz
Type: | Felisiak
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by GitHub <noreply@…>):

In [changeset:"3702819227fd0cdd9b581cd99e11d1561d51cbeb" 37028192]:
{{{
#!CommitTicketReference repository=""
revision="3702819227fd0cdd9b581cd99e11d1561d51cbeb"
Refs #32502 -- Avoided table rebuild when removing fields on SQLite
3.35.5+.

ALTER TABLE ... DROP COLUMN was introduced in SQLite 3.35+ however
a data corruption issue was fixed in SQLite 3.35.5.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/32502#comment:10>

Django

unread,
Feb 11, 2022, 4:27:48 PM2/11/22
to django-...@googlegroups.com
#32502: Replace SQLite schema table rebuild by ALTER functions when supported
-------------------------------------+-------------------------------------
Reporter: Simon Charette | Owner: Mariusz
Type: | Felisiak
Cleanup/optimization | Status: closed
Component: Migrations | Version: dev
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):

* status: assigned => closed
* resolution: => fixed


--
Ticket URL: <https://code.djangoproject.com/ticket/32502#comment:11>

Reply all
Reply to author
Forward
0 new messages