[Django] #24018: Support journal

Django

unread,

Dec 17, 2014, 8:20:21 PM12/17/14

to django-...@googlegroups.com

#24018: Support journal_mode=WAL for sqlite
-------------------------------------+-------------------------------------
Reporter: funkybob | Owner: nobody
Type: New | Status: new
feature | Version: master
Component: Database | Keywords:
layer (models, ORM) | Has patch: 0
Severity: Normal | Needs tests: 0
Triage Stage: | Easy pickings: 0
Unreviewed |
Needs documentation: 0 |
Patch needs improvement: 0 |
UI/UX: 0 |
-------------------------------------+-------------------------------------
SQLite, as of 3.7, supports a WAL journal mode
[https://www.sqlite.org/wal.html]

It's been my experience that enabling this can dramatically improve
performance, especially under heavy writes. I've seen some write-heavy
operations go from a couple of minutes to a few seconds.

It can be enabled by executing the command:

PRAGMA journal_mode=wal;

which will return 'wal' on success.

If it fails to apply for any reason, it will return the mode it is in -
'delete' or 'memory'.

I propose to add JOURNAL_MODE as an option to the sqlite backend.

however, some quick tinkering shows that the sqlite backend doesn't ever
explicitly create the DB, so it's not immediately apparent how to detect
if/when this command should be issued.

I am still investigating.

--
Ticket URL: <https://code.djangoproject.com/ticket/24018>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,

Dec 17, 2014, 8:21:43 PM12/17/14

to django-...@googlegroups.com

#24018: Support journal_mode=WAL for sqlite
-------------------------------------+-------------------------------------

* owner: nobody => funkybob
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:1>

Django

unread,

Dec 18, 2014, 8:19:50 AM12/18/14

to django-...@googlegroups.com

#24018: Support journal_mode=WAL for sqlite
-------------------------------------+-------------------------------------

Reporter: funkybob | Owner: funkybob
Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) | Resolution:

Severity: Normal | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 0 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by timgraham):

* stage: Unreviewed => Accepted

--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:2>

Django

unread,

Jun 13, 2017, 5:33:48 PM6/13/17

to django-...@googlegroups.com

#24018: Support journal_mode=WAL for sqlite
-------------------------------------+-------------------------------------

Reporter: Curtis Maloney | Owner: Curtis
| Maloney

Type: New feature | Status: assigned
Component: Database layer | Version: master
(models, ORM) |

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by tonnzor):

Any progress here?

We could add `init_command` to `OPTIONS` for sqlite backend like for
MySQL.

Then you would be able to use any PRAGMA we need:

{{{
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
'OPTIONS': {
'init_command': 'PRAGMA journal_mode=wal;',
}
}
}
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:3>

Django

unread,

Jun 13, 2017, 5:37:46 PM6/13/17

to django-...@googlegroups.com

#24018: Support journal_mode=WAL for sqlite
-------------------------------------+-------------------------------------

Comment (by tonnzor):

As a workaround, you could use https://stackoverflow.com/a/6843199 (use
`connection_created` signal)

--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:4>

Django

unread,

Sep 1, 2021, 5:11:20 PM9/1/21

to django-...@googlegroups.com

#24018: Support journal_mode=WAL for sqlite
-------------------------------------+-------------------------------------

Reporter: Curtis Maloney | Owner: Aaron
| Linville

Type: New feature | Status: assigned

Component: Database layer | Version: dev

Changes (by Aaron Linville):

* owner: Curtis Maloney => Aaron Linville

--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:5>

Django

unread,

Sep 1, 2021, 5:45:13 PM9/1/21

to django-...@googlegroups.com

#24018: Support journal_mode=WAL for sqlite
-------------------------------------+-------------------------------------

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Aaron Linville):

* has_patch: 0 => 1

Comment:

https://github.com/django/django/pull/14824

I needed the ability to adjust some pragma settings associated with
caching but as Artem noted, this will work for any pragma setting.

--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:6>

Django

unread,

Sep 2, 2021, 2:53:44 AM9/2/21

to django-...@googlegroups.com

#24018: Support journal_mode=WAL for sqlite
-------------------------------------+-------------------------------------

Has patch: 1 | Needs documentation: 1
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mariusz Felisiak):

* needs_better_patch: 0 => 1
* needs_docs: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:7>

Django

unread,

Sep 2, 2021, 3:05:36 AM9/2/21

to django-...@googlegroups.com

#24018: Support journal_mode=WAL for sqlite
-------------------------------------+-------------------------------------

Comment (by Carlton Gibson):

Maybe allowing `PRAGMAS` to be set is a good idea, in general (e.g.
#31765) but the WAL setting is persistent:

> The persistence of WAL mode means that applications can be converted to
using SQLite in WAL mode without making any changes to the application
itself. One has merely to run "PRAGMA journal_mode=WAL;" on the database
file(s) using the command-line shell or other utility, then restart the
application. [https://sqlite.org/wal.html#persistence_of_wal_mode SQLite
Docs].

So narrowly, running that a single time is sufficient, and an entry in the
[https://docs.djangoproject.com/en/3.2/ref/databases/#sqlite-notes SQLite
notes] may be the quickest route forward (for this narrower issue).

--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:8>

Django

unread,

Aug 17, 2023, 6:00:47 AM8/17/23

to django-...@googlegroups.com

#24018: Support journal_mode=WAL for sqlite
-------------------------------------+-------------------------------------

Comment (by Afonso Silva):

Based on the previous answers, a workaround for allowing `init_command`
similar to MySQL is to use a signal receiver for `connection_created`,
e.g.:

{{{#!python
from django.conf import settings
from django.db.backends.signals import connection_created
from django.dispatcher import receiver

@receiver(connection_created)
def init_command(sender, connection, **kwargs) -> None:
command = connection.settings_dict["OPTIONS"].get("init_command")
if connection.vendor == "sqlite" and command:
cursor = connection.cursor()
cursor.execute(command)
}}}

This way you could add `init_command` to your `settings.py`:

{{{#!python

DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',

'NAME': BASE_DIR / 'db.sqlite3',
'OPTIONS': {
'init_command': 'PRAGMA journal_mode=wal;'
}
}
}

}}}

I think having this natively in Django would be useful, is there any help
needed for the previous linked PR?

--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:9>

Django

unread,

Aug 17, 2023, 9:10:12 PM8/17/23

to django-...@googlegroups.com

#24018: Support journal_mode=WAL for sqlite
-------------------------------------+-------------------------------------

Comment (by Aaron Linville):

Replying to [comment:9 Afonso Silva]:

> I think having this natively in Django would be useful, is there any
help needed for the previous linked PR?

I updated the PR and modified it with a more compelling example;
`synchronous=0` will not persist.

--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:10>

Django

unread,

Feb 7, 2024, 4:27:22 PM2/7/24

to django-...@googlegroups.com

#24018: Support journal_mode=WAL for sqlite
-------------------------------------+-------------------------------------

Comment (by bcail):

@Aaron, are you still wanting to work on this? If so, if you fix the tests
you could update the flags on this issue so it goes back in the list of
tickets to be reviewed.
--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:11>

Django

unread,

Feb 8, 2024, 6:27:33 PM2/8/24

to django-...@googlegroups.com

#24018: Support journal_mode=WAL for sqlite
-------------------------------------+-------------------------------------

Comment (by Aaron Linville):

Replying to [comment:11 bcail]:

> @Aaron, are you still wanting to work on this? If so, if you fix the
tests you could update the flags on this issue so it goes back in the list
of tickets to be reviewed.

Yes, I've rebased it on latest and fixed the tests I think (a couple are
still pending). Assume you mean uncheck the Needs doc and Patch needs
improvement?
--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:12>

Django

unread,

Feb 9, 2024, 8:00:08 AM2/9/24

to django-...@googlegroups.com

#24018: Support journal_mode=WAL for sqlite
-------------------------------------+-------------------------------------

Changes (by bcail):

* cc: bcail (added)

Comment:

Yup, exactly - so it shows up on the
[https://code.djangoproject.com/query?status=!closed&needs_better_patch=0&needs_tests=0&needs_docs=0&has_patch=1&stage=Accepted&desc=1&order=changetime
Patch review page].
--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:13>

Django

unread,

Feb 9, 2024, 8:42:15 AM2/9/24

to django-...@googlegroups.com

#24018: Support journal_mode=WAL for sqlite
-------------------------------------+-------------------------------------

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Aaron Linville):

* needs_better_patch: 1 => 0
* needs_docs: 1 => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:14>

Django

unread,

Feb 14, 2024, 9:29:48 AM2/14/24

to django-...@googlegroups.com

#24018: Support setting pragma options for SQLite.

Changes (by Mariusz Felisiak):

* summary: Support journal_mode=WAL for sqlite => Support setting pragma
options for SQLite.

--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:15>

Django

unread,

Feb 16, 2024, 12:55:33 AM2/16/24

to django-...@googlegroups.com

Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:16>

Django

unread,

Feb 16, 2024, 4:35:28 PM2/16/24

to django-...@googlegroups.com

#24018: Support setting pragma options for SQLite.
-------------------------------------+-------------------------------------
Reporter: Curtis Maloney | Owner: Aaron
| Linville

Type: New feature | Status: closed

Component: Database layer | Version: dev
(models, ORM) |

Severity: Normal | Resolution: fixed

Changes (by Mariusz Felisiak <felisiak.mariusz@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"7a05b8a2fac57e32a61726893d4601352c1d1c8d" 7a05b8a]:
{{{#!CommitTicketReference repository=""
revision="7a05b8a2fac57e32a61726893d4601352c1d1c8d"
Fixed #24018 -- Allowed setting pragma options on SQLite.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/24018#comment:17>

Reply all

Reply to author

Forward

[Django] #24018: Support journal_mode=WAL for sqlite

Django

Django

Django

Django

Django

Django

Django

Django

Django

Django

Django

Django

Django

Django

Django

Django

Django

Django