[Django] #29689: Replace os.listdir with os.scandir in FileSystemStorage.listdir

9 views
Skip to first unread message

Django

unread,
Aug 19, 2018, 7:36:58 PM8/19/18
to django-...@googlegroups.com
#29689: Replace os.listdir with os.scandir in FileSystemStorage.listdir
------------------------------------------------+--------------------------
Reporter: Federico Bond | Owner: nobody
Type: Cleanup/optimization | Status: assigned
Component: File uploads/storage | Version: 2.1
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
------------------------------------------------+--------------------------
Now that the minimum supported version is Python 3.5, we can replace
os.listdir with os.scandir in `FileSystemStorage.listdir` for improved
performance.

--
Ticket URL: <https://code.djangoproject.com/ticket/29689>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,
Aug 19, 2018, 7:40:52 PM8/19/18
to django-...@googlegroups.com
#29689: Replace os.listdir with os.scandir in FileSystemStorage.listdir
-------------------------------------+-------------------------------------
Reporter: Federico Bond | Owner: Federico
Type: | Bond
Cleanup/optimization | Status: assigned
Component: File | Version: 2.1
uploads/storage |
Severity: Normal | Resolution:

Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Federico Bond):

* owner: nobody => Federico Bond


--
Ticket URL: <https://code.djangoproject.com/ticket/29689#comment:1>

Django

unread,
Aug 19, 2018, 7:44:51 PM8/19/18
to django-...@googlegroups.com
#29689: Replace os.listdir with os.scandir in FileSystemStorage.listdir
-------------------------------------+-------------------------------------
Reporter: Federico Bond | Owner: Federico
Type: | Bond
Cleanup/optimization | Status: assigned

Component: File | Version: 2.1
uploads/storage |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Federico Bond):

* has_patch: 0 => 1


--
Ticket URL: <https://code.djangoproject.com/ticket/29689#comment:2>

Django

unread,
Aug 20, 2018, 5:17:41 AM8/20/18
to django-...@googlegroups.com
#29689: Replace os.listdir with os.scandir in FileSystemStorage.listdir
-------------------------------------+-------------------------------------
Reporter: Federico Bond | Owner: Federico
Type: | Bond
Cleanup/optimization | Status: assigned

Component: File | Version: 2.1
uploads/storage |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Claude Paroz):

* needs_better_patch: 0 => 1
* stage: Unreviewed => Accepted


--
Ticket URL: <https://code.djangoproject.com/ticket/29689#comment:3>

Django

unread,
Aug 20, 2018, 3:58:31 PM8/20/18
to django-...@googlegroups.com
#29689: Improve performance of FileSystemStorage.listdir() and FilePathField with
os.scandir()

-------------------------------------+-------------------------------------
Reporter: Federico Bond | Owner: Federico
Type: | Bond
Cleanup/optimization | Status: assigned

Component: File | Version: 2.1
uploads/storage |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Tim Graham):

* needs_better_patch: 1 => 0
* stage: Accepted => Ready for checkin


Old description:

> Now that the minimum supported version is Python 3.5, we can replace
> os.listdir with os.scandir in `FileSystemStorage.listdir` for improved
> performance.

New description:

Now that the minimum supported version is Python 3.5, we can replace

`os.listdir()` with `os.scandir()` in `FileSystemStorage.listdir()` and
`FilePathField.__init__()` to eliminate extra syscalls within the loops
for improved performance.

--

--
Ticket URL: <https://code.djangoproject.com/ticket/29689#comment:4>

Django

unread,
Aug 20, 2018, 6:40:33 PM8/20/18
to django-...@googlegroups.com
#29689: Improve performance of FileSystemStorage.listdir() and FilePathField with
os.scandir()
-------------------------------------+-------------------------------------
Reporter: Federico Bond | Owner: Federico
Type: | Bond
Cleanup/optimization | Status: closed

Component: File | Version: 2.1
uploads/storage |
Severity: Normal | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Tim Graham <timograham@…>):

* status: assigned => closed
* resolution: => fixed


Comment:

In [changeset:"a0ca4b5694f43c63ea13ba6908eff2bd53ee7ebb" a0ca4b56]:
{{{
#!CommitTicketReference repository=""
revision="a0ca4b5694f43c63ea13ba6908eff2bd53ee7ebb"
Fixed #29689 -- Improved performance of FileSystemStorage.listdir() and
FilePathField with os.scandir().
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/29689#comment:5>

Django

unread,
Aug 29, 2018, 6:20:01 AM8/29/18
to django-...@googlegroups.com
#29689: Improve performance of FileSystemStorage.listdir() and FilePathField with
os.scandir()
-------------------------------------+-------------------------------------
Reporter: Federico Bond | Owner: Federico
Type: | Bond
Cleanup/optimization | Status: closed
Component: File | Version: 2.1
uploads/storage |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Tim Graham <timograham@…>):

In [changeset:"54b331451cb22ee354beadf31ee42cbd714877f0" 54b33145]:
{{{
#!CommitTicketReference repository=""
revision="54b331451cb22ee354beadf31ee42cbd714877f0"
Refs #29689 -- Moved FilePathField choices sorting outside the loop.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/29689#comment:6>

Reply all
Reply to author
Forward
0 new messages