[Django] #28154: Infinite loop in collectstatic

14 views
Skip to first unread message

Django

unread,
Apr 29, 2017, 8:43:30 AM4/29/17
to django-...@googlegroups.com
#28154: Infinite loop in collectstatic
-----------------------------------------------+------------------------
Reporter: Matthew Somerville | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: 1.11
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------------+------------------------
If the static directory contains a broken symlink in place of a file that
collectstatic wishes to copy*, then the code gets stuck in an infinite
loop, because the os.open fails (O_CREAT | O_EXCL will fail if a symlink
exists, even if it's to a non-existent file), but then get_available_name
returns the same name (as it calls self.exists, which returns False for a
broken symlink).

I guess two possibilities are:
1. don't care about broken symlinks. Change the flags setting in
django/core/files/storage.py to not include O_EXCL if
`os.path.islink(full_path) and not os.path.exists(full_path)`.
2. do care, change os.path.exists in the `exists` function to use
os.path.lexists instead.

I'm not sure which would be preferable to prevent this infinite loop.

*I came across this issue because collectstatic had been run both inside
and outside a Vagrant box, so symlinks created inside the box did not
exist when collectstatic was then run outside.

--
Ticket URL: <https://code.djangoproject.com/ticket/28154>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

Django

unread,
May 9, 2017, 12:27:28 PM5/9/17
to django-...@googlegroups.com
#28154: Infinite loop in collectstatic with broken symlinks
-------------------------------------+------------------------------------

Reporter: Matthew Somerville | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: 1.11
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------
Changes (by Tim Graham):

* stage: Unreviewed => Accepted


--
Ticket URL: <https://code.djangoproject.com/ticket/28154#comment:1>

Django

unread,
Aug 15, 2017, 2:34:40 AM8/15/17
to django-...@googlegroups.com
#28154: Infinite loop in collectstatic with broken symlinks
-------------------------------------+------------------------------------

Reporter: Matthew Somerville | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: 1.11
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------

Comment (by KwonHan Bae):

I think its fixed in master branch.
should we backport to 11.x ?

--
Ticket URL: <https://code.djangoproject.com/ticket/28154#comment:2>

Django

unread,
Aug 15, 2017, 10:06:59 AM8/15/17
to django-...@googlegroups.com
#28154: Infinite loop in collectstatic with broken symlinks
-------------------------------------+------------------------------------

Reporter: Matthew Somerville | Owner: nobody
Type: Bug | Status: new
Component: contrib.staticfiles | Version: 1.11
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+------------------------------------

Comment (by Tim Graham):

Which commit fixed it? Most likely a fix wouldn't qualify for a backport
based on our [https://docs.djangoproject.com/en/dev/internals/release-
process/#supported-versions supported versions policy].

--
Ticket URL: <https://code.djangoproject.com/ticket/28154#comment:3>

Django

unread,
Nov 5, 2017, 10:00:49 AM11/5/17
to django-...@googlegroups.com
#28154: Infinite loop in collectstatic with broken symlinks
-------------------------------------+-------------------------------------
Reporter: Matthew Somerville | Owner:
| ChillarAnand
Type: Bug | Status: assigned
Component: contrib.staticfiles | Version: 1.11

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by ChillarAnand):

* status: new => assigned
* owner: nobody => ChillarAnand


--
Ticket URL: <https://code.djangoproject.com/ticket/28154#comment:4>

Django

unread,
Jun 2, 2021, 12:53:22 AM6/2/21
to django-...@googlegroups.com
#28154: Infinite loop in collectstatic with broken symlinks
-------------------------------------+-------------------------------------
Reporter: Matthew Somerville | Owner: Jacob
| Walls
Type: Bug | Status: assigned
Component: contrib.staticfiles | Version: dev

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Jacob Walls):

* owner: ChillarAnand => Jacob Walls
* version: 1.11 => dev


Comment:

Decided to reproduce this on main. Still can.

I'm preparing a tiny patch using the second possibility from the ticket
description: using `os.path.lexists()` in `FileSystemStorage.exists()`.

--
Ticket URL: <https://code.djangoproject.com/ticket/28154#comment:5>

Django

unread,
Jun 2, 2021, 1:17:51 AM6/2/21
to django-...@googlegroups.com
#28154: Infinite loop in collectstatic with broken symlinks
-------------------------------------+-------------------------------------
Reporter: Matthew Somerville | Owner: Jacob
| Walls
Type: Bug | Status: assigned
Component: File | Version: dev
uploads/storage |

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Jacob Walls):

* has_patch: 0 => 1
* component: contrib.staticfiles => File uploads/storage


Comment:

I reproduced this outside of contrib.staticfiles by just providing a
broken symlink to `FileSystemStorage.save()` and never exiting a `while`
loop. I'm hoping this is a welcome root cause to look at.

[https://github.com/django/django/pull/14474 PR]

--
Ticket URL: <https://code.djangoproject.com/ticket/28154#comment:6>

Django

unread,
Jun 2, 2021, 2:45:01 AM6/2/21
to django-...@googlegroups.com
#28154: Infinite loop in collectstatic with broken symlinks
-------------------------------------+-------------------------------------
Reporter: Matthew Somerville | Owner: Jacob
| Walls
Type: Bug | Status: assigned
Component: File | Version: dev
uploads/storage |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak):

* stage: Accepted => Ready for checkin


--
Ticket URL: <https://code.djangoproject.com/ticket/28154#comment:7>

Django

unread,
Jun 2, 2021, 6:45:20 AM6/2/21
to django-...@googlegroups.com
#28154: Infinite loop in collectstatic with broken symlinks
-------------------------------------+-------------------------------------
Reporter: Matthew Somerville | Owner: Jacob
| Walls
Type: Bug | Status: closed

Component: File | Version: dev
uploads/storage |
Severity: Normal | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Mariusz Felisiak <felisiak.mariusz@…>):

* status: assigned => closed
* resolution: => fixed


Comment:

In [changeset:"ec2727efef605437eb572d51ca9afbb3a60eda40" ec2727ef]:
{{{
#!CommitTicketReference repository=""
revision="ec2727efef605437eb572d51ca9afbb3a60eda40"
Fixed #28154 -- Prevented infinite loop in FileSystemStorage.save() when a
broken symlink with the same name exists.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/28154#comment:8>

Reply all
Reply to author
Forward
0 new messages