[Django] #28200: Do not touch hash-designated files which already exist at the destination storage

[Django]

#28200: Do not touch hash-designated files which already exist at the destination
storage
-------------------------------------+-------------------------------------
Reporter: Michal | Owner: nobody
Krupa |
Type: | Status: new
Cleanup/optimization |
Component: | Version: 1.11
contrib.staticfiles | Keywords: staticfiles,
Severity: Normal | storage, remote
Triage Stage: | Has patch: 1
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
It seems a little silly that, even though local file copies are used to
prevent needing to fetch files from remote destinations, files still get
replaced even when they already exist.

In example, a remote storage implementation like S3 gets queried for the
file, the file gets removed, and then replaced. With the boto3 library,
this means touching the file initially (HEAD), a second request to DELETE
the file, and yet a third to PUT the new file. Since filenames for hashed
files are computed based on the contents of the files, this seems like an
unnecessary 3 requests for every time static assets get processed.

Since a matching hash provides file integrity verification, I propose that
the logic for copying hash-designated files be skipped all together.

PR Available here - https://github.com/django/django/pull/8496

--
Ticket URL: <https://code.djangoproject.com/ticket/28200>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#28200: Do not touch hash-designated files which already exist at the destination
storage
-------------------------------------+-------------------------------------

Reporter: Michal Krupa | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: contrib.staticfiles | Version: 1.11
Severity: Normal | Resolution:
Keywords: staticfiles, | Triage Stage:
storage, remote | Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Tim Graham):

* needs_better_patch: 0 => 1

Comment:

Is this a duplicate of or related to #28055? There are failing tests with
the current PR.

--
Ticket URL: <https://code.djangoproject.com/ticket/28200#comment:1>

[Django]

#28200: Do not touch hash-designated files which already exist at the destination
storage
-------------------------------------+-------------------------------------

Reporter: Michal Krupa | Owner: nobody
Type: | Status: new
Cleanup/optimization |

Component: contrib.staticfiles | Version: 1.11
Severity: Normal | Resolution:
Keywords: staticfiles, | Triage Stage:
storage, remote | Unreviewed

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Michal Krupa):

Replying to [comment:1 Tim Graham]:

> Is this a duplicate of or related to #28055? There are failing tests
with the current PR.

Ah yes I see the failed test results, jobs were still running when this
ticket was opened. I will re-work the submission - thanks for the
response!

--
Ticket URL: <https://code.djangoproject.com/ticket/28200#comment:2>

[Django]

#28200: Do not touch hash-designated files which already exist at the destination
storage
-------------------------------------+-------------------------------------

Reporter: Michal Krupa | Owner: Michal
Type: | Krupa
Cleanup/optimization | Status: assigned

Component: contrib.staticfiles | Version: 1.11
Severity: Normal | Resolution:
Keywords: staticfiles, | Triage Stage:
storage, remote | Unreviewed

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Michal Krupa):

* owner: nobody => Michal Krupa
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/28200#comment:3>

[Django]

#28200: Do not touch hash-designated files which already exist at the destination
storage
-------------------------------------+-------------------------------------

Reporter: Michal Krupa | Owner: Michal
Type: | Krupa
Cleanup/optimization | Status: assigned
Component: contrib.staticfiles | Version: 1.11
Severity: Normal | Resolution:

Keywords: staticfiles, | Triage Stage: Accepted
storage, remote |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Tim Graham):

* stage: Unreviewed => Accepted

Comment:

Tentatively accepting, though it's not certain that this can be fixed.

--
Ticket URL: <https://code.djangoproject.com/ticket/28200#comment:4>