[Django] #23724: Overwrite mode in django
Django
-------------------------------------+--------------------
Reporter: prathik | Owner: nobody
Type: New feature | Status: new
Component: contrib.staticfiles | Version: 1.7
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+--------------------
We had an issue in the production server where the files in STATIC_ROOT
were not being replaced by the ones in the individual apps. As a solution
for this we ran collectstatic with the -c option. This worked but what
happened was the the entire static root was erased, this static root was
configured to point to the static files folder in the container app which
resulted in some of the files being permanently deleted because
collectstatic first clears the folder then collects the staticfiles. While
it is true that it was incorrect to set it up this way, it would be ideal
if it wouldn't have deleted the files.
Another problem was that due to some reason the source files had a
timestamp lesser than the destination files, this could have been due to a
caching mechanism but this resulted in the source files not being
collected.
To solve for the above issues we need a overwrite method which would
overwrite the source files no matter what the timestamp and it would not
perform the copy if the source and the destination are the same.
--
Ticket URL: <https://code.djangoproject.com/ticket/23724>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: prathik | Owner: nobody
Type: New feature | Status: new
Component: contrib.staticfiles | Version: 1.7
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by prathik):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Old description:
> We had an issue in the production server where the files in STATIC_ROOT
> were not being replaced by the ones in the individual apps. As a solution
> for this we ran collectstatic with the -c option. This worked but what
> happened was the the entire static root was erased, this static root was
> configured to point to the static files folder in the container app which
> resulted in some of the files being permanently deleted because
> collectstatic first clears the folder then collects the staticfiles.
> While it is true that it was incorrect to set it up this way, it would be
> ideal if it wouldn't have deleted the files.
>
> Another problem was that due to some reason the source files had a
> timestamp lesser than the destination files, this could have been due to
> a caching mechanism but this resulted in the source files not being
> collected.
>
> To solve for the above issues we need a overwrite method which would
> overwrite the source files no matter what the timestamp and it would not
> perform the copy if the source and the destination are the same.
New description:
We had an issue in the production server where the files in STATIC_ROOT
were not being replaced by the ones in the individual apps. As a solution
for this we ran collectstatic with the -c option. This worked but what
happened was the the entire static root was erased, this static root was
configured to point to the static files folder in the container app which
resulted in some of the files being permanently deleted because
collectstatic first clears the folder then collects the staticfiles. While
it is true that it was incorrect to set it up this way, it would be ideal
if it wouldn't have deleted the files.
Another problem was that due to some reason the source files had a
timestamp lesser than the destination files, this could have been due to a
caching mechanism but this resulted in the source files not being
collected.
To solve for the above issues we need a overwrite method which would
overwrite the source files no matter what the timestamp and it would not
perform the copy if the source and the destination are the same.
Pull request: https://github.com/django/django/pull/3434
--
Comment:
Pull request: https://github.com/django/django/pull/3434
--
Ticket URL: <https://code.djangoproject.com/ticket/23724#comment:1>
Django
Reporter: prathik | Owner: nobody
Type: New feature | Status: new
Component: contrib.staticfiles | Version: 1.7
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
> We had an issue in the production server where the files in STATIC_ROOT
> were not being replaced by the ones in the individual apps. As a solution
> for this we ran collectstatic with the -c option. This worked but what
> happened was the the entire static root was erased, this static root was
> configured to point to the static files folder in the container app which
> resulted in some of the files being permanently deleted because
> collectstatic first clears the folder then collects the staticfiles.
> While it is true that it was incorrect to set it up this way, it would be
> ideal if it wouldn't have deleted the files.
>
> Another problem was that due to some reason the source files had a
> timestamp lesser than the destination files, this could have been due to
> a caching mechanism but this resulted in the source files not being
> collected.
>
> To solve for the above issues we need a overwrite method which would
> overwrite the source files no matter what the timestamp and it would not
> perform the copy if the source and the destination are the same.
>
> Pull request: https://github.com/django/django/pull/3434
New description:
We had an issue in the production server where the files in STATIC_ROOT
were not being replaced by the ones in the individual apps. As a solution
for this we ran collectstatic with the -c option. This worked but what
happened was the the entire static root was erased, this static root was
configured to point to the static files folder in the container app which
resulted in some of the files being permanently deleted because
collectstatic first clears the folder then collects the staticfiles. While
it is true that it was incorrect to set it up this way, it would be ideal
if it wouldn't have deleted the files.
Another problem was that due to some reason the source files had a
timestamp lesser than the destination files, this could have been due to a
caching mechanism but this resulted in the source files not being
collected.
To solve for the above issues we need a overwrite method which would
overwrite the source files no matter what the timestamp and it would not
perform the copy if the source and the destination are the same.
Pull request: https://github.com/django/django/pull/3435
--
--
Ticket URL: <https://code.djangoproject.com/ticket/23724#comment:2>
Django
Reporter: prathik | Owner: nobody
Type: New feature | Status: new
Component: contrib.staticfiles | Version: 1.7
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by timgraham):
Thank-you for the suggestion, but you admitted that you setup things
incorrectly and I don't think we should add features to cater to that. Is
there a reason you cannot fix your incorrect setup so that `--clear` won't
have unintended side-effects?
--
Ticket URL: <https://code.djangoproject.com/ticket/23724#comment:3>
Django
Reporter: prathik | Owner: nobody
Type: New feature | Status: new
Component: contrib.staticfiles | Version: 1.7
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by prathik):
This feature can come in handy for situations where other apps are also
writing into the STATIC_ROOT folder or in scenarios where the files in
STATIC_ROOT have changed. This would be a safer way to do things as we are
not deleting anything,
Since misconfigurations do happen, especially when dev and ops are
different teams, I think Django should be able to stop from unintended
effects from happening, for-example the deletion of STATIC_ROOT folder
here caused the static files of the app to be deleted (since the location
of static folder of the app and STATIC_ROOT are the same) and it couldn't
be retrieved until another release is done. I think it would be nice if
Django alerts the users in these scenarios or we could have an overwrite
method which doesn't delete anything.
It's not necessary that we have an overwrite method but it would be ideal
if Django would prevent such scenarios from happening. Please let me know
what your thoughts/alternative suggestions on this.
--
Ticket URL: <https://code.djangoproject.com/ticket/23724#comment:4>
Django
Reporter: prathik | Owner: nobody
Type: New feature | Status: new
Component: contrib.staticfiles | Version: 1.7
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by timgraham):
I think the documentation contains
[https://docs.djangoproject.com/en/dev/ref/settings/#static-root a clear
warning] about the mistake you made. We also recently
[https://github.com/django/django/commit/faacc54ac8601e8bdbfdc8651d68267d570a9c4b
clarified the warning] for `collectstatic --clear` and
[https://github.com/django/django/commit/4befb3015c26810a68cfcf57e0cd8b062f56f1c5
fixed an issue] where the destination directory wouldn't be displayed. Is
this enough? (I wonder if you are using an older version of Django that
doesn't include these improvements?)
--
Ticket URL: <https://code.djangoproject.com/ticket/23724#comment:5>
Django
Reporter: prathik | Owner: nobody
Type: New feature | Status: new
Component: contrib.staticfiles | Version: 1.7
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by prathik):
Yup it's definitely a mistake but we could be mitigating the damage if
Django prevented `--clear` from happening in this case, at times
developers don't handle the deployment or there could be multiple
developers, someone might have not read that warning, thus I think Django
should at least warn that they are within the app's folder and that what
is deleted can't be restored by collectstatic.
Is there a way in which we can prevent collectstatic from deleting the
folder if it belongs to an app? That is if the STATIC_ROOT points to a
folder within an app then it shouldn't be deleted or atleast user should
be warned about the cost.
Also I think it would be better if --clear only deleted files that belongs
to Django, this would allow multiple apps (non-django) to write into the
STATIC_ROOT too. These were the reasons that led to the development of the
`--overwrite` mode, which doesn't touch any other file apart from that
which belongs to django.
--
Ticket URL: <https://code.djangoproject.com/ticket/23724#comment:6>
Django
Reporter: prathik | Owner: nobody
Type: New feature | Status: closed
Component: contrib.staticfiles | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => wontfix
Comment:
I don't think Django can track which static files it manages. Consider an
app that has some of its static files removed from its app directory.
Django would then have to consider the copies of those files that remain
in `STATIC_ROOT` as "unmanaged" and they would never be deleted.
I think the warnings I linked to in my previous comment should be enough
for people to see that files in their app's static directory will be
deleted if they proceed.
I'm going to close this ticket as "won't fix" as the idea of an "overwrite
option" goes against one of the design decisions of static files. If you
wish to argue for a design decision, could you please start a thread on
the DevelopersMailingList? Thanks!
--
Ticket URL: <https://code.djangoproject.com/ticket/23724#comment:7>
Django
Reporter: prathik | Owner: nobody
Type: New feature | Status: closed
Component: contrib.staticfiles | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by prathik):
Tim, out of curiosity, what would happen if a file has been deleted from
the static files of an app and the user runs `collectstatic` with no
options? Would it be removed from `STATIC_ROOT`?
--
Ticket URL: <https://code.djangoproject.com/ticket/23724#comment:8>
Django
Reporter: prathik | Owner: nobody
Type: New feature | Status: closed
Component: contrib.staticfiles | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by timgraham):
I wouldn't expect it would; you'd have to confirm though.
--
Ticket URL: <https://code.djangoproject.com/ticket/23724#comment:9>
Django
-------------------------------------+-------------------------------------
Reporter: prathik | Owner: nobody
Type: New feature | Status: closed
Component: contrib.staticfiles | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
--
Ticket URL: <https://code.djangoproject.com/ticket/23724#comment:10>
Django
-------------------------------------+-------------------------------------
Reporter: prathik | Owner: nobody
Type: New feature | Status: closed
Component: contrib.staticfiles | Version: 1.7
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
Has patch: 1 | Unreviewed
Needs tests: 0 | Needs documentation: 0
Easy pickings: 0 | Patch needs improvement: 0
| UI/UX: 0
-------------------------------------+-------------------------------------
--
Ticket URL: <https://code.djangoproject.com/ticket/23724#comment:11>