[Django] #35397: override_settings for STORAGES loses OPTIONS

[Django]

#35397: override_settings for STORAGES loses OPTIONS
-------------------------------------+-------------------------------------
Reporter: Will | Owner: nobody
Giddens |
Type: | Status: new
Uncategorized |
Component: File | Version: 4.2
uploads/storage | Keywords: override_settings,
Severity: Normal | storages
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
If you use override_settings to change STORAGES, any OPTIONS set on the
default storage are ignored.

The `Settings.__init__()` checks if STORAGES is overwritten and sets the
DEFAULT_FILE_STORAGE to the new value. `StorageHandler.backends` checks
to see if DEFAULT_FILE_STORAGE is overridden. It proceeds to overwrite the
`STORAGES["default"]` based on the DEFAULT_FILE_STORAGE setting and
overrides the explicitly set OPTIONS.

It looks like `STATICFILES_STORAGE` is handled the same way.
--
Ticket URL: <https://code.djangoproject.com/ticket/35397>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35397: override_settings for STORAGES loses OPTIONS
-------------------------------------+-------------------------------------

Reporter: Will Giddens | Owner: nobody
Type: Uncategorized | Status: new

Component: File | Version: 4.2
uploads/storage |

Severity: Normal | Resolution:
Keywords: override_settings, | Triage Stage:
storages | Unreviewed
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Will Giddens):

It looks like this is a regression from
https://github.com/django/django/commit/6b965c600054f970bdf94017ecf2e0e6e0a4326b.

To me, it seems to make more sense to resolve all config elements in
Settings instead of StorageHandler to avoid these edge cases.
--
Ticket URL: <https://code.djangoproject.com/ticket/35397#comment:1>

[Django]

#35397: override_settings for STORAGES loses OPTIONS
-------------------------------------+-------------------------------------
Reporter: Will Giddens | Owner: nobody

Type: Uncategorized | Status: closed

Component: File | Version: 4.2
uploads/storage |

Severity: Normal | Resolution: needsinfo

Keywords: override_settings, | Triage Stage:
storages | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* resolution: => needsinfo
* status: new => closed

Comment:

Can you share how you are using override_settings to change `STORAGES` and
the original `STORAGES` setting? That `OPTIONS` set on the default storage
are ignored when using `override_settings` to me sounds like expected
behaviour
--
Ticket URL: <https://code.djangoproject.com/ticket/35397#comment:2>

[Django]

#35397: override_settings for STORAGES loses OPTIONS
-------------------------------------+-------------------------------------
Reporter: Will Giddens | Owner: nobody
Type: Uncategorized | Status: closed
Component: File | Version: 4.2
uploads/storage |
Severity: Normal | Resolution: needsinfo
Keywords: override_settings, | Triage Stage:
storages | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Will Giddens):

Sure, here's a simplified snippet.

I'm using django-storages to configure S3 storage, but I confirmed
Django's built-in storage engines also ignore the `OPTIONS`. I can
provide a test case with the built-in storage engines if it helps.

settings.py:
{{{
STORAGES = {
"default": {
"BACKEND": "storages.backends.s3.S3Storage",
"OPTIONS": {
"bucket_name": "prod-bucket",
"file_overwrite": False,
},
},
"staticfiles": {
"BACKEND":
"django.contrib.staticfiles.storage.StaticFilesStorage",
},
}
}}}

The test:
{{{
@override_settings(
STORAGES={
"default": {
"BACKEND": "storages.backends.s3.S3Storage",
"OPTIONS": {
"bucket_name": "test-bucket",
"file_overwrite": False,
},
},
},
)
class ExampleStorageTestCase(TestCase):
def test_override_worked(self):
# This fails with AssertionError: None != 'test-bucket'
self.assertEqual(default_storage.bucket_name, "test-bucket")
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35397#comment:3>

[Django]

#35397: override_settings for STORAGES loses OPTIONS
-------------------------------------+-------------------------------------
Reporter: Will Giddens | Owner: nobody

Type: Uncategorized | Status: new

Component: File | Version: 4.2
uploads/storage |
Severity: Normal | Resolution:

Keywords: override_settings, | Triage Stage:
storages | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Will Giddens):

* resolution: needsinfo =>
* status: closed => new

--
Ticket URL: <https://code.djangoproject.com/ticket/35397#comment:4>

[Django]

#35397: override_settings for STORAGES loses OPTIONS
-------------------------------------+-------------------------------------
Reporter: Will Giddens | Owner: nobody

Type: Uncategorized | Status: closed

Component: File | Version: 4.2
uploads/storage |

Severity: Normal | Resolution: invalid

Keywords: override_settings, | Triage Stage:
storages | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* resolution: => invalid

* status: new => closed

Comment:

> I can provide a test case with the built-in storage engines if it helps.

Please do as this is working for me on main:

{{{#!diff
diff --git a/tests/file_storage/tests.py b/tests/file_storage/tests.py
index 420314573d..4e54968c11 100644
--- a/tests/file_storage/tests.py
+++ b/tests/file_storage/tests.py
@@ -567,6 +567,25 @@ class FileStorageTests(SimpleTestCase):
settings["FILE_UPLOAD_DIRECTORY_PERMISSIONS"],
)

+ @override_settings(
+ STORAGES={
+ DEFAULT_STORAGE_ALIAS: {
+ "BACKEND": "django.core.files.storage.InMemoryStorage",
+ "OPTIONS": {
+ "location": "explicit_location",
+ "base_url": "explicit_base_url/",
+ "file_permissions_mode": 0o666,
+ "directory_permissions_mode": 0o666,
+ },
+ },
+ }
+ )
+ def test_override_settings(self):
+ self.assertEqual(default_storage._location, "explicit_location")
+ self.assertEqual(default_storage._base_url, "explicit_base_url/")
+ self.assertEqual(default_storage._file_permissions_mode, 0o666)
+ self.assertEqual(default_storage._directory_permissions_mode,
0o666)
+
def test_file_methods_pathlib_path(self):
p = Path("test.file")
self.assertFalse(self.storage.exists(p))
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35397#comment:5>

[Django]

#35397: override_settings for STORAGES loses OPTIONS
-------------------------------------+-------------------------------------
Reporter: Will Giddens | Owner: nobody

Type: Uncategorized | Status: new

Component: File | Version: 4.2
uploads/storage |
Severity: Normal | Resolution:

Keywords: override_settings, | Triage Stage:
storages | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Will Giddens):

* resolution: invalid =>

* status: closed => new

Comment:

It looks like your test is good- it works on main for me as well.
Whenever the `DEFAULT_FILE_STORAGE` setting was removed this bug goes
away.

Since this only affects stable/4.2.x and stable/5.0.x should we pursue a
fix or let it be?
--
Ticket URL: <https://code.djangoproject.com/ticket/35397#comment:6>

[Django]

#35397: override_settings for STORAGES loses OPTIONS
-------------------------------------+-------------------------------------
Reporter: Will Giddens | Owner: nobody

Type: Uncategorized | Status: closed

Component: File | Version: 4.2
uploads/storage |

Severity: Normal | Resolution: invalid

Keywords: override_settings, | Triage Stage:
storages | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Mariusz Felisiak):

* resolution: => invalid
* status: new => closed

Comment:

Django 4.2 is in extended support so it no longer receives bugfixes
(except security patches), and this issue doesn't qualify for a backport
to the Django 5.0. Closing as "invalid".
--
Ticket URL: <https://code.djangoproject.com/ticket/35397#comment:7>