[Django] #28401: django doesn't work with FIPS kernels
Django
-----------------------------------------+---------------------------
Reporter: Andrew | Owner: nobody
Type: Bug | Status: new
Component: Uncategorized | Version: master
Severity: Normal | Keywords: FIPS, md5
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-----------------------------------------+---------------------------
Specifically, every use of hashlib.md5() is an issue for FIPS kernels
which lack openssl support for md5. However, at least on RHEL and Centos,
hashlib.new() supports the usedforsecurity=False flag that allows you to
bypass the FIPS prohibition on md5. Many of the cases in which the md5
function is used qualify as non-security uses since it is used to mainly
truncate values. Of course the md5 auth backends would constitute a
security usages and should not include the usedforsecurity=False and thus
should fail on FIPS systems.
Therefore i propose that we add the usedforsecurity=False flag where
warranted and handling in case some versions of python do not support this
flag. (I tested python 2.7 on OSX and it errors when the usedforsecurity
flag is set). If everyone this this is a good plan I will go ahead and
make a PR.
list of all useages of hashlib.md5
[https://github.com/django/django/search?q=hashlib.md5&type=Code&utf8=%E2%9C%93]
django-developers thread
[https://groups.google.com/forum/#!msg/django-
developers/dlUIPzQgnpM/Mtl7CQbPAQAJ]
django-users thread
[https://groups.google.com/forum/#!topic/django-users/THJdhaKo-ng]
Example stack trace:
{{{
Operations to perform:
Apply all migrations: admin, auth, contenttypes, dashboard,
kombu_transport_django, sessions
Running migrations:
Applying contenttypes.0001_initial...Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/lib64/python2.7/site-
packages/django/core/management/__init__.py", line 367, in
execute_from_command_line
utility.execute()
File "/usr/lib64/python2.7/site-
packages/django/core/management/__init__.py", line 359, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib64/python2.7/site-
packages/django/core/management/base.py", line 294, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/lib64/python2.7/site-
packages/django/core/management/base.py", line 345, in execute
output = self.handle(*args, **options)
File "/usr/lib64/python2.7/site-
packages/django/core/management/commands/migrate.py", line 204, in handle
fake_initial=fake_initial,
File "/usr/lib64/python2.7/site-
packages/django/db/migrations/executor.py", line 115, in migrate
state = self._migrate_all_forwards(state, plan, full_plan, fake=fake,
fake_initial=fake_initial)
File "/usr/lib64/python2.7/site-
packages/django/db/migrations/executor.py", line 145, in
_migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake,
fake_initial=fake_initial)
File "/usr/lib64/python2.7/site-
packages/django/db/migrations/executor.py", line 244, in apply_migration
state = migration.apply(state, schema_editor)
File "/usr/lib64/python2.7/site-
packages/django/db/migrations/migration.py", line 129, in apply
operation.database_forwards(self.app_label, schema_editor, old_state,
project_state)
File "/usr/lib64/python2.7/site-
packages/django/db/migrations/operations/models.py", line 532, in
database_forwards
getattr(new_model._meta, self.option_name, set()),
File "/usr/lib64/python2.7/site-
packages/django/db/backends/base/schema.py", line 333, in
alter_unique_together
self.execute(self._create_unique_sql(model, columns))
File "/usr/lib64/python2.7/site-
packages/django/db/backends/base/schema.py", line 913, in
_create_unique_sql
"name": self.quote_name(self._create_index_name(model, columns,
suffix="_uniq")),
File "/usr/lib64/python2.7/site-
packages/django/db/backends/base/schema.py", line 819, in
_create_index_name
index_unique_name = '_%s' % self._digest(table_name, *column_names)
File "/usr/lib64/python2.7/site-
packages/django/db/backends/base/schema.py", line 123, in _digest
h = hashlib.md5()
ValueError: error:060800A3:digital envelope
routines:EVP_DigestInit_ex:disabled for fips
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28401>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
Reporter: Andrew DiPrinzio | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Uncategorized | Version: master
Severity: Normal | Resolution:
Keywords: FIPS, md5 | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Markus Holtermann):
* type: Bug => Cleanup/optimization
Comment:
Thanks for the report.
While I like the idea, {{useforsecurity}} doesn't seem to be part of the
official Python package but rather something that Red Hat added and thus
is only available on RHEL, Centos, etc.
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:1>
Django
--------------------------------------+------------------------------------
Reporter: Andrew DiPrinzio | Owner: nobody
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Keywords: FIPS, md5 | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham):
* stage: Unreviewed => Accepted
* component: Uncategorized => Core (Other)
Comment:
If the [http://bugs.python.org/issue9216 Python patch] is merged, we could
use the new parameter.
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:2>
Django
Reporter: Andrew DiPrinzio | Owner: nobody
Cleanup/optimization |
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Aymeric Augustin):
* stage: Accepted => Someday/Maybe
Comment:
Considering that the patch for Python isn't merged, it could take a bit of
time until there's a Python release including it.
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:3>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Andrew DiPrinzio):
Replying to [comment:1 Markus Holtermann]:
> Thanks for the report.
>
> While I like the idea, `useforsecurity` doesn't seem to be part of the
official Python package but rather something that Red Hat added and thus
is only available on RHEL, Centos, etc.
Thanks all for your insights! I continued to do research after opening
this ticket. You are correct that this flag is only in some fedora
distributions. Also i think it is fair to say that this not really a bug
in django but rather a deficiency in python's hashlib. I will try and move
that ticket forward on the python side.
[https://bugs.python.org/issue9216] However since some some distributions
have this flag would it be acceptable to add support for this flag?
something like this?
{{{
#!div style="font-size: 80%"
{{{#!python
try:
hashlib.md5("blah")
except ValueError e:
# the fedora fix throws value errors for issues with FIPS
hashlib.md5("blah", usedforsecurity=False)
}}}
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:4>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham):
If the patch is merged to Python, I don't mind adding a wrapper function
for that pattern until Django drops support for Python versions without
the new parameter, however, I don't want to support the parameter until
the Python issue resolved.
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:5>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by 01000101):
This ticket was referenced in my PR for a similar fix/feature -
https://github.com/django/django/pull/10605
Was there ever any consensus on the best way to handle these sort of
FIPS-related fixes going forward?
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:6>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Tim Graham):
I'm not aware of any discussion of the issue outside of the ticket. You
can write to the DevelopersMailingList if you want to get a second opinion
on your proposal. As a general solution (replacing all `hashlib.md5()`
calls in Django with a fallback to sha256), I don't like the idea because
of inconsistency possibilities in mixed (fips and non-fips) system
environments the performance differences (perhaps insignificant). Of
course, there's also the issue that md5 and sha256 sums are different
lengths. Perhaps truncating the latter to be the length of md5 would help
eliminate possible bugs due to differences.
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:7>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Joshua Cornutt):
Replying to [comment:7 Tim Graham]:
> I'm not aware of any discussion of the issue outside of the ticket. You
can write to the DevelopersMailingList if you want to get a second opinion
on your proposal. As a general solution (replacing all `hashlib.md5()`
calls in Django with a fallback to sha256), I don't like the idea because
of inconsistency possibilities in mixed (fips and non-fips) system
environments the performance differences (perhaps insignificant). Of
course, there's also the issue that md5 and sha256 sums are different
lengths. Perhaps truncating the latter to be the length of md5 would help
eliminate possible bugs due to differences.
I agree that a full, sweeping change from MD5 to SHA-256 is a Bad Idea
(tm), but some code segments can be pretty easily moved over with
minimal/no impact. I'm a fan of keeping things like this user-configurable
and leaving the default as whatever the code uses today (MD5). That way it
spreads some of the responsibility out. Since this is my first commit to
the Django project, do you (or anyone else) have any tips/hints as to how
to proceed with the change I proposed? How does it get a stamp of approval
and merged?
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:8>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Carlton Gibson):
> ...do you (or anyone else) have any tips/hints as to how to proceed with
the change I proposed? How does it get a stamp of approval and merged?
Tim’s suggestion to mail the DevelopersMailingList is the best approach.
There’s lots of people there that can input, and it’s likely a discussion
can lead to a way forward (especially if it’s pressing).
Can I ask you to make sure you include enough backstory and context to
bring people (including me 😀) up to speed on the issue? Otherwise you
limit the response to those with lots of time (which isn’t many).
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:9>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by daviddavis):
FWIW, the `usedforsecurity` flag will be available in Python 3.9+:
https://github.com/python/cpython/pull/16044/
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:10>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
python3.9 | Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* keywords: FIPS, md5 => FIPS, md5, python3.9
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:11>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Keywords: FIPS, md5, | Triage Stage:
python3.9 | Someday/Maybe
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by CrunchyPie):
So since 'usedforsecurity' flag will be available in python 3.9 we might
as well go forward and add up support.
FIPS compliance will be interesting for government organizations or for
everything else that has to deal with sensitive data.
It would also be a good advancement for data security, auditing, and
compliance in general.
Every developer that uses RHEL, Fedora, or CentOS would profit from this
change as well.
We should not swap the algorithms since this would remove backwards
compatibility.
We also don't have to since Python's Hashlibrary allows to pass the flag
with the new method as the second argument.
( By the way, this is my first comment and contribution to Django so let
me know if I do something wrong)
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:12>
Django
-------------------------------------+-------------------------------------
Type: | sarwar
Cleanup/optimization | Status: assigned
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Keywords: FIPS, md5, | Triage Stage:
python3.9 | Someday/Maybe
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 1
-------------------------------------+-------------------------------------
Changes (by shazad sarwar):
* status: new => assigned
* cc: shazad sarwar (added)
* needs_better_patch: 0 => 1
* needs_tests: 0 => 1
* easy: 0 => 1
* owner: nobody => shazad sarwar
* needs_docs: 0 => 1
* has_patch: 0 => 1
* ui_ux: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:13>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: shazad
Type: | sarwar
Component: Core (Other) | Version: master
Keywords: FIPS, md5, | Triage Stage:
python3.9 | Someday/Maybe
Has patch: 1 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 1
-------------------------------------+-------------------------------------
Changes (by shazad sarwar):
* status: assigned => closed
* resolution: => fixed
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:14>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: shazad
Type: | sarwar
Cleanup/optimization | Status: closed
Component: Core (Other) | Version: master
Severity: Normal | Resolution: fixed
Keywords: FIPS, md5, | Triage Stage:
python3.9 | Someday/Maybe
Has patch: 1 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 1
-------------------------------------+-------------------------------------
Changes (by shazad sarwar):
* cc: shazad sarwar (removed)
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:15>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: shazad
Type: | sarwar
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
python3.9 | Someday/Maybe
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* status: closed => new
* needs_better_patch: 1 => 0
* needs_tests: 1 => 0
* easy: 1 => 0
* needs_docs: 1 => 0
* has_patch: 1 => 0
* ui_ux: 1 => 0
* resolution: fixed =>
Comment:
Shazad, this ticket is not fixed.
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:16>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: shazad
Type: | sarwar
Cleanup/optimization | Status: new
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Keywords: FIPS, md5, | Triage Stage:
python3.9 | Someday/Maybe
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Lars Erhardt):
Ok, so what are the next steps? According to the docs, someone should mail
the developer mail list?
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:17>
Django
-------------------------------------+-------------------------------------
Type: | Status: assigned
Cleanup/optimization |
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: FIPS, md5, | Triage Stage:
python3.9 | Someday/Maybe
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Ade Lee):
* owner: nobody => Ade Lee
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:18>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: Ade Lee
Type: | Status: assigned
Cleanup/optimization |
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: FIPS, md5, | Triage Stage:
python3.9 | Someday/Maybe
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Comment (by Ade Lee):
I have added the following pull request to resolve this issue. Thanks!
This will allow django to run in fips enabled kernels. Eventually, when
support for this
attribute is available everywhere, we can remove the wrapper function and
just use what
is in hashlib.
https://github.com/django/django/pull/14763
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:19>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: Ade Lee
Type: | Status: assigned
Cleanup/optimization |
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Keywords: FIPS, md5, | Triage Stage:
python3.9 | Someday/Maybe
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Ade Lee):
* has_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:20>
Django
Reporter: Andrew DiPrinzio | Owner: Ade Lee
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* stage: Someday/Maybe => Accepted
Comment:
The `usedforsecurity` argument is available in Python 3.9+, see
https://github.com/python/cpython/commit/7cad53e6b084435a220e6604010f1fa5778bd0b1.
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:21>
Django
--------------------------------------+------------------------------------
Reporter: Andrew DiPrinzio | Owner: Ade Lee
Type: Cleanup/optimization | Status: assigned
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:22>
Django
Reporter: Andrew DiPrinzio | Owner: Ade Lee
Cleanup/optimization |
Component: Core (Other) | Version: dev
Severity: Normal | Resolution:
python3.9 | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak):
* needs_better_patch: 1 => 0
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:23>
Django
-------------------------------------+-------------------------------------
Reporter: Andrew DiPrinzio | Owner: Ade Lee
Cleanup/optimization |
Component: Core (Other) | Version: dev
Keywords: FIPS, md5, | Triage Stage: Ready for
python3.9 | checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Mariusz Felisiak <felisiak.mariusz@…>):
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"d10c7bfe56f025ccc690721c9f13e7029b777b9c" d10c7bfe]:
{{{
#!CommitTicketReference repository=""
revision="d10c7bfe56f025ccc690721c9f13e7029b777b9c"
Fixed #28401 -- Allowed hashlib.md5() calls to work with FIPS kernels.
md5 is not an approved algorithm in FIPS mode, and trying to instantiate
a hashlib.md5() will fail when the system is running in FIPS mode.
md5 is allowed when in a non-security context. There is a plan to add a
keyword parameter (usedforsecurity) to hashlib.md5() to annotate whether
or not the instance is being used in a security context.
In the case where it is not, the instantiation of md5 will be allowed.
See https://bugs.python.org/issue9216 for more details.
Some downstream python versions already support this parameter. To
support these versions, a new encapsulation of md5() has been added.
This encapsulation will pass through the usedforsecurity parameter in
the case where the parameter is supported, and strip it if it is not.
Co-authored-by: Mariusz Felisiak <felisiak...@gmail.com>
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/28401#comment:24>