[Django] #36479: Failing test for black formatter missing install simulation

[Django]

#36479: Failing test for black formatter missing install simulation
-----------------------------------------+--------------------------
Reporter: Roelzkie | Owner: Roelzkie
Type: Bug | Status: assigned
Component: Uncategorized | Version: dev
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 1
UI/UX: 0 |
-----------------------------------------+--------------------------
The test on
`test.user_commands.tests.UtilsTests.test_run_formatters_handles_oserror_for_black_path`
attempts to assert a
[https://github.com/django/django/blob/main/tests/user_commands/tests.py#L569
`FileNotFoundError`] when simulating a missing black formatter
installation.

{{{
./runtests.py
user_commands.tests.UtilsTests.test_run_formatters_handles_oserror_for_black_path
}}}


However, it fails on the `Darwin` platform, at least on MacOS v14.7.2
(Sonoma) M1 machine. I tested on a Linux platform, and the test passed.

The root issue is due to the `subprocess.run` command, which yields
different `OSError` results on different OSes when the test reaches the
[https://github.com/django/django/blob/68a45d9a8078db642f0aca7ddab33af6df7ebeb3/django/core/management/utils.py#L175-L177
`subprocess.run(["nonexistent", "--fast", "--"])`]

For Darwin (at least on MacOS v14.7.2 - M1), it yields a
`NotADirectoryError` / `OSError(20, "Not a directory")` which fails the
test.
For Linux, it yields a `FileNotFoundError` / `OSError(2, "File not
found")` which passes the test.

Ideally this test should be able to handle it regardless of the machine
platform.

See failed test full stacktrace: https://dpaste.org/pY48h
--
Ticket URL: <https://code.djangoproject.com/ticket/36479>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36479: Failing test for black formatter missing install simulation

-------------------------------+--------------------------------------

Reporter: Roelzkie | Owner: Roelzkie
Type: Bug | Status: assigned
Component: Uncategorized | Version: dev

Severity: Normal | Resolution:

Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0

Needs tests: 1 | Patch needs improvement: 0

Easy pickings: 1 | UI/UX: 0

-------------------------------+--------------------------------------
Changes (by Roelzkie):

* needs_tests: 0 => 1


Old description:

> The test on
> `test.user_commands.tests.UtilsTests.test_run_formatters_handles_oserror_for_black_path`
> attempts to assert a
> [https://github.com/django/django/blob/main/tests/user_commands/tests.py#L569
> `FileNotFoundError`] when simulating a missing black formatter
> installation.
>
> {{{
> ./runtests.py
> user_commands.tests.UtilsTests.test_run_formatters_handles_oserror_for_black_path
> }}}
>

> However, it fails on the `Darwin` platform, at least on MacOS v14.7.2
> (Sonoma) M1 machine. I tested on a Linux platform, and the test passed.
>
> The root issue is due to the `subprocess.run` command, which yields
> different `OSError` results on different OSes when the test reaches the
> [https://github.com/django/django/blob/68a45d9a8078db642f0aca7ddab33af6df7ebeb3/django/core/management/utils.py#L175-L177
> `subprocess.run(["nonexistent", "--fast", "--"])`]
>
> For Darwin (at least on MacOS v14.7.2 - M1), it yields a
> `NotADirectoryError` / `OSError(20, "Not a directory")` which fails the
> test.
> For Linux, it yields a `FileNotFoundError` / `OSError(2, "File not
> found")` which passes the test.
>
> Ideally this test should be able to handle it regardless of the machine
> platform.
>
> See failed test full stacktrace: https://dpaste.org/pY48h

New description:

The test on
`test.user_commands.tests.UtilsTests.test_run_formatters_handles_oserror_for_black_path`
attempts to assert a
[https://github.com/django/django/blob/main/tests/user_commands/tests.py#L569
`FileNotFoundError`] when simulating a missing black formatter
installation.

{{{
./runtests.py
user_commands.tests.UtilsTests.test_run_formatters_handles_oserror_for_black_path
}}}


However, it fails on the `Darwin` platform, at least on MacOS v14.7.2
(Sonoma) M1 machine. I tested on a Linux platform, and the test passed.

The root issue is due to the `subprocess.run` command, which yields
different `OSError` results on different OSes when the test reaches the
[https://github.com/django/django/blob/68a45d9a8078db642f0aca7ddab33af6df7ebeb3/django/core/management/utils.py#L175-L177

`subprocess.run(["nonexistent", "--fast", "--"])]`.

For Darwin (at least on MacOS v14.7.2 - M1), it yields a
`NotADirectoryError` / `OSError(20, "Not a directory")` which fails the
test.
For Linux, it yields a `FileNotFoundError` / `OSError(2, "File not
found")` which passes the test.

Ideally this test should be able to handle it regardless of the machine
platform.

See failed test full stacktrace: https://dpaste.org/pY48h

Update: Please see https://github.com/django/django/pull/19591

--
--
Ticket URL: <https://code.djangoproject.com/ticket/36479#comment:1>

[Django]

#36479: Failing test for black formatter missing install simulation
-------------------------------+--------------------------------------
Reporter: Roelzkie | Owner: Roelzkie
Type: Bug | Status: assigned
Component: Uncategorized | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed

Has patch: 1 | Needs documentation: 0

Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------+--------------------------------------
Changes (by Roelzkie):

* has_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/36479#comment:2>

[Django]

#36479: Failing test for black formatter missing install simulation
-------------------------------+--------------------------------------
Reporter: Roelzkie | Owner: Roelzkie
Type: Bug | Status: assigned
Component: Uncategorized | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------+--------------------------------------

Changes (by Sarah Boyce):

* cc: Mike Edmunds (added)

Comment:

Mike would you be able to confirm this and that the PR fixes the issue?
(See from #36162 you might have a similar setup)
--
Ticket URL: <https://code.djangoproject.com/ticket/36479#comment:3>

[Django]

#36479: Failing test for black formatter missing install simulation
-------------------------------+--------------------------------------
Reporter: Roelzkie | Owner: Roelzkie
Type: Bug | Status: assigned
Component: Uncategorized | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------+--------------------------------------

Description changed by Roelzkie:

Old description:

> The test on
> `test.user_commands.tests.UtilsTests.test_run_formatters_handles_oserror_for_black_path`
> attempts to assert a
> [https://github.com/django/django/blob/main/tests/user_commands/tests.py#L569
> `FileNotFoundError`] when simulating a missing black formatter
> installation.
>
> {{{
> ./runtests.py
> user_commands.tests.UtilsTests.test_run_formatters_handles_oserror_for_black_path
> }}}
>

> However, it fails on the `Darwin` platform, at least on MacOS v14.7.2
> (Sonoma) M1 machine. I tested on a Linux platform, and the test passed.
>
> The root issue is due to the `subprocess.run` command, which yields
> different `OSError` results on different OSes when the test reaches the
> [https://github.com/django/django/blob/68a45d9a8078db642f0aca7ddab33af6df7ebeb3/django/core/management/utils.py#L175-L177

> `subprocess.run(["nonexistent", "--fast", "--"])]`.
>
> For Darwin (at least on MacOS v14.7.2 - M1), it yields a
> `NotADirectoryError` / `OSError(20, "Not a directory")` which fails the
> test.
> For Linux, it yields a `FileNotFoundError` / `OSError(2, "File not
> found")` which passes the test.
>
> Ideally this test should be able to handle it regardless of the machine
> platform.
>
> See failed test full stacktrace: https://dpaste.org/pY48h
>
> Update: Please see https://github.com/django/django/pull/19591

New description:

The test on
`test.user_commands.tests.UtilsTests.test_run_formatters_handles_oserror_for_black_path`
attempts to assert a
[https://github.com/django/django/blob/main/tests/user_commands/tests.py#L569
`FileNotFoundError`] when simulating a missing black formatter
installation.

{{{
./runtests.py
user_commands.tests.UtilsTests.test_run_formatters_handles_oserror_for_black_path
}}}

However, it fails on the `darwin` platform, at least on MacOS v14.7.2

(Sonoma) M1 machine. I tested on a Linux platform, and the test passed.

The root issue is due to the `subprocess.run` command, which yields

different `OSError` results on different OS's when the test reaches the

[https://github.com/django/django/blob/68a45d9a8078db642f0aca7ddab33af6df7ebeb3/django/core/management/utils.py#L175-L177
`subprocess.run(["nonexistent", "--fast", "--"])`].

For `darwin` (at least on MacOS v14.7.2 - M1), it yields a

`NotADirectoryError` / `OSError(20, "Not a directory")` which fails the
test.

For `linux`, it yields a `FileNotFoundError` / `OSError(2, "File not

found")` which passes the test.

Ideally, this test should be able to handle it regardless of the machine
platform.

See failed test full stacktrace: https://dpaste.org/pY48h (link will
expire, see result below instead).

{{{
FAIL: test_run_formatters_handles_oserror_for_black_path
(user_commands.tests.UtilsTests.test_run_formatters_handles_oserror_for_black_path)
[FileNotFoundError]
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/Users/user1/.pyenv/versions/3.13.2/lib/python3.13/unittest/case.py",
line 58, in testPartExecutor
yield
File
"/Users/user1/.pyenv/versions/3.13.2/lib/python3.13/unittest/case.py",
line 556, in subTest
yield
File "/Users/user1/projects/django/tests/user_commands/tests.py", line
584, in test_run_formatters_handles_oserror_for_black_path
self.assertIn(exception.__qualname__, parsed_error)
^^^^^^^
File
"/Users/user1/.pyenv/versions/3.13.2/lib/python3.13/unittest/case.py",
line 1174, in assertIn
self.fail(self._formatMessage(msg, standardMsg))
^^^^^^^^^^^
File
"/Users/user1/.pyenv/versions/3.13.2/lib/python3.13/unittest/case.py",
line 732, in fail
raise self.failureException(msg)
^^^^^^^^^^^^^^^
AssertionError: 'FileNotFoundError' not found in 'Formatters failed to
launch:Traceback (most recent call last):
File "/Users/user1/projects/django/django/core/management/utils.py",
line 175, in run_formatters
subprocess.run(
~~~~~~~~~~~~~~^
[black_path, "--fast", "--", *written_files],
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
capture_output=True,
^^^^^^^^^^^^^^^^^^^^
)
^
File "/Users/user1/.pyenv/versions/3.13.2/lib/python3.13/subprocess.py",
line 556, in run
with Popen(*popenargs, **kwargs) as process:
~~~~~^^^^^^^^^^^^^^^^^^^^^^
File "/Users/user1/.pyenv/versions/3.13.2/lib/python3.13/subprocess.py",
line 1038, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pass_fds, cwd, env,
^^^^^^^^^^^^^^^^^^^
...<5 lines>...
gid, gids, uid, umask,
^^^^^^^^^^^^^^^^^^^^^^
start_new_session, process_group)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/user1/.pyenv/versions/3.13.2/lib/python3.13/subprocess.py",
line 1974, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
NotADirectoryError: [Errno 20] Not a directory: \'nonexistent\''
​
----------------------------------------------------------------------

}}}


Update: Please see https://github.com/django/django/pull/19591

--

--
Ticket URL: <https://code.djangoproject.com/ticket/36479#comment:4>

[Django]

#36479: Failing test for black formatter missing install simulation
-------------------------------+--------------------------------------
Reporter: Roelzkie | Owner: Roelzkie
Type: Bug | Status: assigned
Component: Uncategorized | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------+--------------------------------------

Comment (by Mike Edmunds):

Replying to [comment:3 Sarah Boyce]:

> Mike would you be able to confirm this and that the PR fixes the issue?
(See from #36162 you might have a similar setup)

I'm not able to reproduce the issue as reported. (But I'm on macOS 15.5
not 14.7. I'm not sure why that would make a difference. Looks like we're
both on Python 3.13.2.)

{{{#!shell
% python ./runtests.py
user_commands.tests.UtilsTests.test_run_formatters_handles_oserror_for_black_path

Testing against Django installed in '~/dev/django/django' with up to 8
processes
Found 1 test(s).
System check identified no issues (0 silenced).
.
----------------------------------------------------------------------
Ran 1 test in 0.009s

OK

% sw_vers
ProductName: macOS
ProductVersion: 15.5
BuildVersion: 24F74

% uname -mrsv
Darwin 24.5.0 Darwin Kernel Version 24.5.0: Tue Apr 22 19:48:46 PDT 2025;
root:xnu-11417.121.6~2/RELEASE_ARM64_T8103 arm64

% python --version
Python 3.13.2

% git rev-parse --short=7 HEAD
23529b6
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36479#comment:5>

[Django]

#36479: Failing test for black formatter missing install simulation
-------------------------------+--------------------------------------
Reporter: Roelzkie | Owner: Roelzkie
Type: Bug | Status: assigned
Component: Uncategorized | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------+--------------------------------------

Comment (by Roelzkie):

Replying to [comment:5 Mike Edmunds]:

Thank you for checking. I'm not sure why it would break. I pulled again
from the latest main branch just to be sure, but the test still failed. I
also followed your commands to show more information.


{{{#!shell
$ python ./runtests.py
user_commands.tests.UtilsTests.test_run_formatters_handles_oserror_for_black_path
Testing against Django installed in '/Users/user1/django/django' with up

to 8 processes
Found 1 test(s).
System check identified no issues (0 silenced).

F
======================================================================

FAIL: test_run_formatters_handles_oserror_for_black_path
(user_commands.tests.UtilsTests.test_run_formatters_handles_oserror_for_black_path)
[FileNotFoundError]
----------------------------------------------------------------------
Traceback (most recent call last):

File "/Users/user1/django/tests/user_commands/tests.py", line 584, in
test_run_formatters_handles_oserror_for_black_path
self.assertIn(exception.__qualname__, parsed_error)
~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

AssertionError: 'FileNotFoundError' not found in 'Formatters failed to

launch:Traceback (most recent call last):\n File
"/Users/user1/django/django/core/management/utils.py", line 175, in
run_formatters\n subprocess.run(\n ~~~~~~~~~~~~~~^\n
[black_path, "--fast", "--", *written_files],\n
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n
capture_output=True,\n ^^^^^^^^^^^^^^^^^^^^\n )\n ^\n File
"/Users/user1/.pyenv/versions/3.13.2/lib/python3.13/subprocess.py", line
569, in run\n with Popen(*popenargs, **kwargs) as process:\n
~~~~~^^^^^^^^^^^^^^^^^^^^^^\n File
"/Users/user1/.pyenv/versions/3.13.2/lib/python3.13/subprocess.py", line
1051, in __init__\n self._execute_child(args, executable, preexec_fn,
close_fds,\n
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n
pass_fds, cwd, env,\n ^^^^^^^^^^^^^^^^^^^\n
...<5 lines>...\n gid, gids, uid, umask,\n
^^^^^^^^^^^^^^^^^^^^^^\n start_new_session,
process_group)\n
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File
"/Users/user1/.pyenv/versions/3.13.2/lib/python3.13/subprocess.py", line
1987, in _execute_child\n raise child_exception_type(errno_num,
err_msg, err_filename)\nNotADirectoryError: [Errno 20] Not a directory:
\'nonexistent\'\n'

----------------------------------------------------------------------
Ran 1 test in 0.008s

FAILED (failures=1)

$ sw_vers
ProductName: macOS
ProductVersion: 14.7.2
BuildVersion: 23H311

$ uname -mrsv
Darwin 23.6.0 Darwin Kernel Version 23.6.0: Fri Nov 15 15:13:56 PST 2024;
root:xnu-10063.141.1.702.7~1/RELEASE_ARM64_T8103 arm64

$ python --version
Python 3.13.2

$ git rev-parse --short=7 HEAD
23529b6
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36479#comment:6>

[Django]

#36479: Failing test for black formatter missing install simulation

-------------------------------------+-------------------------------------
Reporter: Roelzkie | Owner: Roelzkie
Type: | Status: assigned
Cleanup/optimization |
Component: Core (Management | Version: dev
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 1 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Natalia Bidart):

* component: Uncategorized => Core (Management commands)
* easy: 1 => 0
* stage: Unreviewed => Accepted
* type: Bug => Cleanup/optimization

Comment:

Accepting based on the conversations in the ticket and the PR.

Roelzkie, Mike, Jacob: are either of you using a VM to run the tests? Or
all you all running "native" macos? (if that makes sense)
--
Ticket URL: <https://code.djangoproject.com/ticket/36479#comment:7>

[Django]

#36479: Failing test for black formatter missing install simulation
-------------------------------------+-------------------------------------
Reporter: Roelzkie | Owner: Roelzkie
Type: | Status: assigned
Cleanup/optimization |
Component: Core (Management | Version: dev
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Natalia Bidart):

* needs_tests: 1 => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/36479#comment:8>

[Django]

#36479: Failing test for black formatter missing install simulation
-------------------------------------+-------------------------------------
Reporter: Roelzkie | Owner: Roelzkie
Type: | Status: assigned
Cleanup/optimization |
Component: Core (Management | Version: dev
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Roelzkie):

Replying to [comment:7 Natalia Bidart]:

> Accepting based on the conversations in the ticket and the PR.
>
> Roelzkie, Mike, Jacob: are either of you using a VM to run the tests? Or
all you all running "native" macos? (if that makes sense)

I'm running the test on a native macOS in my case.
--
Ticket URL: <https://code.djangoproject.com/ticket/36479#comment:9>

[Django]

#36479: Failing test for black formatter missing install simulation
-------------------------------------+-------------------------------------
Reporter: Roelzkie | Owner: Roelzkie
Type: | Status: assigned
Cleanup/optimization |
Component: Core (Management | Version: dev
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Jacob Walls):

I was also testing on a bare metal macOS.
--
Ticket URL: <https://code.djangoproject.com/ticket/36479#comment:10>

[Django]

#36479: Failing test for black formatter missing install simulation
-------------------------------------+-------------------------------------
Reporter: Roelzkie | Owner: Roelzkie
Type: | Status: assigned
Cleanup/optimization |
Component: Core (Management | Version: dev
commands) |
Severity: Normal | Resolution:

Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Natalia Bidart):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/36479#comment:11>

[Django]

#36479: Failing test for black formatter missing install simulation
-------------------------------------+-------------------------------------
Reporter: Roelzkie | Owner: Roelzkie

Type: | Status: closed

Cleanup/optimization |
Component: Core (Management | Version: dev
commands) |

Severity: Normal | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by nessita <124304+nessita@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"58fc40427f2928190c8d273ceff2121b738e768b" 58fc404]:
{{{#!CommitTicketReference repository=""
revision="58fc40427f2928190c8d273ceff2121b738e768b"
Fixed #36479 -- Improved how FileNotFoundError is triggered in code
formatter tests.

Ensured the test for formatter subprocess FileNotFoundError doesn't rely
on platform-specific behavior, improving reliability on macOS and other
systems by consistently using pathlib to build test paths.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36479#comment:12>