[Django] #35669: Improve `RuntimeError: Max post-process passes exceeded.` error

[Django]

#35669: Improve `RuntimeError: Max post-process passes exceeded.` error
-------------------------------------+-------------------------------------
Reporter: Michael | Type: New
| feature
Status: new | Component:
| contrib.staticfiles
Version: 5.1 | Severity: Normal
Keywords: collect static | Triage Stage:
errors | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Having just spend 3 hours trying to debug a collect static issue, to
prevent others wasting so much time, I recommend printing the files that
caused max depth to be exceeded, often when a file references itself it
causes recursion, but the only clue it currently prints is 'All' which
does not narrow done the problem. The proposed change prints the problem
files only that keep chaning and can't be resolved:

I recommend changing from:
{{{
# contrib/staticfiles/storage.py line 313: in def post_process(self,
paths, dry_run=False, **options):
unresolved_paths = []
for i in range(self.max_post_process_passes):
substitutions = False
for name, hashed_name, processed, subst in self._post_process(
paths, adjustable_paths, hashed_files
):
# Overwrite since hashed_name may be newer.
processed_adjustable_paths[name] = (name, hashed_name,
processed)
if subst and i == self.max_post_process_passes - 1:
unresolved_paths.append(name)
substitutions = substitutions or subst

if not substitutions:
break

if substitutions:
problem_paths_str = ", ".join(unresolved_paths) if
unresolved_paths else "All"
yield problem_paths_str, None, RuntimeError("Max post-process
passes exceeded.")
}}}

I recommend changing to:
{{{
# contrib/staticfiles/storage.py line 313: in def post_process(self,
paths, dry_run=False, **options):
unresolved_paths = [] #
< -- add this line 1/5
for i in range(self.max_post_process_passes):
substitutions = False
for name, hashed_name, processed, subst in self._post_process(
paths, adjustable_paths, hashed_files
):
# Overwrite since hashed_name may be newer.
processed_adjustable_paths[name] = (name, hashed_name,
processed)
if subst and i == self.max_post_process_passes - 1: #
< -- add this line 2/5
unresolved_paths.append(name) #
< -- add this line 3/5
substitutions = substitutions or subst

if not substitutions:
break

if substitutions:
problem_paths_str = ", ".join(unresolved_paths) if
unresolved_paths else "All" # < -- add this line 4/5
yield problem_paths_str, None, RuntimeError("Max post-process
passes exceeded.") # < -- change this line 5/5
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35669>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#35669: Improve `RuntimeError: Max post-process passes exceeded.` error
-------------------------------------+-------------------------------------

Reporter: Michael | Owner: (none)

Type: New feature | Status: new
Component: contrib.staticfiles | Version: 5.1

Severity: Normal | Resolution:

Keywords: collect static | Triage Stage:
errors | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Description changed by Michael:

Old description:

New description:

Having just spend 3 hours trying to debug a collect static issue, to help
future travellers, I recommend printing the files that caused max depth to

be exceeded, often when a file references itself it causes recursion, but

the only clue it currently prints is 'All' which does not narrow down the
problem. We can surely do better than that! The proposed change prints the

--
Ticket URL: <https://code.djangoproject.com/ticket/35669#comment:1>

[Django]

#35669: Improve `RuntimeError: Max post-process passes exceeded.` error
-------------------------------------+-------------------------------------
Reporter: Michael | Owner: (none)
Type: New feature | Status: new
Component: contrib.staticfiles | Version: 5.1
Severity: Normal | Resolution:
Keywords: collect static | Triage Stage:
errors | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by Michael:

Old description:

I recommend changing to (see the comments on the right of the proposed
changed lines 1/5 to 5/5):

Ticket URL: <https://code.djangoproject.com/ticket/35669#comment:2>

[Django]

New description:

Having just spend 3 hours trying to debug a collect static issue, to help
future travellers, I recommend printing the files that caused max depth to
be exceeded, often when a file references itself it causes recursion, but
the only clue it currently prints is 'All' which does not narrow down the
problem. We can surely do better than that! The proposed change prints the
problem files only that keep chaning and can't be resolved:

So instead of getting:
{{{
Post-processing 'All' failed!
}}}
We get the new and improved:
{{{
Post-processing 'jsapp/jsapp/notify.min.js' failed!
}}}
Or if more than one file:
{{{
Post-processing 'jsapp/jsapp/notify.min.js,
jsapp/jsapp/somethingelse.min.js' failed!

Ticket URL: <https://code.djangoproject.com/ticket/35669#comment:3>

[Django]

#35669: Improve `RuntimeError: Max post-process passes exceeded.` error
-------------------------------------+-------------------------------------
Reporter: Michael | Owner: (none)

Type: | Status: new
Cleanup/optimization |

Component: contrib.staticfiles | Version: 5.1
Severity: Normal | Resolution:

Keywords: collect static | Triage Stage: Accepted
errors |

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* stage: Unreviewed => Accepted
* type: New feature => Cleanup/optimization

Comment:

Adding the suggestion as a diff
{{{#!diff
diff --git a/django/contrib/staticfiles/storage.py
b/django/contrib/staticfiles/storage.py
index 04a5edbd30..5c41fd6828 100644
--- a/django/contrib/staticfiles/storage.py
+++ b/django/contrib/staticfiles/storage.py
@@ -309,7 +309,7 @@ class HashedFilesMixin:

paths = {path: paths[path] for path in adjustable_paths}
substitutions = False
-
+ unresolved_paths = []

for i in range(self.max_post_process_passes):
substitutions = False
for name, hashed_name, processed, subst in
self._post_process(

@@ -318,12 +318,15 @@ class HashedFilesMixin:

# Overwrite since hashed_name may be newer.
processed_adjustable_paths[name] = (name, hashed_name,
processed)

substitutions = substitutions or subst

+ if subst and i == self.max_post_process_passes - 1:
+ unresolved_paths.append(name)

if not substitutions:
break

if substitutions:

- yield "All", None, RuntimeError("Max post-process passes
exceeded.")
+ problem_paths_str = ", ".join(unresolved_paths) if
unresolved_paths else "All"
+ yield problem_paths_str, None, RuntimeError("Max post-process
passes exceeded.")

# Store the processed paths
self.hashed_files.update(hashed_files)
}}}

Appreciate clear error messages so sounds good to me
--
Ticket URL: <https://code.djangoproject.com/ticket/35669#comment:4>

[Django]

#35669: Improve `RuntimeError: Max post-process passes exceeded.` error
-------------------------------------+-------------------------------------

Reporter: Michael | Owner: Jae Hyuck
Type: | Sa
Cleanup/optimization | Status: assigned

Component: contrib.staticfiles | Version: 5.1
Severity: Normal | Resolution:
Keywords: collect static | Triage Stage: Accepted
errors |

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jae Hyuck Sa ):

* has_patch: 0 => 1
* owner: (none) => Jae Hyuck Sa
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/35669#comment:5>

[Django]

#35669: Improve `RuntimeError: Max post-process passes exceeded.` error
-------------------------------------+-------------------------------------
Reporter: Michael | Owner: Jae Hyuck
Type: | Sa
Cleanup/optimization | Status: assigned
Component: contrib.staticfiles | Version: 5.1
Severity: Normal | Resolution:
Keywords: collect static | Triage Stage: Accepted
errors |
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* needs_better_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/35669#comment:6>

[Django]

#35669: Improve `RuntimeError: Max post-process passes exceeded.` error
-------------------------------------+-------------------------------------
Reporter: Michael | Owner: Jae Hyuck
Type: | Sa
Cleanup/optimization | Status: assigned
Component: contrib.staticfiles | Version: 5.1
Severity: Normal | Resolution:
Keywords: collect static | Triage Stage: Accepted
errors |
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jae Hyuck Sa ):

* needs_better_patch: 1 => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/35669#comment:7>

[Django]

#35669: Improve `RuntimeError: Max post-process passes exceeded.` error
-------------------------------------+-------------------------------------
Reporter: Michael | Owner: Jae Hyuck
Type: | Sa
Cleanup/optimization | Status: assigned
Component: contrib.staticfiles | Version: 5.1
Severity: Normal | Resolution:

Keywords: collect static | Triage Stage: Ready for
errors | checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/35669#comment:8>

[Django]

#35669: Improve `RuntimeError: Max post-process passes exceeded.` error
-------------------------------------+-------------------------------------
Reporter: Michael | Owner: Jae Hyuck
Type: | Sa

Cleanup/optimization | Status: closed
Component: contrib.staticfiles | Version: 5.1
Severity: Normal | Resolution: fixed

Keywords: collect static | Triage Stage: Ready for
errors | checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Sarah Boyce <42296566+sarahboyce@…>):

* resolution: => fixed
* status: assigned => closed

Comment:

In [changeset:"2ff00251f929cc3e014dd447f6847196e66e69b8" 2ff0025]:
{{{#!CommitTicketReference repository=""
revision="2ff00251f929cc3e014dd447f6847196e66e69b8"
Fixed #35669 -- Improved max post-process passes exceeded error message in
HashedFilesMixin.

Signed-off-by: SaJH <wogur...@gmail.com>
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/35669#comment:9>