[Django] #37235: --debug-sql prints SQLParseError instead of falling back to unformatted SQL

[Django]

#37235: --debug-sql prints SQLParseError instead of falling back to unformatted SQL
-----------------------------+---------------------------------------------
Reporter: Jacob Walls | Type: Bug
Status: new | Component: Testing framework
Version: 6.0 | Severity: Release blocker
Keywords: debug-sql | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------+---------------------------------------------
The `--debug-sql` option in the test runner formats SQL with `sqlparse`.
To fix uncontrolled resource consumption,
[https://sqlparse.readthedocs.io/en/latest/changes.html#release-0-5-5-dec-19-2025
version 0.5.5 raises] `SQLParseError` instead of swallowing SQL with
`None`.


I'm suggesting that we should catch the `SQLParseError` and fallback to
the unformatted SQL. Right now, we're getting the traceback and its
containing logging error spewed to the output. (Of course, this test and
its neighbor could have used a lower limit, but at least that caught this
issue!)

Happy to take backport advice: my first thought is at least 6.1, as it's a
compatibility issue with an upstream dependency. I'd consider 6.0 as well.

{{{
./tests/runtests.py
bulk_create.tests.BulkCreateTests.test_explicit_batch_size_respects_max_batch_size
--debug-sql
}}}
{{{#!py
Testing against Django installed in '/Users/jwalls/django/django' with up
to 8 processes
Found 1 test(s).
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
--- Logging error ---
Traceback (most recent call last):
File
"/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/logging/__init__.py",
line 1151, in emit
msg = self.format(record)
File
"/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/logging/__init__.py",
line 999, in format
return fmt.format(record)
~~~~~~~~~~^^^^^^^^
File "/Users/jwalls/django/django/test/runner.py", line 53, in format
record.args = (args[0], formatted_sql := format_sql(sql), *args[2:])
~~~~~~~~~~^^^^^
File "/Users/jwalls/django/django/db/backends/base/operations.py", line
876, in format_debug_sql
return sqlparse.format(sql, reindent=True, keyword_case="upper")
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jwalls/my314/lib/python3.14/site-
packages/sqlparse/__init__.py", line 65, in format
return "".join(stack.run(sql, encoding))
~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jwalls/my314/lib/python3.14/site-
packages/sqlparse/engine/filter_stack.py", line 41, in run
stmt = grouping.group(stmt)
File "/Users/jwalls/my314/lib/python3.14/site-
packages/sqlparse/engine/grouping.py", line 474, in group
func(stmt)
~~~~^^^^^^
File "/Users/jwalls/my314/lib/python3.14/site-
packages/sqlparse/engine/grouping.py", line 77, in group_brackets
_group_matching(tlist, sql.SquareBrackets)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jwalls/my314/lib/python3.14/site-
packages/sqlparse/engine/grouping.py", line 37, in _group_matching
raise SQLParseError(
f"Maximum number of tokens exceeded ({MAX_GROUPING_TOKENS})."
)
sqlparse.exceptions.SQLParseError: Maximum number of tokens exceeded
(10000).
Call stack:
File "/Users/jwalls/django/./tests/runtests.py", line 786, in <module>
failures = django_tests(
File "/Users/jwalls/django/./tests/runtests.py", line 413, in
django_tests
failures = test_runner.run_tests(test_labels)
File "/Users/jwalls/django/django/test/runner.py", line 1133, in
run_tests
result = self.run_suite(suite)
File "/Users/jwalls/django/django/test/runner.py", line 1060, in
run_suite
return runner.run(suite)
File
"/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/unittest/runner.py",
line 257, in run
test(result)
File
"/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/unittest/suite.py",
line 84, in __call__
return self.run(*args, **kwds)
File
"/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/unittest/suite.py",
line 122, in run
test(result)
File "/Users/jwalls/django/django/test/testcases.py", line 314, in
__call__
self._setup_and_call(result)
File "/Users/jwalls/django/django/test/testcases.py", line 369, in
_setup_and_call
super().__call__(result)
File
"/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/unittest/case.py",
line 725, in __call__
return self.run(*args, **kwds)
File
"/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/unittest/case.py",
line 669, in run
self._callTestMethod(testMethod)
File
"/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/unittest/case.py",
line 615, in _callTestMethod
result = method()
File "/Users/jwalls/django/django/test/testcases.py", line 1591, in
skip_wrapper
return test_func(*args, **kwargs)
File "/Users/jwalls/django/tests/bulk_create/tests.py", line 293, in
test_explicit_batch_size_respects_max_batch_size
Country.objects.bulk_create(objs, batch_size=max_batch_size + 1)
File "/Users/jwalls/django/django/db/models/manager.py", line 87, in
manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/Users/jwalls/django/django/db/models/query.py", line 899, in
bulk_create
returned_columns = self._batched_insert(
File "/Users/jwalls/django/django/db/models/query.py", line 2178, in
_batched_insert
self._insert(
File "/Users/jwalls/django/django/db/models/query.py", line 2140, in
_insert
return query.get_compiler(using=using).execute_sql(returning_fields)
File "/Users/jwalls/django/django/db/models/sql/compiler.py", line 1936,
in execute_sql
cursor.execute(sql, params)
File "/Users/jwalls/django/django/db/backends/utils.py", line 121, in
execute
with self.debug_sql(sql, params, use_last_executed_query=True):
File
"/Library/Frameworks/Python.framework/Versions/3.14/lib/python3.14/contextlib.py",
line 148, in __exit__
next(self.gen)
File "/Users/jwalls/django/django/db/backends/utils.py", line 151, in
debug_sql
logger.debug(
Message: '(%.3f) %s; args=%s; alias=%s'
Arguments: (0.0017766660002962453, 'INSERT INTO "bulk_create_country"
("name", "iso_two_letter", "description") VALUES (\'Country 0\', \'\',
\'\'), (\'Country 1\', \'\', \'\'), (\'Country 2\', \'\', \'\'),
(\'Country 3\', \'\', \'\'), (\'Country 4\', \'\', \'\'), (\'Country 5\',
\'\', \'\'), (\'Country 6\', \'\',

... truncated ...

'', 'Country 984', '', '', 'Country 985', '', '', 'Country 986', '', '',
'Country 987', '', '', 'Country 988', '', '', 'Country 989', '', '',
'Country 990', '', '', 'Country 991', '', '', 'Country 992', '', '',
'Country 993', '', '', 'Country 994', '', '', 'Country 995', '', '',
'Country 996', '', '', 'Country 997', '', '', 'Country 998', '', '',
'Country 999', '', ''), 'default')
.
----------------------------------------------------------------------
Ran 1 test in 0.081s

OK
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/37235>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#37235: --debug-sql prints SQLParseError instead of falling back to unformatted SQL

-----------------------------------+------------------------------------
Reporter: Jacob Walls | Owner: (none)

Type: Bug | Status: new
Component: Testing framework | Version: 6.0

Severity: Release blocker | Resolution:
Keywords: debug-sql | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-----------------------------------+------------------------------------
Changes (by Sarah Boyce):

* stage: Unreviewed => Accepted

Comment:

Thank you!
Agree we can backport to 6.0. I think we have done compatibility backports
to a mainstream supported branch before (e.g. #36535) so there is a
precedent
--
Ticket URL: <https://code.djangoproject.com/ticket/37235#comment:1>