[Django] #37076: assertWarnsMessage() only checks the first warning for a matching message

[Django]

#37076: assertWarnsMessage() only checks the first warning for a matching message
-------------------------------------+-------------------------------------
Reporter: Mike Edmunds | Type: Bug
Status: new | Component: Testing
| framework
Version: 6.0 | Severity: Normal
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
`assertWarnsMessage(category, message)` fails if the ''first'' warning
issued in `category` does not contain `message`, even if some later
warning does match.

As a result, test cases depend on the specific order warnings are issued.
This can become confusing when there are several related deprecations
going in at once, or when a feature being tested uses some other feature
that also issues a warning.

For example, you might expect this test to pass, but it fails with
`AssertionError: 'Deprecated in MyApp' not found in 'Deprecated in
Django'`:

{{{#!python
class MyAppTests(SimpleTestCase):
def test_deprecated_in_my_app(self):
with self.assertWarnsMessage(DeprecationWarning, "Deprecated in
MyApp"):
# MyApp calls something in Django that happens to be
deprecated:
warnings.warn("Deprecated in Django",
RemovedInNextVersionWarning)
# Then MyApp issues the warning being tested:
warnings.warn("Deprecated in MyApp", DeprecationWarning)
}}}

If you switch the order of the two warnings, the test passes. (And the
second warning is then ignored—see #37072.)

There is a workaround by combining a very carefully crafted
`ignore_warnings()` context with `assertWarnsMessage()`. (It requires
knowing that the semi-documented `ignore_warnings()` also accepts a
`message` argument. And that it interprets it quite differently from
`assertWarnsMessage()`.)
--
Ticket URL: <https://code.djangoproject.com/ticket/37076>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#37076: assertWarnsMessage() only checks the first warning for a matching message

-----------------------------------+--------------------------------------
Reporter: Mike Edmunds | Owner: (none)

Type: Bug | Status: new
Component: Testing framework | Version: 6.0

Severity: Normal | Resolution:

Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-----------------------------------+--------------------------------------
Comment (by Artyom Kotovskiy):

Would love to work on it in case it gets accepted.
--
Ticket URL: <https://code.djangoproject.com/ticket/37076#comment:1>

[Django]

#37076: assertWarnsMessage() only checks the first warning for a matching message
-----------------------------------+------------------------------------

Reporter: Mike Edmunds | Owner: (none)
Type: Bug | Status: new
Component: Testing framework | Version: 6.0
Severity: Normal | Resolution:

Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------------+------------------------------------

Changes (by Jacob Walls):

* stage: Unreviewed => Accepted

Comment:

Ouch, great find. This is supposed to be a non-regex version of the stdlib
`assertWarnsRegex`, which isn't order-sensitive.
--
Ticket URL: <https://code.djangoproject.com/ticket/37076#comment:2>

[Django]

#37076: assertWarnsMessage() only checks the first warning for a matching message

-----------------------------------+---------------------------------------
Reporter: Mike Edmunds | Owner: artirix1927
Type: Bug | Status: assigned

Component: Testing framework | Version: 6.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0

-----------------------------------+---------------------------------------
Changes (by Jacob Walls):

* owner: (none) => artirix1927
* status: new => assigned

--
Ticket URL: <https://code.djangoproject.com/ticket/37076#comment:3>

[Django]

#37076: assertWarnsMessage() only checks the first warning for a matching message
-----------------------------------+---------------------------------------
Reporter: Mike Edmunds | Owner: artirix1927
Type: Bug | Status: assigned
Component: Testing framework | Version: 6.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------------+---------------------------------------

Comment (by Mike Edmunds):

I would suggest trying to fix #37072 at the same time—they're closely
related. I outlined a ''possible'' approach in a comment there. There are
3-4 existing tests that seem to unintentionally depend on either this or
#37072's behavior, so will need to be updated as part of the fix.
--
Ticket URL: <https://code.djangoproject.com/ticket/37076#comment:4>

[Django]

#37076: assertWarnsMessage() only checks the first warning for a matching message
-----------------------------------+---------------------------------------
Reporter: Mike Edmunds | Owner: artirix1927
Type: Bug | Status: assigned
Component: Testing framework | Version: 6.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------------+---------------------------------------

Changes (by Artyom Kotovskiy):

* has_patch: 0 => 1

Comment:

Pr -> [https://github.com/django/django/pull/21228]
--
Ticket URL: <https://code.djangoproject.com/ticket/37076#comment:5>

[Django]

#37076: assertWarnsMessage() only checks the first warning for a matching message
-----------------------------------+---------------------------------------
Reporter: Mike Edmunds | Owner: artirix1927
Type: Bug | Status: assigned
Component: Testing framework | Version: 6.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0
-----------------------------------+---------------------------------------

Changes (by Mike Edmunds):

* needs_better_patch: 0 => 1

--
Ticket URL: <https://code.djangoproject.com/ticket/37076#comment:6>