[Django] #36859: SimpleTestCase.assertContains can't be called multiple times on a streaming response

[Django]

#36859: SimpleTestCase.assertContains can't be called multiple times on a streaming
response
-------------------------------------+-------------------------------------
Reporter: Baptiste | Owner: Baptiste Mispelon
Mispelon |
Type: New | Status: assigned
feature |
Component: Testing | Version: 6.0
framework | Keywords: assertContains,
Severity: Normal | streaming
Triage Stage: | Has patch: 1
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
I recently got an unexpected test failure when using
`SimpleTestCase.assertContains()`:

{{{#!python
def test_some_view(self):
response = self.client.get("/some/url/")
self.assertContains(response, "something")
self.assertContains(response, "another thing") # This failed
}}}

Even though my view at `/some/url/` was returning a response that did
include both `"something"` and `"another thing"`, the test was failing on
the second assertion. After some confusion, I eventually figured out that
the cause of this failure was that the view was returning a
`StreamingHttpResponse`.

This failure makes sense if you understand the internals of streaming
responses, but I'd argue that the test code shouldn't have to care about
the nature of the response, and that repeated calls to `assertContains`
should work transparently.

As far as I can tell, this behavior is not documented (one way or another)
and has always been present (see
82b3e6ffcb9d810cc0e3ec27d25f89ce1fd525e0).

(PR with test case and tentative fix will be attached to this ticket soon)
--
Ticket URL: <https://code.djangoproject.com/ticket/36859>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36859: SimpleTestCase.assertContains can't be called multiple times on a streaming
response
-------------------------------------+-------------------------------------

Reporter: Baptiste Mispelon | Owner: Baptiste
| Mispelon
Type: New feature | Status: assigned
Component: Testing framework | Version: 6.0
Severity: Normal | Resolution:
Keywords: assertContains, | Triage Stage:
streaming | Unreviewed
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Baptiste Mispelon):

PR here: https://github.com/django/django/pull/20527
--
Ticket URL: <https://code.djangoproject.com/ticket/36859#comment:1>

[Django]

#36859: SimpleTestCase.assertContains can't be called multiple times on a streaming
response
-------------------------------------+-------------------------------------
Reporter: Baptiste Mispelon | Owner: Baptiste
| Mispelon

Type: Bug | Status: assigned

Component: Testing framework | Version: 6.0
Severity: Normal | Resolution:

Keywords: assertContains, | Triage Stage: Accepted
streaming |

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Natalia Bidart):

* cc: Carlton Gibson (added)
* stage: Unreviewed => Accepted
* type: New feature => Bug

Comment:

Thank you Baptiste, I think it makes sense to expect multiple calls of
`assertContains` to work on a streamed response.
--
Ticket URL: <https://code.djangoproject.com/ticket/36859#comment:2>

[Django]

#36859: SimpleTestCase.assertContains can't be called multiple times on a streaming
response
-------------------------------------+-------------------------------------
Reporter: Baptiste Mispelon | Owner: Baptiste
| Mispelon
Type: Bug | Status: assigned
Component: Testing framework | Version: 6.0
Severity: Normal | Resolution:
Keywords: assertContains, | Triage Stage: Accepted
streaming |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Carlton Gibson):

I'm not sure about calling this a bug. It seems very much a new feature. —
The existing behaviour is long standing, and will be expected by anyone
who's used streaming responses over time. So, if we are going to change
it, it would merit a release note, etc.

> I'd argue that the test code shouldn't have to care about the nature of

the response...

Yeah. Gah. Streaming responses are their own thing. Our test probably
should know it's dealing with a streaming response, and perform extra
checks accordingly. Collecting the content into a variable and then
asserting on that, would be pretty standard.

But 🤷 — I don't have a particular objection if it doesn't break anything.
(Only commenting because I was CC'd.)
--
Ticket URL: <https://code.djangoproject.com/ticket/36859#comment:3>

[Django]

#36859: SimpleTestCase.assertContains can't be called multiple times on a streaming
response
-------------------------------------+-------------------------------------
Reporter: Baptiste Mispelon | Owner: Baptiste
| Mispelon

Type: New feature | Status: assigned

Component: Testing framework | Version: 6.0
Severity: Normal | Resolution:
Keywords: assertContains, | Triage Stage: Accepted
streaming |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Natalia Bidart):

* type: Bug => New feature

Comment:

Replying to [comment:3 Carlton Gibson]:

> But 🤷 — I don't have a particular objection if it doesn't break
anything. (Only commenting because I was CC'd.)

Thank you Carlton, I cc'd you because I was very keen to knowing your view
on this matter. Happy to restore the New Feature type.
--
Ticket URL: <https://code.djangoproject.com/ticket/36859#comment:4>

[Django]

#36859: SimpleTestCase.assertContains can't be called multiple times on a streaming
response
-------------------------------------+-------------------------------------
Reporter: Baptiste Mispelon | Owner: Baptiste
| Mispelon
Type: New feature | Status: assigned
Component: Testing framework | Version: 6.0
Severity: Normal | Resolution:
Keywords: assertContains, | Triage Stage: Accepted
streaming |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Comment (by Carlton Gibson):

👍
--
Ticket URL: <https://code.djangoproject.com/ticket/36859#comment:5>