Bug when running tests with tags(?) - module level exceptions fail silently, and tests pass

[Cammil Taank]

Hello all,

I have encountered what I think is very weird behaviour. When I run tests with the --tag option, modules that raise exceptions appear to be silently ignored.

Steps to reproduce:

1. Create django app.

2. Add this test module:

from django.test import TestCase

from django.test import tag

assert 0

@tag('abc')

class TempTest(TestCase):

    def test_basic(self):

        self.assertFalse(True)

3. Run python manage.py test

This spits out the assertion error on stdin, as I would expect.

4. Run python manage.py test --tag=abc

This DOES NOT spit out the assertion error on stdin, as I would expect. In fact it says tests were run successfully.

This to me seems pretty dangerous. A module level error should be seen clearly, and tests should certainly not pass.

Tested on python 3.8.3, django 3.1.4.

Cammil