Flakey tests and django.db.utils.InterfaceError: connection already closed

[thinkwell]

I could use some assistance in troubleshooting a psycopg2.InterfaceError problem that has erratic behavior. Here is the test module suite that I'm running to test these models, and in this state all the tests in the module pass. However, it's extremely easy to trigger a psycopg2.InterfaceError, although not relating to the lines changed. For example, if I change:

class TestSilly(SimpleTestCase):
 
    def test_is_not_false(self):
        self.assertTrue(False is not True)

to

class TestSilly(TestCase):
 
    def test_is_not_false(self):
        self.assertTrue(False is not True)

 

Then this error is raised, but referencing TestPolicyModel instead of TestSilly!

======================================================================
ERROR: test_toggle_autofix (accounts.tests.test_models.TestPolicyModel)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 137, in _cursor
    return self.create_cursor()
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/postgresql_psycopg2/base.py", line 212, in create_cursor
    cursor = self.connection.cursor()
psycopg2.InterfaceError: connection already closed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "apps/accounts/tests/test_models.py", line 174, in test_toggle_autofix
    p = Policy.objects.first()
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/manager.py", line 127, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/query.py", line 490, in first
    objects = list((self if self.ordered else self.order_by('pk'))[:1])
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/query.py", line 162, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/query.py", line 965, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/query.py", line 238, in iterator
    results = compiler.execute_sql()
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/sql/compiler.py", line 838, in execute_sql
    cursor = self.connection.cursor()
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 164, in cursor
    cursor = self.make_cursor(self._cursor())
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 137, in _cursor
    return self.create_cursor()
  File "/usr/local/lib/python3.4/dist-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python3.4/dist-packages/django/utils/six.py", line 658, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 137, in _cursor
    return self.create_cursor()
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/postgresql_psycopg2/base.py", line 212, in create_cursor
    cursor = self.connection.cursor()
django.db.utils.InterfaceError: connection already closed

For another example, if I change

TestPolicyModel(TestCase)

to

TestPolicyModel(SimpleTestCase)

then, I get the following error regarding TestRedwoodModel

======================================================================
ERROR: setUpClass (accounts.tests.test_models.TestRedwoodModel)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 137, in _cursor
    return self.create_cursor()
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/postgresql_psycopg2/base.py", line 212, in create_cursor
    cursor = self.connection.cursor()
psycopg2.InterfaceError: connection already closed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "apps/accounts/tests/test_models.py", line 195, in setUpClass
    cls.ru = RedwoodUsers.objects.first()
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/manager.py", line 127, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/query.py", line 490, in first
    objects = list((self if self.ordered else self.order_by('pk'))[:1])
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/query.py", line 162, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/query.py", line 965, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/query.py", line 238, in iterator
    results = compiler.execute_sql()
  File "/usr/local/lib/python3.4/dist-packages/django/db/models/sql/compiler.py", line 838, in execute_sql
    cursor = self.connection.cursor()
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 164, in cursor
    cursor = self.make_cursor(self._cursor())
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 137, in _cursor
    return self.create_cursor()
  File "/usr/local/lib/python3.4/dist-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python3.4/dist-packages/django/utils/six.py", line 658, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/base/base.py", line 137, in _cursor
    return self.create_cursor()
  File "/usr/local/lib/python3.4/dist-packages/django/db/backends/postgresql_psycopg2/base.py", line 212, in create_cursor
    cursor = self.connection.cursor()
django.db.utils.InterfaceError: connection already closed

If I comment out TestRedwoodModel, then TestPolicyModel passes, whether it inherits SimpleTestCase or TestCase.

So I'm thoroughly confused, and can't seem to discern a pattern in these tracebacks. Any assistance is greatly appreciated!

[Christopher R.]

Have you been able to fix the problem? I am encountering the same issue on both Django 1.8.4 and 1.8.5.

Any help would be appreciated.

...

[Tim Graham]

A test project that reproduces the problem would be helpful.

[Collin Anderson]

Hello,

Here are some questions that might help debugging this:

Do other tests that use the database run ok?

If you run _only_ this test, is it still a problem?

Are you using threads at all?

Is there an exception that's being silenced somewhere else?

Collin

On Tuesday, October 27, 2015 at 9:14:13 AM UTC-4, Christopher R. wrote:

[thinkwell]

I'd parked this problem for quite some time, until it finally became incontestably necessary to wrestle the issue to the ground.

What I found, after many hours of experimenting - too much like the random walk, I suppose, is that every time I overrode setUpClass, I also had to override tearDownClass. Before, if I had nothing to clean up after the test, I wouldn't bother overriding tearDownClass.

When I went to all my tests and added

@classmethod
def tearDownClass(cls):
     pass

Then my problems went away. I reckon that's in the docs somewhere. But it seems to me that tearDownClass should be able to degrade more gracefully. :weary:

[Peter Nociar]

Thanks, this fixed my issue.