How can I hide my stack frames in a TestCase subclass?

[David Banks]

I want to add a custom assert method to a TestCase subclass. I tried to
copy my implementation from the unittest module so that it would match
the behaviour of the regular TestCase as closely as possible. (I would
prefer to just delegate to self.assertEqual() but this causes even more
backtrace noise, see below.) The unittest module seems to automatically
hide some internal details of its implementation when reporting failed
assertions.

import unittest

class MyTestCase(unittest.TestCase):
def assertLengthIsOne(self, sequence, msg=None):
if len(sequence) != 1:
msg = self._formatMessage(msg, "length is not one")
raise self.failureException(msg)

class TestFoo(MyTestCase):
seq = (1, 2, 3, 4, 5)

def test_stock_unittest_assertion(self):
self.assertEqual(len(self.seq), 1)

def test_custom_assertion(self):
self.assertLengthIsOne(self.seq)


unittest.main()

The output of this is as such:

amoe@vuurvlieg $ python unittest-demo.py
FF
======================================================================
FAIL: test_custom_assertion (__main__.TestFoo)
----------------------------------------------------------------------
Traceback (most recent call last):
File "unittest-demo.py", line 16, in test_custom_assertion
self.assertLengthIsOne(self.seq)
File "unittest-demo.py", line 7, in assertLengthIsOne
raise self.failureException(msg)
AssertionError: length is not one

======================================================================
FAIL: test_stock_unittest_assertion (__main__.TestFoo)
----------------------------------------------------------------------
Traceback (most recent call last):
File "unittest-demo.py", line 13, in test_stock_unittest_assertion
self.assertEqual(len(self.seq), 1)
AssertionError: 5 != 1

----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (failures=2)

Note that the custom assert method causes a stack trace with two frames,
one inside the method itself, whereas the stock unittest method only has
one frame, the relevant line in the user's code. How can I apply this
frame-hiding behaviour to my own method?

[Peter Otten]

Move MyTestCase in a separate module and define a global variable

__unittest = True

$ cat mytestcase.py
import unittest

__unittest = True

class MyTestCase(unittest.TestCase):
def assertLengthIsOne(self, sequence, msg=None):
if len(sequence) != 1:
msg = self._formatMessage(msg, "length is not one")
raise self.failureException(msg)

$ cat mytestcase_demo.py
import unittest
from mytestcase import MyTestCase

class TestFoo(MyTestCase):
seq = (1, 2, 3, 4, 5)

def test_stock_unittest_assertion(self):
self.assertEqual(len(self.seq), 1)

def test_custom_assertion(self):
self.assertLengthIsOne(self.seq)

if __name__ == "__main__":
unittest.main()

$ python mytestcase_demo.py

FF
======================================================================
FAIL: test_custom_assertion (__main__.TestFoo)
----------------------------------------------------------------------
Traceback (most recent call last):

File "mytestcase_demo.py", line 11, in test_custom_assertion
self.assertLengthIsOne(self.seq)

AssertionError: length is not one

======================================================================
FAIL: test_stock_unittest_assertion (__main__.TestFoo)
----------------------------------------------------------------------
Traceback (most recent call last):

File "mytestcase_demo.py", line 8, in test_stock_unittest_assertion

self.assertEqual(len(self.seq), 1)
AssertionError: 5 != 1

----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (failures=2)

$

[Manuel Pégourié-Gonnard]

Peter Otten scripsit :

> David Banks wrote:
>
>> Note that the custom assert method causes a stack trace with two frames,
>> one inside the method itself, whereas the stock unittest method only has
>> one frame, the relevant line in the user's code. How can I apply this
>> frame-hiding behaviour to my own method?
>
> Move MyTestCase in a separate module and define a global variable
>
> __unittest = True
>

Hum, is it documented somewhere? I can't find it in the doc. Also, I'm
curious to know what kind of magic it's using.

--
Manuel P�gouri�-Gonnard - http://people.math.jussieu.fr/~mpg/

[Peter Otten]

Manuel Pégourié-Gonnard wrote:

> Peter Otten scripsit :
>
>> David Banks wrote:
>>
>>> Note that the custom assert method causes a stack trace with two frames,
>>> one inside the method itself, whereas the stock unittest method only has
>>> one frame, the relevant line in the user's code. How can I apply this
>>> frame-hiding behaviour to my own method?
>>
>> Move MyTestCase in a separate module and define a global variable
>>
>> __unittest = True
>>
> Hum, is it documented somewhere? I can't find it in the doc. Also, I'm
> curious to know what kind of magic it's using.

I took advantage of the fact that Python is open source and had a look into
the source code ;)

$ cd /usr/lib/python2.7/unittest
$ grep frame *.py -C2
...
result.py-
result.py- def _is_relevant_tb_level(self, tb):
result.py: return '__unittest' in tb.tb_frame.f_globals
result.py-
...

$ grep _is_relevant_tb_level *.py -C5
result.py-
result.py- def _exc_info_to_string(self, err, test):
result.py- """Converts a sys.exc_info()-style tuple of values into a
string."""
result.py- exctype, value, tb = err
result.py- # Skip test runner traceback levels
result.py: while tb and self._is_relevant_tb_level(tb):
result.py- tb = tb.tb_next
result.py-
...

And so on. I actually used an editor, not grep -- but you get the idea.

[Manuel Pégourié-Gonnard]

Peter Otten scripsit :

> Manuel P�gouri�-Gonnard wrote:
>
>> Peter Otten scripsit :

>>
>>> __unittest = True
>>>
>> Hum, is it documented somewhere? I can't find it in the doc. Also, I'm
>> curious to know what kind of magic it's using.
>
> I took advantage of the fact that Python is open source and had a look into
> the source code ;)
>

Fair enough.

However, there was an implied question in the "documented" part: can
we rely on it? Isn't it considered an implementation detail (names
starting with underscores)?

> $ cd /usr/lib/python2.7/unittest
> $ grep frame *.py -C2
> ...
> result.py-
> result.py- def _is_relevant_tb_level(self, tb):
> result.py: return '__unittest' in tb.tb_frame.f_globals
> result.py-
> ...
>
> $ grep _is_relevant_tb_level *.py -C5
> result.py-
> result.py- def _exc_info_to_string(self, err, test):
> result.py- """Converts a sys.exc_info()-style tuple of values into a
> string."""
> result.py- exctype, value, tb = err
> result.py- # Skip test runner traceback levels
> result.py: while tb and self._is_relevant_tb_level(tb):
> result.py- tb = tb.tb_next
> result.py-
> ...
>
> And so on. I actually used an editor, not grep -- but you get the idea.

Sure, thanks.

[Peter Otten]

Manuel Pégourié-Gonnard wrote:

> However, there was an implied question in the "documented" part: can
> we rely on it? Isn't it considered an implementation detail (names
> starting with underscores)?

"Not documented" was my implied answer.

I think you have a valid use case, though, so you could make a feature
request for an official way to hide stack frames on the bugtracker
http://bugs.python.org or the python-ideas mailing list.