Memory leak when a test fails

41 views
Skip to first unread message

Johannes Wilken

unread,
Oct 9, 2015, 2:37:06 AM10/9/15
to nose-users
Hello everyone,

i was observing an increase in memory usage when my test are failing and after a bit of fumbling around, i decided to setup a minimal example:

from nose.tools import ok_
import unittest
import os

import nose
import time
import gc
import sys

class TC_MemLeak(unittest.TestCase):
   
    def tearDown(self):
        gc.collect()
        unittest.TestCase.tearDown(self)
   
    @staticmethod
    def yield_leak(num):
        list1 = [9876543210] * 2048*2048*3
        time.sleep(1)
       
        ok_(False, "Failed")
       
    @staticmethod
    def yield_no_leak(num):
        list1 = [9876543210] * 2048*2048*3
        time.sleep(1)
       
        ok_(True, "Failed")

class TC_MemLeak_Gen(object):
   
    def test_method(self):
       
        for i in range(50):
            yield TC_MemLeak.yield_leak, i
           
if __name__ == "__main__":
    print(sys.version)
    print (nose.__version__)
    os.environ["NOSE_TESTMATCH"] = "(?:^|[b_./-])(TC|[Tt]est)"
    nose.runmodule()


When the generated test method fails, the memory allocated during that function is seemingly not freed. When the test passes, everything is fine. The example script above does nothing big or fancy, it just creates a rather huge list to show this behaviour.
In my own tests, each failed method leaks about 20MB, but as the generator yields about 300 test methods, this can become quite critical.
The only workaround I found so far is to call "del" on each allocated object explicitly, which is not really nice.

My current setup is nose version 1.3.4 and python 3.3.5 64bit. Due to other circumstances it would not be simple for me to switch to nose2, but if somebody can confirm that nose2 shows a different behaviour here, I'm willing to try and migrate everything.

So, my general questions are:
Can somebody here verify this behaviour?
Is it intended behaviour?
Is there a better workaround than deleting everything explicitly?


Johannes Wilken

unread,
Oct 9, 2015, 4:00:30 AM10/9/15
to nose-users
//EDIT:
The reason is not nose, so basically I'm wrong here, but maybe somebody has an idea. I wrote a basic test class for unittest where each test method again creates a large list and fails. These method do not free their memory if they fail. So much for further investigation up to now

John T

unread,
Oct 9, 2015, 8:07:41 PM10/9/15
to nose-...@googlegroups.com
When you have an error or failure the test and err tuple is appended to a list in the Test Result. The 3rd element of that tuple is the traceback object which references the location where the exception occurred. Unfortunately if you drill down the frames you'll find references to the local variables in the test function.

err[2].tb_frame.tb_next...tb_next.tb_frame.__internals__.f_locals


    def addFailure(self, test, err):
        """Called when an error has occurred. 'err' is a tuple of values as
        returned by sys.exc_info()."""
        self.failures.append((test, err))  <--- here is where the tb object gets appended
        self.printLabel('FAIL', err)


Solution?
1. Try to hide the fat locals from the traceback. Obviously this might not work since data and test logic usually live in the same context. 

        def x():
            list1 = [9876543210] * 2048*2048*3
        x()
        raise

2. Monkey patch Test Result to not store tracebacks (if you don't care about them).


I hope that helps.

-J

--
You received this message because you are subscribed to the Google Groups "nose-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nose-users+...@googlegroups.com.
To post to this group, send email to nose-...@googlegroups.com.
Visit this group at http://groups.google.com/group/nose-users.
For more options, visit https://groups.google.com/d/optout.

Johannes Wilken

unread,
Oct 13, 2015, 5:02:13 AM10/13/15
to nose-users
Thanks for the help!

I'm sorry for my late reply, I tried to make the changes you suggested, but it seems like I slipped up somewhere.

I dug a bit into the plugin API of nose to create my own TestResult and TestRunner. Both are pretty slim:



import nose.core
import logging

class MyTestResult(nose.core.TextTestResult):
   
   
   
def addError(self, test, err):
        logging
.debug("Removing Stackframe")
        tp
, vl, tb = err
       
       
del tb
       
       
super(MyTestResult, self).addError(test, (tp,vl,None))
       
   
def addFailure(self, test, err):
        logging
.debug("Removing Stackframe")
        tp
, vl, tb = err
       
       
del tb
         
       
super(MyTestResult, self).addFailure(test, (tp, vl, None))

import nose.core
import MyTestResult
import logging

class MyTestRunner(nose.core.TextTestRunner):
    resultclass
= MyTestResult

   
def _makeResult(self):
        logging
.debug("Using MyTestResult")
       
return MyTestResult(self.stream, self.descriptions, self.verbosity, self.config)
   
   
def run(self, test):
        logging
.debug("Running test using MyTestRunner")
       
return nose.core.TextTestRunner.run(self,test)


Now, the good news is, the logging messages are printed correctly. So it seems like my classes are being used. The bad news is, it didn't help, the memory still keeps on growing. So, obviously, I missed something or made an error, but from all I know, this should work.
Could it be that passing None as traceback causes nose or unittest to do something internally? However, I don't see a traceback, as expected. Are there any other places which may hold a reference to these variables?

John T

unread,
Oct 13, 2015, 4:30:01 PM10/13/15
to nose-...@googlegroups.com
My guess is the old code is still being run somehow. You can raise an exception in addFailure() and see if you get a traceback.

Also, take a look at prepareTestResult(), prepareTestLoader(), prepareTestRunner() plugin methods.

-J
Reply all
Reply to author
Forward
0 new messages