Async unittest for "async def" methods?

[László Nagy]

Environment: Python 3, tornado 4.4. The normal unittests cannot be used because methods are asynchronous. There is ttp://www.tornadoweb.org/en/stable/testing.html that explains how to do unit testing for asynchronous code. But that works with tornado coroutines ONLY. The classes I want to test are using the async def statements, and they cannot be tested this way. For example, here is a test case that uses ASyncHTTPClient.fetch and its callback parameter:

class MyTestCase2(AsyncTestCase):
    def test_http_fetch(self):
        client = AsyncHTTPClient(self.io_loop)
        client.fetch("http://www.tornadoweb.org/", self.stop)
        response = self.wait()
        # Test contents of response
        self.assertIn("FriendFeed", response.body)

But my methods are declared like this:

    class Connection:

        async def get_data(url, *args):

            # ....

And there is no callback. There cannot be, because async def methods MUST be awaited for! But I cannot await from inside the test method, because it is not an async method. I can make it an async method (I have tried) but then I get this error:

TypeError: Generator and coroutine test methods should be decorated with tornado.testing.gen_test

raised here: https://github.com/tornadoweb/tornado/blob/master/tornado/testing.py#L138

So how can I test async def methods from unit tests?

(Also asked here: http://stackoverflow.com/questions/42040248/howto-do-unittest-for-tornado-async-def )

[László Nagy]

And BTW if I do decorate that method with gen_test and run this:

python -m tornado.test.runtests

Then I get "Python.exe has stopped working" error (under Windows at least).

[László Nagy]

Ended up using this pattern for now:

import unittest
import tornado.ioloop
from weedstorm.weed import WeedFS


class MyTests(unittest.TestCase):
    async def main(self):
        self.setUp()

        for mname in sorted(dir(self)):
            if mname.startswith("test_"):
                item = getattr(self, mname)
                if callable(item):
                    print("Testing",mname)
                    await item()

        tornado.ioloop.IOLoop.current().stop()


    def setUp(self):
        # Initialize your tests here
        pass

    async def test_01(self):
        pass # You can await here...

    async def test_02(self):
        pass # You can await here...


if __name__ == '__main__':

    io_loop = tornado.ioloop.IOLoop.current()
    io_loop.run_sync(WeedFSTests().main)
    io_loop.start()

But this is a very ugly workaround, because it cannot be used with normal unit test tools.

Is there a better way?

[A. Jesse Jiryu Davis]

Hi Laszlo, copying over my Stack Overflow answer here. You can use a native coroutine as a test function:

import unittest

from tornado.httpclient import AsyncHTTPClient

from tornado.testing import AsyncTestCase, gen_test

class MyTestCase2(AsyncTestCase):

    @gen_test

    async def test_http_fetch(self):

        client = AsyncHTTPClient(self.io_loop)

        response = await client.fetch("http://www.tornadoweb.org/")

        # Test contents of response

        self.assertIn("FriendFeed", response.body.decode())

unittest.main()

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python-tornado+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[László Nagy]

I'm sorry I'll need some more help. Here is the exact MWE that I'm trying now, saved in "test.py":

import unittest

import tornado.gen

from tornado.testing import AsyncTestCase, gen_test

class MyTests(AsyncTestCase):

    @gen_test

    async def test_01(self):

        print("#1")

        self.assertEqual(1+1, 2)

        print("#2")

        

    @gen_test

    async def test_02(self):

        print("#3")

        with self.assertRaises(ValueError):

            print("#4")

            await self.throw_exception()

            print("#7")

    @gen_test

    async def throw_exception(self):

        print("#5")

        await tornado.gen.sleep(10)

        print("#6")

        raise ValueError("Test")

        

if __name__ == '__main__':

    tornado.testing.main()

    

        

Then I tried to run the test this way:

C:\Temp\aaa>py -3 -m  test

E

======================================================================

ERROR: all (unittest.loader._FailedTest)

----------------------------------------------------------------------

AttributeError: module '__main__' has no attribute 'all'

----------------------------------------------------------------------

Ran 1 test in 0.000s

FAILED (errors=1)

[E 170205 09:42:13 testing:731] FAIL

C:\Temp\aaa>

I'm not sure what is wrong. There is no traceback to look at. Otherwise the code looks okay - if I replace the test case with a manually started ioloop then it works as expected:

import tornado.ioloop

import tornado.gen

class MyTests:

    async def test_all(self):

        await self.test_01()

        await self.test_02()

        io_loop.stop()

    

    async def test_01(self):

        print("#1")

        print(1+1==2) # self.assertEqual(1+1, 2)

        print("#2")

        

    async def test_02(self):

        print("#3")

        #with self.assertRaises(ValueError):

        try:

            print("#4")

            await self.throw_exception()

            print("# I SHALL NOT PRINT!")

        except ValueError:

            print("OK")

    async def throw_exception(self):

        print("#5")

        await tornado.gen.sleep(10)

        print("#6")

        raise ValueError("Test")

        

if __name__ == '__main__':

    io_loop = tornado.ioloop.IOLoop.current()

    io_loop.run_sync(MyTests().test_all)

    io_loop.start()

Test run:

C:\Temp\aaa>py -3 -m test2

#1

True

#2

#3

#4

#5

#6

OK

C:\Temp\aaa>

Please help me understand, What am I doing wrong?

[László Nagy]

Okay, finally it works. Some comments below in the code. There is one thing I still don't understand. Why do I need to use unittest.main() instead of tornado.testing.main()? It is not explicitely written in the docs that we HAVE TO use tornado.testing.main, but there is a hint ( http://www.tornadoweb.org/en/stable/testing.html#tornado.testing.main ):

"This test runner is essentially equivalent to unittest.main from the standard library, but adds support for tornado-style option parsing and log formatting."

For this reason, I supposed that I had to use this instead of unittest.main(). But I was wrong, because it raises an AttributeError.

If you also thing that the documentation is not clear enough, then please vote for changing it.

The problem may not be not with the documentation but with my head.

import unittest

import tornado.gen

from tornado.testing import AsyncTestCase, gen_test

class MyTests(AsyncTestCase):

    @gen_test

    async def test_01(self):

        print("#1")

        self.assertEqual(1+1, 2)

        print("#2")

        

    @gen_test

    async def test_02(self):

        print("#3")

        with self.assertRaises(ValueError):

            print("#4")

            await self.throw_exception()

            print("#7")

    # @gen_test - not a test entry point!

    async def throw_exception(self):

        print("#5")

        await tornado.gen.sleep(3) # Increase os.environ["ASYNC_TEST_TIMEOUT"] , default is 5 seconds.

        print("#6")

        raise ValueError("Test")

        

if __name__ == '__main__':

    #tornado.testing.main() # raises "AttributeError: module '__main__' has no attribute 'all'

    unittest.main() # This works, but WHY????

[A. Jesse Jiryu Davis]

I think tornado.testing.main has a small bug related to "all". If your Python file is called SCRIPTNAME.py, you can run your test like this from the command line:

python -m tornado.testing SCRIPTNAME

Or, you can define "all" in your script:

if __name__ == '__main__':

    all = MyTestCase2

    main()  # This is tornado.testing.main

Or you can just use unittest.main() like I demonstrated.

[Ben Darnell]

tornado.testing.main exists because the python 2.6 unittest module had no good way to run all of a projects tests, so it required you to define a function `all()` that would initialize a unittest.TestSuite for the project.

Now, of course, the unittest module is greatly improved, so there is little reason to use tornado.testing.main instead of unittest.main. (The main reason at this point is that tornado.testing.main will call tornado.options.parse_command_line for you if you use tornado.options). This function should probably just be deprecated. 

And in any case, using tornado.testing.main was never a requirement for testing with tornado and AsyncTestCase - the AsyncTestCase classes are self-contained and don't require any particular runner (so you can run them with nose or pytest too).

-Ben

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornad...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.

To unsubscribe from this group and stop receiving emails from it, send an email to python-tornad...@googlegroups.com.

[László Nagy]

2017. február 6., hétfő 5:16:04 UTC+1 időpontban Ben Darnell a következőt írta:

tornado.testing.main exists because the python 2.6 unittest module had no good way to run all of a projects tests, so it required you to define a function `all()` that would initialize a unittest.TestSuite for the project.

Now, of course, the unittest module is greatly improved, so there is little reason to use tornado.testing.main instead of unittest.main. (The main reason at this point is that tornado.testing.main will call tornado.options.parse_command_line for you if you use tornado.options). This function should probably just be deprecated. 

And in any case, using tornado.testing.main was never a requirement for testing with tornado and AsyncTestCase - the AsyncTestCase classes are self-contained and don't require any particular runner (so you can run them with nose or pytest too).

Thanks for the explanation! 

I would change this in the docs:

"This test runner is essentially equivalent to unittest.main from the standard library, but adds support for tornado-style option parsing and log formatting. AsyncTestCase classes are self-contained and don't require tornado.testing.main or any particular runner."

but again, I might not know enough to judge.

Side note: the name was choosen badly. "all" is a built-in function, and owerwritting it makes it impossible to use the built-in one inside the modul being tested.