Status report: new unit testing

[Edward K. Ream]

Work in the ekr-unit-test is complete. For now, that is.

Aha: get out of the way!

Preliminary work actively interfered with the normal operation of unittest and pytest. I have done the following:

- Removed the `if __name__ == "__main__":` clauses that called leo_test_main, and leo_test_main itself.  leo_test_main tried to determine how to run unit tests, but this is wrong! unittest and pytest are designed to be run from the command line. They don't need help.

- Removed the RunAllLeoTests class and the 'run-all-tests' command. Again, unittest and pytest already do this, and more flexibly than my code.

- Renamed the 'unittest' directory to 'unittests'. This prevents confusion between the directory and the unittest module itself.

- Removed the Utils class in leoTest2.py. Instead, leoTest2.py now contains a top-level compareOutlines function, and a helper function.

- All test classes are now subclasses of unittest.TestCase, not some silly base class.

Understanding pytest --cov

I finally understand what the arguments to pytest --cov, and where the coverage reports go.  I use this script to run coverage tests on leo/commands/leoCommands.py:

cd C:\leo.repo\leo-editor
pytest --cov=leo.commands.editCommands --cov-report=html --cov-report=term-missing leo\unittests\commands\test_editCommands.py

The --cov argument is the name of the module to be tested. It can be given as shown regardless of the directory from which pytest is run.

The last argument is the name of the test file.

The coverage report appears in the current working directory:

    C:\leo.repo\leo-editor\htmlcov

Running tests from Leo

The following @command node runs the 'leo-unittest' script shown above:

import os
g.cls()
cwd = os.getcwd()
os.system('leo-unittest')
os.chdir(cwd)

At a stopping point

I am at a stopping point for #1758.

I could strengthen the unit tests in EditCommandsTest.run_test right away, but there is no great need to do this. Indeed, strengthening the tests will not change the coverage of the tests, so nothing would be gained immediately.

Summary

leoTest2.py has collapsed in complexity. There is no need to "help" unittest or pytest.

It's easy to run unit tests from the command line (with a script), or within Leo using an @command node that invokes the same script.

Lots of work remains on #1758, but that work can (and should) be done much later, after the sabbatical.

Edward

[Edward K. Ream]

On Thursday, December 3, 2020 at 7:30:09 AM UTC-6 Edward K. Ream wrote:

> Work in the ekr-unit-test is complete.

I'll merge ekr-unit-test into devel later today.

> leoTest2.py has collapsed in complexity.

Almost nothing remains in leoTest2.py!

- leoTest2.py contains only a few top-level functions. Only the create_app function is essential.

- leoTest2.py also contains two classes that convert tests in unitTest.leo to nodes containing normal unittest.TestCase code. These nodes get moved (by hand) to classes in leo/unittests. After that, the classes are served their purpose. Eventually, these conversion classes will migrate to scripts.leo.

Summary

Heh. It has taken several days of intense effort to free myself of the mindset of the old unit testing world. Python's unit test tools don't need any help! At last, leoTest2.py has gotten out of the way.

Edward