@test and @suite are deprecated, and will go away in Leo 6.5

[Edward K. Ream]

`@test` and `@suite` nodes rank as one of the worst ideas I have ever had. They interfere with coverage tests while adding almost no useful features. The support code in leoTest.py is wretchedly complex.

I have recently removed most mentions of `@test` and `@suite` nodes from Leo's documentation. In particular, Leo's tutorials and cheat sheet no longer mention `@test` and `@suite`.

#1766 will be one of the defining features of Leo 6.5. This issue suggests eliminating unitTest.leo and leoTest.py, thereby eliminating support for `@test` and `@suite` nodes.

All comments welcome.

Edward

[Edward K. Ream]

On Monday, August 23, 2021 at 12:58:38 PM UTC-5 Edward K. Ream wrote:

`@test` and `@suite` nodes rank as one of the worst ideas I have ever had. They interfere with coverage tests...The support code in leoTest.py is wretchedly complex.

I am rewriting all of Leo's unit tests in the ekr-unit-tests branch. This work can't be part of Leo 6.4. In any case, the work will take at least several weeks.

The more I work on this project, the more revolted I am by @test and @suite. Moving to typical unit tests will eliminate multiple layers of cruft and complexity.  On a more positive note, the new scheme will have the following benefits:

- Test files are separate from the code under test. This separation simplifies coverage testing.

- leoTest.py will completely disappear, as will support for @test and @suite in the mod_scripting plugin.

- unitTest.leo will go away.

- Most (all?) of the test files in leo/test will disappear.

At present, the new test files are in the leo/unittests subdirectory. Eventually, these tests are likely to migrate to the cleaned-out leo/test directory.

Summary

@test nodes, @suite nodes, and unitTest.leo (including several horrendous hacks therein) made Leo look bad. Moving to a proper unit testing scheme will simplify Leo and make full coverage testing possible.

Edward

[Edward K. Ream]

On Monday, August 23, 2021 at 12:58:38 PM UTC-5 Edward K. Ream wrote:

> `@test` and `@suite` nodes rank as one of the worst ideas I have ever had.

I have been converting tests for the last several days. Completing this project will take about a week. To repeat, the new unit tests will be part of Leo 6.5, not 6.4.

I am astounded at how easy the new (proper) unit tests are. Unlike with @test and @suite nodes, there is no need to ensure that tests don't change unitTest.leo. Avoiding teardown issues drastically simplifies most tests.  unittest.assertEqual typically suffices to show test failures--no need for bespoke tracing code.

Each unit test runs in a pristine (almost) environment. I say "almost" because setUpClass creates a single instance of Leo (the LeoApp class, shared by all tests of a particular test file.

Each unit test gets a pristine new outline. Some subclasses of LeoUnitTest create more complex test outlines for each test. Just a few lines of code in a test suffice to create any subtrees needed by a particular test.

Finally, there is no need to reload Leo to run tests. Running `python -m unittest` always uses the very latest code. Avoiding reload issues is a huge advantage. No need for "prefix" code or explicit calls to importlib.reload!!!

Summary

I created @test and @suite to make Leo outlines (and Leo itself!) available to all unit tests. But the LeoUnitTest class shows how easy is it to recreate Leo's total environment from proper unit tests.

Tests in @test and @suite nodes are much harder to write and maintain than proper unit tests. The work I am doing will be the foundation of a new era in unit testing for Leo.

Edward