SB: Status report on the unit testing project

[Edward K. Ream]

Yesterday's Engineering Notebook post was full of details. Today's report will be shorter and less technical.

Work has reached a big milestone. All the new unit tests for leo/commands/editCommands.py now pass. The key was creating a parent node to hold Leo directives. Many commands depend on @pagewidth and @tabwidth directives, so the testing framework must provide a place for them.

Other tests failed because of ludicrous testing hacks in the commands themselves, which individual (new style) tests now work around in various ways. For now, and probably "forever", I'll leave Leo's core as it is so that unitTest.leo will continue to work.

Considerable work remains

1. At present, the unit tests only verify that the various commands change the body text as expected. Tests that the selection range changes as expected fail. This should be straightforward to fix. Similarly, the undo/redo tests should be re-enabled.

2. The new-style tests don't need the "workNode" node. This node is useful when using unitTest.leo, but not in the new testing world.

3. I shall soon create a leo/unittest directory to hold separate testing files. I now see that I don't want to "pollute" Leo's sources with test code:

- Using separate test files is the usual practice.

- The test code adds unnecessary length to the coverage reports and skews coverage statistics.

Important: I don't want to put test files in the existing leo/test directory. That directory has lots of cruft that would complicate test discovery. leo/test will stay exactly as it is.

Summary

All the new unit tests for leo/commands/editCommands.py now pass. Several more days of work remain.

My focus remains on the sabbatical. However, it would be unbearable to leave the new unit testing work unfinished. Yes, this "interrupts" the sabbatical, but I can see no practical psychological alternative.

Leaving leoTest.py unchanged has been the correct decision. I'll deprecate unitTest.leo, but unitTest.leo, and all its support code in leoTest.py, will remain "forever".

Edward

[Edward K. Ream]

On Wednesday, December 2, 2020 at 5:55:52 AM UTC-6 Edward K. Ream wrote:

> I'll deprecate unitTest.leo, but unitTest.leo, and all its support code in leoTest.py, will remain "forever".

When I first announced this project, someone commented that using unitTest.leo doesn't preclude coverage testing. I dismissed the comment at the time, but it's been stewing in the back of my mind.

Yesterday was a long and tiring day. Just after all the tests started to work I started to wonder whether the whole idea wasn't just a foolish sideshow. I know enough not to give much weight to such exhausted thoughts, but the question is worth a second look.

My verdict: The comment is "technically" correct. The present (hairy) code in leoTest.py could run pytest-cov rather than unittest. But there are overwhelming reasons to migrate away from unitTest.leo:

1. The new testing scheme is standard. There is much less for new Leo devs to learn.

2. The new testing scheme is much simpler than the old. leoTest.py is a horror show.  leoTest2.py is straightforward.

3. It is far easier to run the new unit tests from within Leo with @command run-test @key=ctrl-4:

g.cls()
import importlib
import leo.core.leoTest2 as leoTest2
importlib.reload(leoTest2)
import os

args = '' # '-v'
module = '-m leo.commands.editCommands'
test = '' # '-k "test_rectangle_yank"'
output_file = ''  # r'> C:\Users\edreamleo\Desktop\trace.txt'
command = f"python {args} {module} {test} {output_file}"
print(command)
os.system(command)

This is much easier than re-launching unitTest.leo! And it's easy to run similar commands from the command line.

And it's faster. Yes, create_app takes 1 second to create a mock Leo app, but launching unitTest.leo takes a lot longer!

Edward

[Edward K. Ream]

On Wednesday, December 2, 2020 at 5:55:52 AM UTC-6 Edward K. Ream wrote:

> My focus remains on the sabbatical. However, it would be unbearable to leave the new unit testing work unfinished. Yes, this "interrupts" the sabbatical, but I can see no practical psychological alternative.

I want to bring the present frenzied work to a "sustainable" conclusion asap. This means completing only the immediate objectives:

1. Get the new unit tests working for leo.commands.editCommands, including tests for selection range and undo/redo.

2. Move per-file test classes to separate files in the new leo/unittest directory, not to be confused with the existing leo/test/unittest directory.

3. Merge the work into devel.

Those objectives do not include covering all of Leo with new unit tests! Many other important projects beckon.

Edward