ENB: unit testing for leoserver.py

[Edward K. Ream]

This Engineering Notebook post shows the workflows I use to unit tests for leoserver.py. I won't say much about the unit tests themselves. If you are interested in the details, see the TestLeoServer class in leoserver.py and the _get_action_list function in leoserver.py.

The big Aha

Unit tests must run in the server. Running unit tests from the client doesn't work. Just run unit tests on leoserver.py, as with any other python file.

No, such tests won't test the actual communication code. For that we need to run leoclient.py.  I use # pragma: no cover to tell coverage.py that we don't need to cover comm-related code.

Three driver scripts

Whenever I change leoserver.py or leoclient.py, I run three tests, bound to Ctrl-4, Ctrl-5, and Ctrl-6. I run them in this order until all tests pass and coverage is 100%.

@command run-server @key=Ctrl-4

g.cls()
import time
if c.changed:
    c.save()
g.execute_shell_commands('start cmd /k "python leoserver.py"')
time.sleep(0.5)

g.execute_shell_commands('start cmd /k "python leoclient.py"')

This Leo script runs the client and server in two processes, each distinct from Leo. This script does test the communication between client and server. I always run this script first.

The commands shown above work only on Windows.  If you like, you can easily modify them to work on other platforms.

@command unit-test @key=Ctrl-5

"""Run unittest on leoserver.py: good for debugging."""
g.cls()
import os
if c.changed:
    c.save()
prolog = "python -m pytest --cov-report html --cov-report term-missing --cov "
os.chdir(os.path.join(g.app.loadDir, '..', '..'))
# leoserver.main detects and removes the --unittest command-line arg.
g.execute_shell_commands('python -m leo.core.leoserver --unittest')

Next, I run this script. The script runs all unit tests in the TestLeoServer class using python's unittest module. I prefer to do this before running coverage tests so I can see all output.

@command py-cov @key=Ctrl-6

"""Run py-cov on leoserver.py"""
g.cls()
import os
if c.changed:
    c.save()
os.chdir(os.path.join(g.app.loadDir, '..', '..'))
prolog = "python -m pytest --cov-report html --cov-report term-missing --cov "
command = prolog + r"leo.core.leoserver leo\core\leoserver.py"
g.execute_shell_commands(command)

Finally, I run this script. This script runs all unit tests, generating a coverage report. An example is attached.

Summary

I do the following whenever I change leoserver.py or leoclient.py:

1. If necessary, I add a "fake" unit test in the _get_action_list function in leoclient.py. I may also add a unit test to the TestLeoServer

2. Do Ctrl-4 to test the communication between client and server. The unit tests in leoserver.py do not (and can not) test the comm code directly.

3. Do Ctrl-5 to run all unit tests with unittest.main(). All traces are visible.

4. Do Ctrl-6 to run coverage tests with pytest.

And that's it!  All questions and comments are welcome.

Edward

[Viktor Ransmayr]

Hello Edward,

Edward K. Ream schrieb am Samstag, 13. Februar 2021 um 13:58:26 UTC+1:

This Engineering Notebook post shows the workflows I use to unit tests for leoserver.py. I won't say much about the unit tests themselves. If you are interested in the details, see the TestLeoServer class in leoserver.py and the _get_action_list function in leoserver.py.

The big Aha

Unit tests must run in the server. Running unit tests from the client doesn't work. Just run unit tests on leoserver.py, as with any other python file.

No, such tests won't test the actual communication code. For that we need to run leoclient.py.  I use # pragma: no cover to tell coverage.py that we don't need to cover comm-related code.

Three driver scripts

Whenever I change leoserver.py or leoclient.py, I run three tests, bound to Ctrl-4, Ctrl-5, and Ctrl-6. I run them in this order until all tests pass and coverage is 100%.

@command run-server @key=Ctrl-4

g.cls()
import time
if c.changed:
    c.save()
g.execute_shell_commands('start cmd /k "python leoserver.py"')
time.sleep(0.5)

g.execute_shell_commands('start cmd /k "python leoclient.py"')

This Leo script runs the client and server in two processes, each distinct from Leo. This script does test the communication between client and server. I always run this script first.

The commands shown above work only on Windows.  If you like, you can easily modify them to work on other platforms.

Is there a specific reason, why you make it Windows specific here - and - don't use the same generic variant, that you use below for '@command unit-test ...'?

I did not try it out myself yet - but -

g.cls()
import time
if c.changed:
    c.save()

g.execute_shell_commands('python -m leo.core.leoserver')

time.sleep(0.5)

g.execute_shell_commands('python -m leo.core.leoclient')

should work in a portable way ...

@command unit-test @key=Ctrl-5

"""Run unittest on leoserver.py: good for debugging."""
g.cls()
import os
if c.changed:
    c.save()
prolog = "python -m pytest --cov-report html --cov-report term-missing --cov "
os.chdir(os.path.join(g.app.loadDir, '..', '..'))
# leoserver.main detects and removes the --unittest command-line arg.
g.execute_shell_commands('python -m leo.core.leoserver --unittest')

Next, I run this script. The script runs all unit tests in the TestLeoServer class using python's unittest module. I prefer to do this before running coverage tests so I can see all output.

With kind regards,

Viktor

[Edward K. Ream]

On Mon, Feb 15, 2021 at 3:30 PM Viktor Ransmayr <viktor....@gmail.com> wrote:

> Is there a specific reason, why you make it Windows specific here - and - don't use the same generic variant, that you use below for '@command unit-test ...'?

Yes. The script runs the two processes in separate consoles.

Edward

[Viktor Ransmayr]

Thanks a lot for your answer. - I should have invested a bit more time to run / test the proposed 'improvement' before in a Linux VM on my side ...

With kind regards,

Viktor

[Edward K. Ream]

On Tue, Feb 16, 2021 at 2:24 PM Viktor Ransmayr <viktor....@gmail.com> wrote:

Thanks a lot for your answer.

You're welcome. It was a reasonable question.

Edward