Weightless unit tests: the holy grail has been found

[Edward K. Ream]

As of rev 4725, unit testing in leoPy.leo feels *completely*
weightless. The last change: running any unit test externally saves
the .leo file automatically first.

Thus, Alt-4 runs all desired unit tests immediately. To select/
deselect unit tests, just mark/unmark @test nodes.

The difference in workflow is *huge*. There is no need to select
@test nodes in any way, except for marking them.

So I create an @test node, make changes to that node, or to the code
being tested, and hit Alt-4. Done.

When I am satisfied with the @test node, I can leave it where it is
(unmarked) or move it to unitTest.leo.

The energy difference between weightless and heavy is astounding. Try
the new way: you will surely like it.

Edward

[Edward K. Ream]

On Nov 3, 9:17 pm, "Edward K. Ream" <edream...@gmail.com> wrote:
> As of rev 4725, unit testing in leoPy.leo feels *completely*
> weightless.  The last change: running any unit test externally saves
> the .leo file automatically first.

I found that saving unitTest.leo automatically was off-putting, so
running local tests does *not* pre-save the .leo file.

Edward

[Edward K. Ream]

On Nov 3, 9:20 pm, "Edward K. Ream" <edream...@gmail.com> wrote:

> > As of rev 4725, unit testing in leoPy.leo feels *completely*
> > weightless.  The last change: running any unit test externally saves
> > the .leo file automatically first.

As of rev 4729 , the save is done only if the .leo file has changed.

This *tiny* decrease in weight makes a *big* difference when doing
alt-4 repeatedly.

For example, now doing ctrl-s before doing alt-4 doesn't waste *any*
time.

Edward

[Edward K. Ream]

On Nov 4, 10:58 am, "Edward K. Ream" <edream...@gmail.com> wrote:

> As of rev 4729 , the save is done only if the .leo file has changed.
>
> This *tiny* decrease in weight makes a *big* difference when doing
> alt-4 repeatedly.
>
> For example, now doing ctrl-s before doing alt-4 doesn't waste *any* time.

At present, I think of the "weight" as tiny grains of sand in the unit
testing gears.

This "friction" builds up with repeated use.

Rev 4733 removes one of these grains of sand. The run-marked-unit-
tests commands now say::

No marked @test or @suite nodes in the entire outline

rather than

No @test or @suite nodes in the entire outline

This slight change prevents a jarring "huh?" that would otherwise
happen.

I have been looking closely for any more sand--I don't know of any.
If you do know of *any* slight glitch in the run-marked-unit-tests
commands, please let me know. It's important.

Edward

[Edward K. Ream]

On Nov 4, 1:22 pm, "Edward K. Ream" <edream...@gmail.com> wrote:

> I have been looking closely for any more sand--I don't know of any.

Rev 4734 removes support for hidden tests.

Compared to present best practices (Alt-4 everywhere) they are anti-
pattern.

I've just converted the few hidden tests in leoPy.leo to proper @test
nodes, and the result is quite remarkably better.

Leo's users will be well rid of this Easter Egg. One less piece of
techie junk to ignore.

Leo's unit testing machinery is running more quietly and smoothly...

Edward

[Edward K. Ream]

On Nov 4, 3:29 pm, "Edward K. Ream" <edream...@gmail.com> wrote:

> > I have been looking closely for any more sand--I don't know of any.

Just found another bit of grit: there is little or no machinery to
set up config settings in external unit tests.

**Important**: This is far from a gotcha: I'm still in love with
Alt-4 :-)

As a workaround, the unit test that I have been working on all day
today (the test that demonstrates the html whitespace problem) does
the following::

# When run externally, c.config.getData will return None.
html_tags = ('body','head','html','table','xxx',)
setting = 'import_html_tags'
existing_tags = c.config.getData(setting)
if not existing_tags:
g.app.config.set(None,setting,'data',html_tags)
g.app.config.set(None,'full_import_checks','bool',True)

But this is not satisfactory.

Calling g.app.config.set isn't hard: the problem is realizing that the
test does have to set particular settings!

I certainly had no memory of the @bool full_import_checks setting. So
I had to figure out why the external unit test was always passing,
when the local test failed as expected.

Yes, one could define a default setting for full_import_checks, so
that external unit tests will use them, but that only works for
*today's* unit test. It's not a general solution.

Furthermore, it's not obvious what to use for settings in external
unit tests. We can use the existing settings in unitTest.leo, because
we have an @settings node there that can define settings for unit
tests. But outside of unitTest.leo the @settings node isn't going to
be tailored for unit testing!

As I write this, I see that the code that creates dynamicUnitTest.leo
should probably copy the @settings tree from unitTest.leo, plus all
the settings in leoSettings.leo. Yuck. Fully initing settings might
slow down external tests a lot, and that would not be good.

For now, I'll live with this problem, but I think it will have to be
fixed eventually...

Edward

[Edward K. Ream]

On Nov 5, 7:09 pm, "Edward K. Ream" <edream...@gmail.com> wrote:

> Just found another bit of grit:  there is little or no machinery to
> set up config settings in external unit tests.

On second thought, this really isn't such a big deal. It was always
obvious that today's unit test had to simulate @data
'import_html_tags. The problem with @bool full_import_checks can be
solved by overriding the setting when executing unit tests. Doh!

So yes, there can be cases when settings cause problems, but it
shouldn't be that big a deal to hack Leo's code to accommodate extra
checks during unit testing. There is already plenty of unit-test
support code in Leo--a little more won't hurt. That is, provided the
new code doesn't break existing unit tests, but that is very easy to
check.

Edward

[Edward K. Ream]

On Nov 5, 7:16 pm, "Edward K. Ream" <edream...@gmail.com> wrote:
> On Nov 5, 7:09 pm, "Edward K. Ream" <edream...@gmail.com> wrote:
>
> > Just found another bit of grit:  there is little or no machinery to
> > set up config settings in external unit tests.

It may be useful to create a method called setUnitTestSetting, which
would determine whether the test is being run locally or externally,
and take proper action. Easy to write. Usefulness to be determined
by experience.

Edward

[Edward K. Ream]

On Nov 4, 12:22 pm, "Edward K. Ream" <edream...@gmail.com> wrote:

> I have been looking closely for any more sand--I don't know of any.

Running unit tests externally on gui-related code is problematic. I
wouldn't necessarily call this "grit", because it's not a small
problem that could be fixed easily. It's more of a long-term project.

However, there *is* one piece of grit that has just appeared: there
should be some *easy* way to determine whether a test is, in fact,
being run externally. Part of being "easy" is that I have to be able
to remember it easily :-)

As I write this, I don't know how easy it will be--he process of
running external unit tests is baroque. However, it should be
possible *somehow*, and that's all that matters.

Edward

[vitalije]

Maybe it would be simplest to add argument '--test-externally' in
Code-->Testing-->@file leoTest.py-->runUnitTestLeoFile
just before spawning new process, and than check for such option in processing command line's arguments.
Vitalije

[Edward K. Ream]

I considered that, but there was an easier way.

EKR