best practice for test driven development leo script?

[Xu Wang]

Dear Leo Developer,

I am writing a script in Leo, it is located in the body of a @button node.

As adding more and more logic to the script, I feel I need to have some test case to make me confident on the development progress.

As the moment, I am adding test case as below:

def func1():

...

def func2():

...

def main():

.....

def test_scenario1():

func1()

func2()

def test_scenario2():

func2()

func1()

def test_main():

#test_scenario1()

test_scenario2()

c.redraw()

test_main()

#main()

I created a test_main() and put the test logic there. Using ctrl-b to see the result of test_main().

It's not a good way:

1) The test cases and production code are mixed

2) Need manually disable/enable for different test scenarios.

I'm thinking to use python unittest library, but the way I use unittest before was in command line, create a dedicated "test_button_body_code.py" and "import button_body_code" to test button_body_code.py.

How shall I do the similar thing for leo @button script?

Or any other best practice for testing the script in the body of @button?

Best Regards,

Austin

[Edward K. Ream]

​On Thu, Feb 27, 2020 at 8:17 PM Xu Wang <xu4...@gmail.com> wrote:

I am writing a script in Leo, it is located in the body of a @button node.

As adding more and more logic to the script, I feel I need to have some test case to make me confident on the development progress.

​[snip]​

I created a test_main() and put the test logic there. Using ctrl-b to see the result of test_main().

It's not a good way:

1) The test cases and production code are mixed

2) Need manually disable/enable for different test scenarios.

I'm thinking to use python unittest library, but the way I use unittest before was in command line, create a dedicated "test_button_body_code.py" and "import button_body_code" to test button_body_code.py.

How shall I do the similar thing for leo @button script?

Or any other best practice for testing the script in the body of @button?

​It's a good question. Here are some ideas.

1. Do everything in the @button node, and its children.

Complex @button node probably are best written using a controller class.  The pattern in the top-level @button node would be something like:

<< imports >>
class Controller:
    @others
Controller(c).run()

You could extend this pattern by using a Test class, a subclass of unittest.TestCase.  A single node can't use more than one @others directive, so the top-level pattern would become:

<< imports >>
@others  # define all classes
Controller(c).run()

The @button script could the run the Test class using functions in the unittest class.

2. Use a helper .py file

Your script could import the file, say x.py.  Something like this.

<< imports >>
from x import Controller
Controller(c).run()

Here is the @test button I used to test leoAst.py:

g.cls()
import os
leo_editor_dir = os.path.join(g.app.loadDir, '..', '..')
os.chdir(leo_editor_dir) # must be done here.
file_ = 'leo.core.leoAst'
if 1: # Run all tests
    commands = f"&python -m unittest {file_}"
    g.execute_shell_commands(commands, trace=False)
else:
    file_ = 'leo.core.leoAst'
    class_, test = '.TestOrange', '.test_blank_lines_after_function'
    commands = f"&python -m unittest {file_}{class_}{test}"
    g.execute_shell_commands(commands, trace=False)

This works very well.   Furthermore, if x.py ends with:

if __name__ == '__main__':
    unittest.main()

everything will be set up to run pytest coverage tests.

I used the following .bat file to run coverage tests:

pytest --cov-report html --cov-report term-missing --cov leo.core.leoAst leo/core/leoAst.py

HTH.

Edward

[Xu Wang]

Thanks Edward for the advice. I will try it later.  For now, I'm starting from the basic one documented in unittest.leo

1) created one script-button with only one line python code "c.testManager.doTests(all=False)" , it's appears on the top right corner of screen, as expected.

2) create one node  "@test first test" with a "assert(True)", click to focus it, click "Do-@test" button.  I saw 

----------------------------------------------------------------------

Ran 1 test in 0.008s

OK

in the console, as expected.

3) create another one "@test 2nd test" with below python code:

text = p.h

assert(text == "@test 2nd test")

This is to make sure the code being tested can access current position p.  focus on "@test 2nd test", "Do-@test", it passed!

4) click the parent node of "@test first test" and "@test 2nd test", which is "unnittest-leo" on the screenshot attached, click "Do-@test". I got:

..

----------------------------------------------------------------------

Ran 2 tests in 0.017s

OK

It's NOT as expected, the second test case should fail since p.h should be  "unnittest-leo".

The moment when I click "Do-@test", the current position is "unittest-leo" since I was trying to run two test cases together.

Is there something need to be taking care when accessing p in the test code?

BR,Austin

Edward K. Ream <edre...@gmail.com> 于2020年2月28日周五 下午6:39写道：

--
You received this message because you are subscribed to the Google Groups "leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email to leo-editor+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/leo-editor/60f6e1c5-1023-4ae2-89e0-9c309baeea8d%40googlegroups.com.

[Edward K. Ream]

On Fri, Feb 28, 2020 at 9:31 PM Xu Wang <xu4...@gmail.com> wrote:

Thanks Edward for the advice. I will try it later.  For now, I'm starting from the basic one documented in unittest.leo

That's fine. It will be more convenient for tests that involve outlines.

Edward

[Xu Wang]

is the issue I raised in my previous email a Leo bug regarding accessing position in the testcase?

BR,Austin

Edward K. Ream <edre...@gmail.com> 于2020年2月29日周六 下午8:17写道：

--

You received this message because you are subscribed to the Google Groups "leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email to leo-editor+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/leo-editor/CAMF8tS1ofKwjoE%2B%2BL6B__nk3j%2BSPbKBvosyWBuTpi-Bh0E5C_g%40mail.gmail.com.

[Edward K. Ream]

On Sat, Feb 29, 2020 at 7:56 AM Xu Wang <xu4...@gmail.com> wrote:

is the issue I raised in my previous email a Leo bug regarding accessing position in the testcase?

No, I would not call a bug. I simply meant that it's a lot easier to create outlines in unitTest.leo.

Within a Test class, called by g.run_unit_test_in_separate_process, there will always be difficulties creating outlines, traversing them, etc. These difficulties can be overcome, it just takes more work.

Edward

[Xu Wang]

No, I would not call a bug. I simply meant that it's a lot easier to create outlines in unitTest.leo.

Within a Test class, called by g.run_unit_test_in_separate_process, there will always be difficulties creating outlines, traversing them, etc. These difficulties can be overcome, it just takes more work.

Understood.

I tried to use run-all-unit-tests-locally command to execute the two test cases, still got same result. The position read by the test case code is not the real current position.

 

Edward

[Edward K. Ream]

On Sat, Feb 29, 2020 at 12:32 PM Xu Wang <xu4...@gmail.com> wrote:

> I tried to use run-all-unit-tests-locally command to execute the two test cases, still got same result. The position read by the test case code is not the real current position.

Are you sure? I've never had this kind of trouble.

Please show the code that appears to have this problem.

Edward

[Xu Wang]

Strange...please find attached a mini outline file to reproduce.

Click node  "simple test cases collection"  and then "Do-@test" button.  

Below is the output on my console.

Start of Do @test

..

----------------------------------------------------------------------

Ran 2 tests in 0.013s

OK

End of Do @test

Edward K. Ream <edre...@gmail.com> 于2020年3月1日周日 下午6:56写道：

--

You received this message because you are subscribed to the Google Groups "leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email to leo-editor+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/leo-editor/CAMF8tS3vWN2_BuP8KSau0pnFnTMyu%3D%3DaH4vTmOBOoS12QOidsg%40mail.gmail.com.

[Xu Wang]

same result for minibuffer command: run-all-unit-tests-locally

Xu Wang <xu4...@gmail.com> 于2020年3月1日周日 下午8:20写道：

[Edward K. Ream]

On Sun, Mar 1, 2020 at 7:20 AM Xu Wang <xu4...@gmail.com> wrote:

Strange...please find attached a mini outline file to reproduce.

Click node  "simple test cases collection"  and then "Do-@test" button.  

Below is the output on my console.

Everything seems fine to me. Both tests pass, as I think they should.

Within an @test node, p is the position of the @test node itself.

What do you think the problem is?

Edward

[Xu Wang]

Everything seems fine to me. Both tests pass, as I think they should.

Within an @test node, p is the position of the @test node itself.

What do you think the problem is?

I clicked "simple test cases collection" node first, so the p.h should be "simple test cases collection" when running the test.

The test case at node "@test 2nd test" should failed.

text = p.h

assert(text == "@test 2nd test")

"simple test cases collection"  !=  "@test 2nd test"

[Thomas Passin]

Maybe you overlooked it because it scrolled way off the screen, but the output of your test outline, when I ran it, contained this:

  File "D:\Tom\git\leo-editor\leo\core\leoNodes.py", line 1353, in copy
    return Position(self.v, self._childIndex, self.stack)
RecursionError: maximum recursion depth exceeded

I omitted the rest of the stack trace.

On Sunday, March 1, 2020 at 8:20:34 AM UTC-5, Austin(Xu) Wang wrote:

Strange...please find attached a mini outline file to reproduce.

Click node  "simple test cases collection"  and then "Do-@test" button.  

Below is the output on my console.

Start of Do @test

..

----------------------------------------------------------------------

Ran 2 tests in 0.013s

OK

End of Do @test

Edward K. Ream <edre...@gmail.com> 于2020年3月1日周日 下午6:56写道：

On Sat, Feb 29, 2020 at 12:32 PM Xu Wang <xu4...@gmail.com> wrote:

> I tried to use run-all-unit-tests-locally command to execute the two test cases, still got same result. The position read by the test case code is not the real current position.

Are you sure? I've never had this kind of trouble.

Please show the code that appears to have this problem.

Edward

--
You received this message because you are subscribed to the Google Groups "leo-editor" group.

To unsubscribe from this group and stop receiving emails from it, send an email to leo-e...@googlegroups.com.

[Xu Wang]

the two test cases were all passed when I (and Edward) execute them. 

The second one in my opinion should fail.

Thomas Passin <tbp1...@gmail.com> 于 2020年3月1日周日 下午11:42写道：

To unsubscribe from this group and stop receiving emails from it, send an email to leo-editor+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/leo-editor/1ad68899-ae32-4ad5-9a60-2a6ad72026e7%40googlegroups.com.

[Edward K. Ream]

On Sun, Mar 1, 2020 at 7:40 AM Xu Wang <xu4...@gmail.com> wrote:

>> What do you think the problem is?

> I clicked "simple test cases collection" node first, so the p.h should be "simple test cases collection" when running the test.

That's not how @test nodes work. For each @test node, p is bound to the position of the @test node itself. There is no way (and no need) to know the position of the node that was originally selected.  Clear?

Edward

[Xu Wang]

ah... Ok

Edward K. Ream <edre...@gmail.com> 于 2020年3月2日周一 下午6:41写道：

--

You received this message because you are subscribed to the Google Groups "leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email to leo-editor+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/leo-editor/CAMF8tS3GdSs6UD409MswrvpS5LHUsdF43DSaEwNM_ddiAvzt%3DQ%40mail.gmail.com.