Unit testing of tasks

[Kristoffer Gryte]

Hi all,

I have some basic knowledge of the testing framework in DUNE, and have used it a little bit for tests of my own libraries etc. Is there a way to run tests in a similar manner on entire tasks? I do not have intricate knowledge of ROS, but I have understood that they have a way to test both on a library level, on a task level, and on an inter-task level (see http://wiki.ros.org/action/show/Quality/Tutorials/UnitTesting). Does there exist some way to run task-level and inter-task-level tests today? Has there been any discussions about this? 

From my understanding at this point, the first, and possibly very stupid, way that comes to mind to add this functionality to DUNE would be to have a testing framework that would run a bare-minimum dune, with only the task(s) needed for the test. Then the developer could write some testing tasks (that ideally are not compiled into the dune packages) for passing test data and addressing the correctness of the response IMC messages from the task being tested. 

I do realize that it is of course possible to run a full instance of dune, with the task you would like to test, and pass IMC messages from e.g. the neptus interface, but it would be nice to be able to set up automated tests.

Regards,

Kristoffer

[José Pinto]

Hi Kristoffer,

I think one could use the lsf-replay program as a base to this (mocking) test framework.

The test cases would have: 

  1) DUNE configuration file (ini with needed tasks and settings, including the mocking task)

  2) Timed messages

  3) Expected messages

Would something like this fit most of your unit testing needs?

Regards,

José

--

http://www.lsts.pt | http://www.fe.up.pt

--
www.github.com/LSTS/
www.lsts.pt
---
You received this message because you are subscribed to the Google Groups "LSTS Toolchain" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lsts-toolchai...@googlegroups.com.
Visit this group at https://groups.google.com/group/lsts-toolchain.
To view this discussion on the web visit https://groups.google.com/d/msgid/lsts-toolchain/fcde5a4c-5af6-4870-a710-ee6da386a476%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Morten Fyhn Amundsen]

I have some experience with ROS node testing, and I think a similar approach should work. In ROS, your test works like a node/task, and you just program the test task to dispatch messages to the task you're testing. Then you consume messages output by the tested task and run your assertions on that. You'll typically want to be able to run several separate test scenarios from one test file, so that each scenario gets a freshly started instance of the task you're testing.

[Kristoffer Gryte]

Hi Jose,

Just to see if I understand you correctly; 

The test program would to the following:

• start the lsf-replay program (or a derivative thereof) which would send both the "raw" messages and the expected results.
• start a minimal instance of dune from a ini-file, to run the task being tested (which consumes the raw messages) (and possibly a task for verification (which consumes the expected results))
• if possible: consume the results from the task being tested and compare these results with the expected. This would remove the need for the verification task above, which will both separate the tests from the code and integrate the task testing better with the existing tests (print out passed/failed in a similar manner). I'm a bit unsure how this could be done, but including the messages in question in Transports.UDP and have the test program listen for UDP sounds like it should be doable.

Was this what you had in mind? I I think this will satisfy our needs :)

lørdag 5. mai 2018 10.33.19 UTC+2 skrev José Pinto følgende: