Unit Testing: Multiple Configurations for the same functions

[Christian Heigele]

Hey everyone, I have the following situation:

I have a framework that interacts with real hardware devices and I want to test this.
Currently I have a list of the devices under test as a TestParameters.

Now I want to run a sub-set of those DUTs more often than the others. Mainly timing reasons, the whole test-suite takes multiple hours to test, and I don't want to check everything with every possible device after each commit in our cvs.

My plan was that I make my former test class abstract, move this dutList parameter into two different classes that inherit from this class, fill them with different information and tag the sub-classes. Then my CI-system could filter based on the tags which tests should be executed.

The problem is now that the functions pick their tag from the file where they are located and not the class they belong to.

E.g. if I have the following situation:

classdef (TestTags = {'BaseTag'}) BaseClass < matlab.unittest.TestCase
methods (Tests)
function something(self)
end
end
end

classdef (TestTags = {'ChildTag'}) ChildClass < BaseClass
methods (Tests)
function somethingElse(self)
end
end
end

If I then run the tests on the ChildClass I get two tests, one for ChildClass / something with only the tag 'BaseTag' and ChildClass / somethingElse with both tags.

Has anyone an idea how I could solve this problem?

Regards,
Christian

[Steven Lord]

"Christian Heigele" wrote in message <o5t88f$99s$1...@newscl01ah.mathworks.com>...

> Hey everyone, I have the following situation:
>
> I have a framework that interacts with real hardware devices and I want to test this.
> Currently I have a list of the devices under test as a TestParameters.
>
> Now I want to run a sub-set of those DUTs more often than the others. Mainly timing reasons, the whole test-suite takes multiple hours to test, and I don't want to check everything with every possible device after each commit in our cvs.

I think using matlab.unittest.TestSuite.selectIf will do what you want.

https://www.mathworks.com/help/matlab/ref/matlab.unittest.testsuite.selectif.html

See the suite stored in the variable s3 in the "Select Test Elements Using Parameterization" example on that page. When you want to run the tests for a device associated with a specific parameter, create the suite then select only the tests associated with that particular parameter value to execute.

*snip*

--
Steve Lord
sl...@mathworks.com
To contact Technical Support, use the Contact Us link at the top of http://www.mathworks.com

[Christian Heigele]

Hi Steven

Cool, thats pretty close.

But currently I store my device informations in a cell array of structs:

classdef MyTestClass < matlab.unittest.TestCase
properties (TestParameter)
dutSetup = {struct('Port', 'Com1', 'Image', 'Ver1'}, struct('Port', 'Com1', 'Image', 'Ver1')};
end
...
end

Then I can filter it by calling
suite.selectIf(~HasParameter('Property', 'dutSetup ', 'Value', SubPropSelector('Port', Com1)));

And SubPropSelector inherits from matlab.unittest.constraints.Constraint and checks if the "actual" elements has those fields with the given value. Thanks for the hint in the right direction.

[Andy Campbell]

Hi Christian,

It sounds like Steve's suggestion has you moving in the right direction here, however note that the test tags are only applied to the base class and all subclass test methods when they are included as a class level attribute (e.g defined in the classdef block).

However, you can restrict your tags to work on only the base class methods by including them in the methods block, something like this:

classdef BaseClass < matlab.unittest.TestCase
methods (Tests, TestTags = {'BaseTag'})

function something(self)
end
end
end

classdef (TestTags = {'ChildTag'}) ChildClass < BaseClass
methods (Tests)
function somethingElse(self)
end
end
end

Then, 'BaseTag' will only apply to the "something" test method (as well as any other method defined in that methods block) and not "somethingElse".

HTH,
Andy

"Christian Heigele" wrote in message <o5t88f$99s$1...@newscl01ah.mathworks.com>...