[erlang-questions] Common Test Question

[Jordan West]

I've recently been using common_test on several new projects and am really liking it. I ran across something today I found odd, however, and wanted to know if it was the expected behavior.  I was expecting that if I have a group with no tests that group should, essentially, be ignored. This seems to be the case if the group is contained in another but not if the group is an element in the list returned by all/0. In the latter case the suite errors out with "Error in suite detected: 'Invalid reference to group [GROUP NAME] in [SUITE NAME]:all/0'". 

Is this intentional and if so, why are sub-groups treated differently? Is there a reason its not a warning instead? For the most part I'm just curious. I do like setting up my groups sometimes ahead of writing tests and ran into this when I tried to leave a top-level group empty for a short while.

A minimal code example is below. I'm using R14B02 on OS-X with common_test 1.5.3 (I installed today off the Erlang Factory USB stick, thanks for a great conference everyone). Oh, and this is my first message to the list. Hello, all.

%% a.erl

-module(a).

-compile(export_all).

my_fun() ->

    1.

%% a_SUITE.erl

-module(a_SUITE).

-compile(export_all).

groups() ->

    [{group_1,

      [shuffle],

      [test_1, {group, group_3}]},

     {group_2,

      [shuffle],

      []}, % this empty test list causes issues if empty

     {group_3,

      [],[]}]. % this empty test list seems to be able to be empty

all() ->

    [

     {group, group_1},

     {group, group_2} %% If this is remove the suite won't error out

    ].

test_1(_) ->

    1 = a:my_fun().

>>  run_test -suite a_SUITE.erl 

...

Error in suite detected: 'Invalid reference to group group_2 in a_SUITE:all/0'

Jordan West

[Peter Andersson]

Hi Jordan,

I learned about this problem just the other day. I consider it a bug
really. An empty group should be ignored, or possibly, the config
functions for the group should be executed (even if there's no test
case to configure for). I have written a ticket on it and a fix will
come shortly.

Thanks for reporting!

Best regards,
Peter

Ericsson AB, Erlang/OTP

2011/3/29 Jordan West <jw...@makingfun.com>:

> _______________________________________________
> erlang-questions mailing list
> erlang-q...@erlang.org
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Jordan West]

Peter,

Thanks for letting me know. I considered it a bug as well. On the topic of the config functions, I think other testing frameworks like Ruby's RSpec ignore them when no tests in the group exist. I assume this is to prevent hard to debug side-effects in tests and as a small optimization.

Jordan West

[Peter Andersson]

Thanks for the input Jordan. I have been trying to come up with a good
reason for letting init- and end_per_group execute even for an empty
group, but can't really see that being a useful feature. So I will
probably let Common Test ignore it, like you suggest.

Best,