[erlang-questions] what is the best way to test Erlang FSM processes?

[Roman Shestakov]

hello,

I have a dependency tree of connected fsm processes, the state of children fsm'
depends on state of parents and in some cases, timers attached to fsm can also
trigger state change.

I would like to be able to test the state of entire system with Common Test but
can't figure out a good way to use CT for testing fsm states.

does anybody have some ideas if it is possible to use CT in such scenario? any
examples? other ideas?

Regards, Roman

[Peter Andersson]

Hi Roman,

This is an interesting question. Actually, a while back we had some
discussions about this with some people using Common Test in a project
at Ericsson. They were testing event driven state machines and they were
forced to implement difficult recursive test case functions with nested
receive and case expressions to test these FSMs.

The idea these guys had was to implement test suites as gen_fsm
behaviours to test their FSMs instead. This way, the suites become much
better structured, more intuitive to read, and easier to modify and extend.

We've decided to implement support for this type of suite in Common
Test, so that's in the pipeline. We haven't decided exactly what this
support should look like, but we have ideas along the lines of:

- let Common Test recognize an fsm suite and handle start/stop of the
test fsm process automatically (per suite or test case)
- a test is executed in the scope of a test case, as usual, and the
test case function name will be a parameter in the state of the test fsm
process
- print info about state transitions, events and messages during a
test to the test case log
- indicate test case pass or fail by means of state transitions and/or
reports to the test case process
- provide i/f to modify the state of the test fsm from the init/end
config functions

We don't really add any new functionality to what the gen_fsm already
provides. What we want you to be able to do, you can already do with a
gen_fsm behaviour test suite today! We will let Common Test take care of
common operations though, to make your test fsm implementations simpler
and more compact. Also we want the users to "have to" structure the fsm
suites the same way, and use predefined names for common
functions/states/events. It'll be easier to share and re-use test code
this way.

If you'd already like to use a Common Test suite with a gen_fsm
behaviour to test your FSMs, here's a very simple (and very incomplete!)
example:

-module(fsm_SUITE).

-behaviour(gen_fsm).

-export([code_change/4,
handle_event/3,
handle_info/3,
handle_sync_event/4,
init/1,
terminate/3]).

-include_lib("common_test/include/ct.hrl").

-define(FSM, ?MODULE).
-define(TO, 5000).
-record(fsm_state, {testcase, result}).

%% --- Test suite callback functions ---

all() -> [open_connection, ...].

suite() -> [{require,connection_data}].

init_per_testcase(TC, Config) ->
{ok,_} = gen_fsm:start({local, ?FSM}, ?MODULE, [TC], []),
Config.

end_per_testcase(_, _Config) ->
gen_fsm:sync_send_all_state_event(?FSM, stop, ?TO).

open_connection(_Config) ->
ok = gen_fsm:send_event(?FSM, open_connection),
verify_result(open, ok).

verify_result(State, Result) ->
{State,Result} = gen_fsm:sync_send_all_state_event(?FSM, report, ?TO).

%% --- FSM behaviour callback functions ---

init([TestCase]) ->
ct:log("-> Idle", []),
{ok, idle, #fsm_state{testcase = TestCase}}.

idle(open_connection, State) ->
Result = server_utils:open(ct:get_config(connection_data)),
ct:log("-> Open", []),
{next_state, open, State#fsm_state{result = Result}}.

open(Event, State) ->
...

handle_event(_Event, StateName, State) ->
{next_state, StateName, State}.

handle_sync_event(report, From, StateName, State) ->
{reply, {StateName,State#fsm_state.result}, StateName, State};
handle_sync_event(stop, _From, _StateName, State) ->
{stop, normal, ok, State}.

handle_info(_Info, StateName, State) ->
{next_state, StateName, State}.

terminate(_Reason, _StateName, _State) ->
ok.

If you have any comments or suggestions on this topic, let us know!

/Peter

Erlang/OTP, Ericsson AB

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questio...@erlang.org

[Max Lapshin]

If you have really clean state machine, it's state is always described
by initial state and command words sequence.

You may try several techniques:

1) most simple and hard to support: test each transition from any
state by each possible command word.
2) fuzzy testing. Log down initial state, log stream of random input
command words and perform some state cheks.
If failed, write you an email with sequence of command words. Well
known technique in gamedev.

[Roman Shestakov]

thanks Peter for suggestion - this sounds like a good idea, will give a try
Regards, Roman

________________________________
From: Peter Andersson <pe...@erix.ericsson.se>
To: Roman Shestakov <romansh...@yahoo.co.uk>
Cc: erlang-q...@erlang.org
Sent: Thu, 26 August, 2010 15:08:01
Subject: Re: [erlang-questions] what is the best way to test Erlang FSM
processes?