testing large, interactive processes quickly
I. E. Smith-Heisters
I have a somewhat complicated process definition, most of which is
comprised of putting the workitem in a StorageParticipant and waiting
for a human user to do something to it (via a web interface). The
problem I'm running into is in testing: as I test the deeper parts of
the process, my tests become slower and slower. Since Ruote is
asynchronous, I have to sleep the test process, and as the tests go
deeper, the sleeps build up into unacceptable lag.
For instance, in a simple case, I might do something like "launch,
sleep(0.1), interact, sleep(0.1), test". Of course, to test an
interaction three steps down in the process tree, there are 4 sleeps
required. Further aggravating the issue, the required sleep time isn't
consistent across development machines (or even on the same machine),
so I usually find I need to sleep for 0.2 - 0.5 seconds *per step* in
order to have consistent results.
I've tried Engine#wait_for, but none of the options work for me:
passing a wfid won't work because the process isn't terminating,
passing a participant just hangs the test process indefinitely, and an
integer won't work because I've no idea how many messages need to be
passed.
I see two potential solutions:
1. some way to "jump" the process instance to a given state, kind of
like stubbing or database fixtures
2. some way to reliably detect and respond to engine events,
effectively making my tests run synchronously with the engine (ie. get
#wait_for working)
Unfortunately, I've no idea how I would approach either solution. Are
there other solutions? Help approaching one of these?
Thanks,
Ian
John Mettraux
On Thu, May 27, 2010 at 05:27:38PM -0700, I. E. Smith-Heisters wrote:
>
> I've tried Engine#wait_for, but none of the options work for me:
> passing a wfid won't work because the process isn't terminating,
> passing a participant just hangs the test process indefinitely, and an
> integer won't work because I've no idea how many messages need to be
> passed.
Hello Ian,
it's true that the wait_for(participant_name_as_a_symbol) might not be applicable in your case.
I use it with success in ruote's test suite, but it's using the Ruote::TestLogger instead of the Ruote::WaitLogger coming by default with the engine.
The TestLogger keeps track of seen messages, it's more reliable, for cases when the engine is faster than the wait_for() order. The symptom of "engine faster than wait_for()" is simply the wait_for() never returning since the message it's waiting for is gone.
To use the TestLogger, you have to make sure the worker/engine/storage uses it as the logger service, like in this example :
---8<---
@engine =
Ruote::Engine.new(
Ruote::Worker.new(
determine_storage(
's_logger' => [ 'ruote/log/test_logger', 'Ruote::TestLogger' ])))
--->8---
The downside is that this TestLogger is only meant for testing, its log of seen messages is not bound, if no wait_for 'consumes' it, it may grow too large.
> I see two potential solutions:
>
> 1. some way to "jump" the process instance to a given state, kind of
> like stubbing or database fixtures
There has already been demand for this kind of feature. It should be easy to wrap participants to force some of them to "fast forward". I will think about it, I think it's a necessary feature.
Somehow, it can already be realize by registering "fast forward" participant for the sake of the test.
> 2. some way to reliably detect and respond to engine events,
> effectively making my tests run synchronously with the engine (ie. get
> #wait_for working)
I could help if you tell me how (regex, absolute names) you register your participant and how you except to wait_for them.
> Unfortunately, I've no idea how I would approach either solution. Are
> there other solutions? Help approaching one of these?
There could be a third solution : since you are using StorageParticipant, you could extend it for your testing purposes.
see http://gist.github.com/416620 :
---8<---
class TestStorageParticipant < Ruote::StorageParticipant
def initialize (options)
super(options)
@seen = []
end
def consume (workitem)
@seen << workitem.dup
# dup in order to return the workitem as 'received' by the participant
super(workitem)
check_waiting
end
def wait_for (*participant_names)
@waiting = [ Thread.current, participant_names ]
check_waiting
Thread.stop if @waiting
Thread.current['__result__']
# return the waited for workitem as a result
end
protected
def check_waiting
return unless @waiting
while workitem = @seen.shift
break if check_workitem(workitem)
end
end
def check_workitem (workitem)
if @waiting[1].include?(workitem.participant_name)
thread = @waiting[0]
@waiting = nil
thread['__result__'] = workitem
thread.wakeup
true
else
false
end
end
end
--->8---
Warning : this is not tested, it's been written during a train ride (right now).
Your email made me realize I'm using sleep instead of wait_for in ruote-cukes :
http://github.com/jmettraux/ruote-cukes
I have to change that.
Glad to help, best regards,
--
John Mettraux - http://jmettraux.wordpress.com
I. E. Smith-Heisters
> The TestLogger keeps track of seen messages, it's more reliable, for cases when the engine is faster than the wait_for() order. The symptom of "engine faster than wait_for()" is simply the wait_for() never returning since the message it's waiting for is gone.
----8<----
module ProcessSpecHelper
def stub_hash_storage
storage =
Ruote::Engine.new(Ruote::Worker.new(Ruote::HashStorage.new('s_logger'
'Ruote::StorageHistory')
part = Ruote::StorageParticipant.new(storage)
Workflow.stub!(:engine).and_return(storage)
Workflow.stub!(:workitems).and_return(part)
Participants.register_all
end
def wait_for identifier
Workflow.engine.wait_for identifier
end
end
---->8----
... which of course assumes Workflow::engine and Workflow::workitems
(which are quite simple). I had initially tried using one HashStorage
across multiple tests, but clearing the processes became problematic,
and I found that it wasn't too expensive to create a new HashStorage
for every test (this is called in before(:each), usually).
However, this introduces a problem in that my specs run on a different
engine than is used in production. For this reason, I've left my
integration tests (cucumber) using sleep. It's not as big of a problem
there because the tests are less granular requiring less sleeping.
Thanks for your help with this.