Why does Thucydides execute steps AFTER a step that Assert.fail()s?

[nrzzz]

Why after asserting a failure (or throwing an exception) does Thucydides continue to execute further steps?

I see in my logs StepInterceptor "STEP FAILED", but then it continues to enumerate steps, logging "SKIPPED STEP" -- but the skipped steps are still run (by StepInterceptor.runSkippedMethod(...). Why is that?

[John Smart]

Yes, it needs to run subsequent steps in 'dry-run' mode - without doing any webdriver calls - to discover what steps exist. Without this, there is no way of knowing what steps are being missed out on. If this is a problem, we could probably make it configurable, but you would loose the extra step data.

On 7 September 2013 05:16, nrzzz <phl...@gmail.com> wrote:

Why after asserting a failure (or throwing an exception) does Thucydides continue to execute further steps?

I see in my logs StepInterceptor "STEP FAILED", but then it continues to enumerate steps, logging "SKIPPED STEP" -- but the skipped steps are still run (by StepInterceptor.runSkippedMethod(...). Why is that?

--
You received this message because you are subscribed to the Google Groups "Thucydides Users Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to thucydides-use...@googlegroups.com.
To post to this group, send an email to thucydid...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

___________________________________________________
John Smart | Wakaleo Consulting  |  +61 407 247 642
Optimizing your software development process
http://www.wakaleo.com  |  john....@wakaleo.com
___________________________________________________

Check out our upcoming BDD/TDD Master classes, coming 

soon to Sydney, Melbourne, Canberra, Brisbane and Perth

___________________________________________________

[nrzzz]

I have some logging code and other stuff that does a confusing amount of work after the test has already failed, due to these "skipped" steps being executed. Perhaps I should check StepEventBus.getEventBus().aStepInTheCurrentTestHasFailed() before performing certain tasks within my Steps -- this would preserve the Thucydides semantics while allowing me to opt-out. Does that sound reasonable?

[Mikhail Goncharov]

I have experience similar problems with skipped tests. I think that ability to define own StepsInterceptor will be nice for two reasons

1. allows you to hard-stop steps execution - when you use something other that web-client (for example I use API calls to application as it times faster and not everything is exposed in UI) and do not care much about what was skipped
2. allows you to collect and aggregate info about steps - for example I would like to know which steps are candidates for performance improvement (for example I can decide to switch to low-level calls instead of UI when I assume that this UI part is working but I have to configure something)

moreover I implement this too if you like the idea. If so - how to approach this? Should I create issue first?

[John Smart]

I think it would be safer to do this with listeners (which you can use for (2)) and attributes on the @Step annotation to be able to not execute a step method at all in "dry-run" mode. 

To post to this group, send email to thucydid...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

___________________________________________________
John Smart | Wakaleo Consulting  |  +61 407 247 642
Optimizing your software development process
http://www.wakaleo.com  |  john....@wakaleo.com
___________________________________________________

The dates for the 2014 BDD workshops have been scheduled! Check out our upcoming BDD/TDD Master classes and our Advanced BDD Requirements Workshops, coming soon to Sydney and Melbourne!

___________________________________________________

[Mikhail Goncharov]

Thank you, John! That's definitely better

[Mikhail Goncharov]

Hi John!

As far as I understand current codebase there is no such annotation on @Step that prevent step from running, isn't it?

From here I see three ways:

1 add some attribute to @step annotation and add handlers

2 lean privacy of ThucydidesRunner.initStepFactory() to protected (and other method / fields) . This way I can create descendants of runner, stepfactory and stepinterceptor and do whatever I want

3 copy-and-paste ThucydidesRunner, factory etc. and use them (least preferred as I bind myself to specific version of Thucydides and give nothing to community)

I like 2 most as it gives more options to control step execution then implementation of specific branching in control flow.

What do you think?

Cheers,

Mikhail

[John Smart]

In the next release, you will be able to use the callNestedMethods attribute for the @Step annotation to force Thucydides not to run the code inside a step method. Have you tried adding your own listener classes for your other requirements? 

[Mikhail Goncharov]

Yea, I have created file under /src/test/META-INF/services/ with classname of my ServiceListener implementation (just like SampleStepListener in Thucydides tests).

I don't get that idea about "callNestedMethods" attribute - is it implemented but not released yet? I don't see it in 'master' at github. Should I look somewhere else?

Thank you!

[Mikhail Goncharov]

For googlers:

You can now use the callNestedMethods attribute of the @Step annotation to force Thucydides not to run a given step after a previous step has been skipped, has failed, etc. This can be useful to avoid unnecessary database queries, for example. Note that this behaviour already happens automatically for WebDriver calls.

     @Step(callNestedMethods=false)
     public void shouldnt_be_called_after_a_previous_step_failure(){
         databaseAPI.call("some query");
     }

 

[Pareshkumar Chaudhari]

Thank you.

I have one question.

Is there a simple way for not SKIPPING all methods after any failure (at least finally block) ?