Soft assertions in Spock - RFC

Skip to first unread message

Marcin Zajączkowski

Sep 18, 2014, 8:42:06 AM9/18/14

Recently I wanted to make a soft condition verification (collect all the
failures and display them instead of fail on the first one [1]) in the
integration test in Spock and I haven't found any satisfactory solution.
I did a quick (and successful) PoC and it seems quite easy to add it to
Spock. Before polishing the implementation and making a PR I would like
to discuss the API first.

[1] -
- Soft assertions in AssertJ

As each statement has to be executed separately (and potential
AssertionError be caught) I see no easy way to pass all the things as
one big Closure (and later split into chunks) and every condition has to
be passed in a separate Closure:
void assert(Closure<?>... conditions)`

def softly = new SoftConditions()

{ assert 1 },
{ assert foo == "Bar" }

SoftConditions could be (probably) also wrapped with PollingContitions
and similar.
I have seen in PollingConditions that assert keyword can be omitted with
the help of @ConditionBlock, so hopefully in SoftConditions it would be
possible as well.

To make basic usage easier there would a method `void
softly(Closure<?>... conditions)` in the Specification class which
creates SoftConditions and passes assertions to it:

{ assert 1 },
{ assert foo == "Bar" }

I predict the misuse with one big Closure and two prevent that I have
two ideas:
1. Detect the situation when only one Closure is passed and throw very
verbose exception explaining that "there is no sense to use
SoftAssertions with just one statement and in case there are many
statements different statements should be passed in a separate Closures.
2. Changes method signature to `softly(Closure<?> condition1, Closure<?>
condition2, Closure<?>... conditions)`, but it will generate only a
warning (not a compilation error) and MissingMethodException at runtime
is much less meaningful.

What do you think about SoftAssertions in Spock in general and the
proposed API?


-- - Working code is not enough

Marcin Zajączkowski

Sep 18, 2014, 10:35:47 AM9/18/14
Will be possible, but with the same restrictions as with
PollingCondtitions - only when a type is declared explicit, like:
SoftConditions softly = new SoftConditions()
(will not work also with softly method declared in Specification class -
at AST level (MethodCallExpression/VariableExpression) there is no
information about concrete class type).


Peter Niederwieser

Sep 19, 2014, 3:22:26 AM9/19/14

soft assertions can definitely be useful, and it would be great to have support for them. I think I'd favor a deeper integration that allows something like:

@SoftConditions // can alternatively go on class
def "killing me softly"() {
  expect: "weirdness"
  1 == 2 // evaluated
  2 == 3 // evaluated

Or perhaps:

// could additionally require annotation here
def "killing me softly"() {
  expect: "weirdness"
  1 == 2 // evaluated
  2 == 3 // not evaluated

  and: "more weirdness"
  3 == 4 // evaluated
  4 == 5 // not evaluated

What I like about the second proposal:
* Each individual block is still evaluated in the usual way.
* Dependent conditions are still possible (in the same block).
* There can still be just one failure per named unit, just that the unit is now a block rather than a method.

Implementing one of these proposals may take more effort than a standard API approach, but feels more in the spirit of Spock to me.


Maciej Gawinecki

Oct 23, 2017, 12:14:55 PM10/23/17
to Spock Framework - User
What is the current state of this feature request?

Leonard Brünings

Dec 3, 2017, 8:22:20 AM12/3/17
to Spock Framework - User
Reply all
Reply to author
0 new messages