Should PowerAsserts be more verbose? (repost from user@groovy)

[tibo...@googlemail.com]

Hi all,

I asked this yesterday on the user@groovy mailing list, but since Spock users would be affected by this mostly (I believe Spock uses Groovy power asserts since they moved into groovy), I thought it might be wise to also ask here.

I am wondering whether the output of PowerAsserts should be increased
to help understand test failures more easily.
Here are some examples (column alignment will look bad with
non-monospace fonts):

groovy:000> def id(it) {it}
===> true

groovy:000> assert [false, false] == id(['false', 'false'])
assert [false, false] == id(['false', 'false'])
                      |  |
                      |  [false, false]
                      false

groovy:000> assert [1, 2, 3, 4] == id('1'..'4')
assert [1, 2, 3, 4] == id('1'..'4')
                    |  |
                    |  [1, 2, 3, 4]
                    false

groovy:000> assert [1, 2, 3, 4] == new HashSet([1, 2, 3, 4])
assert [1, 2, 3, 4] == new HashSet([1, 2, 3, 4])
                    |  |
                    |  [1, 2, 3, 4]
                    false

groovy:000> assert [1, 2, 3, 4] == id('[1, 2, 3, 4]')
assert [1, 2, 3, 4] == id('[1, 2, 3, 4]')
                    |  |
                    |  [1, 2, 3, 4]
                    false

groovy:000> assert [1, 2, 3, 4] == id([1, 2, '3, 4'])
assert [1, 2, 3, 4] == id([1, 2, '3, 4'])
                    |  |
                    |  [1, 2, 3, 4]
                    false

As you can see the output in these cases does not help the user
understand what went wrong (assume the right hand side of the == being
something like foo()). This could be helped e.g. by adding hyphens
around Strings.

Another problem is checking escape symbols:

groovy:000> assert '\\\' \\\'' == id('\\\'\\\' \'')
assert '\\\' \\\'' == id('\\\'\\\' \'')
                   |  |
                   |  \'\' '
                   false

Here the power assert output is the final form of the string, not the
code-form. And it is a pain to mentally (un)escape to compare and find
differences.

So I would suggest extending the output such that:
- String values generally get quoted (with escaped special symbols),
- non-List collections get Type information added like "[x, y] as Set"
- Ranges are printed in code notation (e.g. 1..4 instead of [1, 2, 3, 4])

One of the small problems is that Strings like
'one\ntwo\rthree\r\nfour', which are rendered as multiline values,
will be rendered using \n, looking a bit uglier maybe.

Would that change be helpful? Can the behavior be changed without
breaking user code unduly?
What do you think?

Should I post updates about this here as well?

[Kostis Kapelonis]

Hello

I find your examples very interesting. However I think that Spock has
more targeted power asserts because they are the last step in unit
tests.
For example if I am writing a unit test where I expect a certain
HashSet in the end, then I find it hard to believe that I will pass as
expected result something that is not a hashset.
(I refer to your third example)

Also I think that Spock already does something extra in some cases.

Pure Groovy

assert "now" == "now there"
|
false

Spock

"now" == "now there"
|
false
6 differences (33% similarity)
now(------)
now( there)

I personally have never found cases where I wished for more
information in Spock tests (others might have a different opinion)
Would it be possible to perform your changes only for failed tests? If
you could modify just the output when a power assert fails then no
user code will be broken.
Spock tests that already pass will continue to pass, while Spock tests
that fail will just gain more information.

Kostis

> --
> You received this message because you are subscribed to the Google Groups
> "Spock Framework - User" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to spockframewor...@googlegroups.com.
> To post to this group, send email to spockfr...@googlegroups.com.
> Visit this group at http://groups.google.com/group/spockframework.
> For more options, visit https://groups.google.com/d/optout.

[tibo...@googlemail.com]

Interesting, I had no idea Spock had extended the asserts further. So if Groovy changes, I guess Spock will not be immediately be affected, but Spock could change in the similar way.

The code change for Spock then would really be just calling another method to print the values.

And of course, the extended value would only change anything for failed tests.