WIKI: Assert Libraries for Testing

[Romenigue Thier]

Hello!

Recently I learn the page TestComments on Golang Wiki (on Github).

The section about Assert Libraries discourage the use of them.

But I guess there's a little bit mistake in this approach because we have good libraries that provide assert functions.

The argument omits interesting information about what the test got right is not the real true, is more easy to understand what the test does with 2 or 3 asserts than a if with 2 or 3 ORs inline. Furthermore libraries as Testify provide rich and easy-readable informations when the test fails in the assert functions.

Finally, using asserts or ifs don't guarantee that the test is right, the argument Assert libraries make it too easy to write imprecise tests is more about the quality of test than the use of an assert library.

Any thoughts?

(I tried to create an issue on Github, to promove the discussion about it, but was closed before the discussion start =|)

[Dan Kortschak]

I have thoughts about the impact of assertion libraries in software
engineering. I came from a background where they are commonplace into
Go and in my earlier projects I used some the earlier iterations on
them (go-check).

While it's true that developers *can* use assertion libraries to
provide rich information about failures, in the vast majority of cases
they don't and this gets past review because without enrichment there
is just enough information to not be completely useless; you end up
with something along the lines of `assert.Equal(got, want)` which will
emit the line and a diff, but not any additional context, nor an
contextually relevant diff, while with a plain pkg/testing-based tests,
the author *must* include explicit context tailored for the setting to
get past any reasonable code review. The outcome then is weaker
information in the failing case and often more of it, so the work
needed to debug failures is compounded.

The other impact of assertion libraries is that they have gone to the
effort to compose a multitude of different tests from the small number
of language primitives that actually do the work of assessing a test's
success or failure which means that instead of needing to know just the
language, which I already do because I am writing code in it that is
being tested, I also need to know the assertion library's idioms and
quirks — often requiring digging several abstraction layers into the
assertion library — so there is a not insignificant additional
cognitive burden at the very time when cognitive burdens need to be
minimised (while debugging an issue).

Having seen both sides of approaches to testing, I am very glad that Go
does not promote the use of assertion libraries and I am sad that so
many people who write Go code have adopted them despite the project's
recommendation not to.

[Romenigue Thier]

Thanks @kortschak!

I understand your concerns, I work on the "both sides" too.

The code (test) should be readable, easy to understand and a simple assertion can be more "clean way" that a long conditional if. A "ugly test" can be created with or without an assertion library.

The concern about "different idiom" need to be measured, some libraries is more disruptive like Gomega/Ginkgo because use a different approach to create and structure the tests, in this cases should understand what is the layer/purpose of the test: is based on ATDD/BDD? is auto documented? use Gherkin definition?
But tests using Testify shouldn't be a problem to read, the asserts from it are very easy to understand. We can pass some "additional context" to troubleshooting the assertion on Testify (as you can see on https://github.com/stretchr/testify#assert-package), and maybe this will more rich because, as mentioned, we also have the diff.

And don't be sad, you have a free choice to use or not an assertion library (and for me there's no wrong choice)

[Rheza]

Go author has their own ideal. 

Writing Go should be simple without too much 3rd party lib that can make mini-language inside Go language. 

It's up to you if you want to adopt. But the Wiki https://github.com/golang/go/wiki/TestComments is a good place to consider how you write better code and simple code.