Proper way of mocking interfaces in unit tests - the golang way

[Krishna C]

We are working on a full scale production app with go backend.

For unit tests, we need to mock the dependent interfaces either by creating our own mock implementation of the interface or by using third party packages like testify/mocks.

If we use testify, there will be a tendency for the developers to use assert statements which is against the best practices in go unit tests. (We use t.Fail instead of asserts)

If we create manual mocks, we will end up creating two implementations for each interface, one for the program and another for the unit testing. Also we cannot test by mocking with different outputs from the dependencies. example if the dependency return an error.

Please suggest which the right approach here.

[Vladimir Varankin]

Hey,

I don't think the statement about "assert going against the best practices in go unit tests" stands against the reality, sorry. One definitely doesn't have to use a separate assertion package to write unit-tests in Go, comparing to some other programming languages. But there is really no much difference between using an assertion of t.Fail-ing manually.

In our project, we use testify assertions (only for statistics: our monorepo is ~400K lines of Go code with every package covered with unit-tests) and it works great just by reducing the total number of lines in our test files and making them much more manageable (we have unit-tests files that over years has grown over 3K+ LOC now).

Depending on the interface, we usually create a separate "controlled" implementation, that we use in unit-tests. E.g. if we implemented a DataStore, we would have an in-memory implementation with additional Setter-methods, that accepted an optional "hook", that allowed us to return an error.

We don't use "testify/mock" because we aren't interested in verifying the calls or the arguments that were passed to the mock. Usually, it's easier to write a fake implementation. I.e. our testing implementations are more "fakes" than "mocks".

One neat thing about that approach is that testing implementation doesn't always have to expose lots of knobs. If one unit-test needs to tune the behaviour, you can always stub the dependency right in-place and use it for this single test only. E.g.

type testDataStore struct {

    callFooFunc() error

}

func (s *testDataStore) Foo() error {

    return s.callFooFunc()

}

func TestSomething(t *testing.T) {

    ds := &testDataStore {

        callFooFunc: func() error {

           return fmt.Errorf("a very specific error that make sence to test against in this test only")

        }

    }

    // ···

}

Hope that makes sense.

[Jason Phillips]

In the general case, all testify assertions are doing under the hood is call t.Errorf (see here). If you're calling the "FailNow" set of functions that's followed by t.FailNow, which ends the test early. As others have stated, that's no different than you calling those methods on testing.T, directly.

What led you to believe that using testify/assert is against Go best practices?

[Krishna C]

Hey,

Thanks for both the explanations. They really make sense to me.

I referred to the following link and thought assertions are against go best practices => https://golang.org/doc/faq#testing_framework.

After these discussions, it makes sense to move with using testify and use the package's assert.

For interface mocking, the approach suggested by Vladimir looks good. But I will have to implement both "fake" and "mock" approach and see what best works for us.

Thanks a lot for the suggestions.

[jake...@gmail.com]

On Saturday, October 3, 2020 at 6:00:10 AM UTC-4 krish...@gmail.com wrote:

Hey,

Thanks for both the explanations. They really make sense to me.

I referred to the following link and thought assertions are against go best practices => https://golang.org/doc/faq#testing_framework.

 

I find the link, https://golang.org/doc/faq#testing_framework, to be confusing as well. I'm not clear on how the hypothetical "assert" it refers to is different from testing.T.Fatal().

 

 

[Joop Kiefte]

The assert it refers to is the C and other languages assert which makes the program fail immediately on not satisfying the condition and is used inline with the rest of the code. In short, the elements that make it bad Go style are 1. adding the tests together with the running code and 2. having it crash the program.

The Go way of doing it is putting it away from the main program, keeping the main program always safe to run.

The testify assert copies semantics but not these problems, so there is no issue at all using it.

Joop Kiefte - Chat @ Spike

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/c96d148e-6b73-40d0-9e3c-b8666b2e9d6dn%40googlegroups.com.

[Jesper Louis Andersen]

On Thu, Oct 1, 2020 at 6:25 PM Krishna C <krish...@gmail.com> wrote:

For unit tests, we need to mock the dependent interfaces either by creating our own mock implementation of the interface or by using third party packages like testify/mocks.

Is there any particular reason as to why you want to mock dependent interfaces other than "best practices" ?

The reason I'm asking is that whenever I have large cumbersome interfaces I find that these mocking endeavours end up in a lot of work for very little gain. In my experience, it is better to find an interface where the interface part is as small as possible. For example, look at the HTTP test system in the stdlib: There is a HTTPWriter hidden inside the response object, and it is fairly small. This interface is what can be replaced with another implementation for testing, but most of the interface actually stays the same when you use it.

Another reason I'm asking is because it looks like you've stumbled into a set of large dependent interfaces and are now seeking a cookie cutter solution to the problem, namely mocking. If at all possible, restructuring your code to simplify it often avoids the need for mocking and makes unit-tests far easier to get along, but without code, it is hard to get an idea of what you are facing here.

[Bryan C. Mills]

I think that FAQ entry really is referring to jUnit-style assert functions, including those provided by testify.

The core of the problem is that `assert` functions generally cannot provide enough context to produce useful test failures. An `assert` call compares values, but does not report the relevant inputs and calls leading to those values — and the description of the inputs and the operations performed on those inputs are what the person diagnosing the test failure will need in order to figure out what went wrong.

If a Go test fails, the person running the test should be able to diagnose the problem from the test log alone, without reading the source code for the test. I just don't see testify-based tests hitting that standard.

[Bryan C. Mills]

As for mocking dependencies: in my experience, the best strategy is often not to.

It is a good idea to avoid depending on services or resources external to the test, such as other production services. However, in my experience you'll get much better test fidelity — you'll find more real bugs, and spend less time fixing erroneous test assumptions — if you have the test use its own hermetic instance of the real dependency.

[Viktor Kojouharov]

I don't find any difference between calling t.Errorf and assert.Something with a provided message. Both will populate the test log, with the later giving you more details exactly where things differ from the expectation.

[Robert Engels]

Truth. I would even go farther and say that mocks are a code smell - and lead to very brittle tests. 

On Oct 5, 2020, at 8:14 AM, 'Bryan C. Mills' via golang-nuts <golan...@googlegroups.com> wrote:

As for mocking dependencies: in my experience, the best strategy is often not to.

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/2b1affd7-1599-4943-8419-d9718f17780fn%40googlegroups.com.

[Ian Lance Taylor]

On Mon, Oct 5, 2020 at 8:47 AM Viktor Kojouharov <vkojo...@gmail.com> wrote:
>
> I don't find any difference between calling t.Errorf and assert.Something with a provided message. Both will populate the test log, with the later giving you more details exactly where things differ from the expectation.

The difference is that since people write t.Error while writing the
test, it's very easy to provide all the relevant information, which in
many cases will involve more than just the values being compared. A
common example would be a message like "MyFunction(%v) = %v, want %v".
When using an assert style function, the message will tend to lose the
value passed to MyFunction. Or, it will be written as
assert.Equal(got, want, fmt.Sprintf("MyFunction(%v)", input)), but
that is harder to write, and therefore less likely to be written.
(It's also less efficient in the common case, though for a test that
is unlikely to matter.)

Ian

[Vladimir Varankin]

> Or, it will be written as assert.Equal(got, want, fmt.Sprintf("MyFunction(%v)", input)), but that is harder to write, and therefore less likely to be written.

That obviously depends on the implementation details but since we talk about testify the API is: assert.Equal(t, want, got, format, [args...]).

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/ZoJ5isoeea4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXNV6bLQQUFmVn14xnxheALTKbv1_oQODovboLEziPKSw%40mail.gmail.com.

--

Vladimir Varankin

vlad...@varank.in

[Marcin Romaszewicz]

On Mon, Oct 5, 2020 at 10:31 AM Vladimir Varankin <vlad...@varank.in> wrote:

> Or, it will be written as assert.Equal(got, want, fmt.Sprintf("MyFunction(%v)", input)), but that is harder to write, and therefore less likely to be written.

That obviously depends on the implementation details but since we talk about testify the API is: assert.Equal(t, want, got, format, [args...]).

The testify assert/require code may be a little backwards and verbose when compared to a naked t.Error(), however, it provides a lot of helper functions for comparing types and values which results in much clearer, shorter tests in the end. You can assert.EqualValues() to ignore type, for example, or do deep compares without having to code that yourself. I'd say it's wonderful.

 

On Mon, 5 Oct 2020 at 18:45, Ian Lance Taylor <ia...@golang.org> wrote:

On Mon, Oct 5, 2020 at 8:47 AM Viktor Kojouharov <vkojo...@gmail.com> wrote:
>
> I don't find any difference between calling t.Errorf and assert.Something with a provided message. Both will populate the test log, with the later giving you more details exactly where things differ from the expectation.

The difference is that since people write t.Error while writing the
test, it's very easy to provide all the relevant information, which in
many cases will involve more than just the values being compared.  A
common example would be a message like "MyFunction(%v) = %v, want %v".
When using an assert style function, the message will tend to lose the
value passed to MyFunction.  Or, it will be written as
assert.Equal(got, want, fmt.Sprintf("MyFunction(%v)", input)), but
that is harder to write, and therefore less likely to be written.
(It's also less efficient in the common case, though for a test that
is unlikely to matter.)

Ian

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/ZoJ5isoeea4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXNV6bLQQUFmVn14xnxheALTKbv1_oQODovboLEziPKSw%40mail.gmail.com.

--

Vladimir Varankin

vlad...@varank.in

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAGLqCMmj7NB3Sem0BDdyJBj-cxGXVmBY-6StRXAP6MvwZUyOnQ%40mail.gmail.com.

[Jesper Louis Andersen]

On Mon, Oct 5, 2020 at 3:03 PM 'Bryan C. Mills' via golang-nuts <golan...@googlegroups.com> wrote:

The core of the problem is that `assert` functions generally cannot provide enough context to produce useful test failures. An `assert` call compares values, but does not report the relevant inputs and calls leading to those values — and the description of the inputs and the operations performed on those inputs are what the person diagnosing the test failure will need in order to figure out what went wrong.

This is an excellent example of boolean blindness. A pass or panic is essentially the same as the values true and false. But the astute reader will realize that just knowing that an expression evaluates to "true" doesn't give you any information as to why. Many programs then have other functions you can call to get a description of what is wrong in the program, but this leads to a style where you first compute a boolean value, then have a large chunk of code which is used to figure out what is wrong.

It is often better to return a more complex data structure for the error in a program, and make it a Stringer so it has a human readable variant. This ensures the program is able to work with the data structure underneath, without relying on the human text, while also providing the human a way to understand what is wrong[1].

Another example of this in action would be the government telling you that you have to pay $10K in taxes, but upon request they cannot tell you how much of that is from gifts, income, wealth, etc. You become "blind" to the details of the computation and are only given a final number to look at. If you have a data structure, you can have an "Eval()" method on it which can compute the final number. This claws at the concept of Free constructions and universal mapping properties in Category Theory: you have a data structure which throws away no information. Then you attach several interpreters to this structure, each with their own view of the data at hand. Sometimes you just want the sum you have to pay in taxes. Some times you want it as an HTML table, JSON or Spreadsheet. And some times as an R data frame. Hence you use the general structure such that all other derivations factorize through it.

Finally, a mathematical variant is knowing that a theorem is true without knowing its proof.

[1] And if you have proper pattern matching in the language, you can then get rid of if..then..else which becomes redundant: match x with true -> .. | false -> ..end