How to mock structs with interdependent interface methods?

[karth...@gmail.com]

I am having trouble writing unit tests in Go for a rather common use-case/pattern.

Imagine, if you will, something like this:

package main

type Resource struct {
    name string
}

type ResourceManager interface {
    GetResource(id string) (*Resource, error)
    GetAllResources() ([]*Resource, error)
}

type ResourceManagerImpl struct {
}

func (r *ResourceManagerImpl) GetResource(id string) (*Resource, error) {
    resource := &Resource{}
    var err error

    // fetch resource.
    // ...

    return resource,  err
}

func (r *ResourceManagerImpl) GetAllResources() ([]*Resource, error) {
    var resources []*Resource
    var err error

    // for _, id := range ids {
    //  resource = r.GetResource(id)
    //  resources = append(resources, resource)
    // }

    return resources, err
}

It's a common pattern to have the GetAllResources invoke GetResource repeatedly as-needed.

I can use gomock or testify to test all permutations of GetResource. But, when testing GetAllResource, I'd like to mock GetResource. Otherwise, testing would become sort-of-a nightmare. This is how one would do it in easymock or mockito in case of Java using partial mocks. But, it's not clear how the same could be achieved in Golang.

Specifically, I couldn't find how to partially mock the struct. Most suggestions revolve around breaking such structs but in this case, the struct is already at its bare minimum. It seems like a fair ask to not break the ResourceManager interface (into single and multi) for the sake of testing as that would not make much sense and would be kludgy at best and wouldn't scale well as more such methods make into the interface.

[Marcin Romaszewicz]

I face similar problems in a lot of production Go code which I write. I do have an answer for this, but it's not pretty. Your code would look something like this:

type ResourceGetter interface {

    GetResource(id string) (*Resource, error)

}

type ResourceManagerImpl struct {

  rg ResourceGetter

}

func (r *ResourceManagerImpl) GetResource(id string) (*Resource, error) {

  return r.rg.GetResource(id)

}

Then, you'd move your implementation of GetResource to ResourceGetterImpl. You can then stub in a mock ResourceGetter in your test, and the GetAllResources function is none the wiser.

Using so many interfaces has a nice side effect - I can pass things around using their minimal function footprint to make the code generally easier to refactor and test.

-- Marcin

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/3c22e3fc-b3ad-47e7-9e41-400856423e58%40googlegroups.com.

[Ian Davis]

On Thu, 12 Dec 2019, at 10:27 PM, karth...@gmail.com wrote:

Specifically, I couldn't find how to partially mock the struct. Most suggestions revolve around breaking such structs but in this case, the struct is already at its bare minimum. It seems like a fair ask to not break the ResourceManager interface (into single and multi) for the sake of testing as that would not make much sense and would be kludgy at best and wouldn't scale well as more such methods make into the interface.

I think your problems likely stem from having interfaces that are too large. Interfaces in Go are more flexible than in many other languages since there is no "implements" keyword that binds a struct to a particular interface. This means that you can have many more specialised interfaces that your struct may automatically satisfy.

One consequence of this is that In Go it is usually preferable to define the interface where it is consumed instead of where the implementations are. This tends to make interfaces smaller, having just the methods needed for the consumer. It also implies that testing via the interface is the wrong approach.

First write your tests against the struct to verify that getting one or all resources work. Break the interface into smaller specialised ones and move them next to the consuming code. In your case you may have an endpoint that uses a GetResource method, so you define that interface. Another endpoint may need just the GetAllResources method: another interface. You can then test your endpoints by supplying a test struct that implements the smaller interface needed.

-- Ian

[Serge Voilokov]

You can partially mock the interface. For example I want to simulate AWS S3:

// real S3 client

type realAWSClient struct {

  s3iface.S3API

}

func processMyBuckets(s3 *s3iface.S3API) {

  list := s3.ListBuckets(...)

  // work with list

}

func main() {

  cli := &realAWSClient{S3API: s3.New(...)}

  processMyBuckets(cli)

}

// mock struct in _test.go code

type mockAWSClient struct {

  s3iface.S3API

}

func (mock *mockAWSClient) ListBuckets(*s3.ListBucketsOutput, error) {

   out := createMockOutput()

   return out, nil

}

func TestMyBuckets() {

  cli := &mockAWSClient{}

  processMyBuckets(cli)

}

S3API interface has many methods, but in this case I mock only ListBuckets method.

If my code will call other methods then I'll get nil pointer error and I'll mock new methods as needed.

[Karthik Krishnamurthy]

Thanks for the response Ian. My apologies for a delayed response (inline).

On Friday, December 13, 2019 at 6:11:20 AM UTC-8, Ian Davis wrote:

On Thu, 12 Dec 2019, at 10:27 PM, karth...@gmail.com wrote:

Specifically, I couldn't find how to partially mock the struct. Most suggestions revolve around breaking such structs but in this case, the struct is already at its bare minimum. It seems like a fair ask to not break the ResourceManager interface (into single and multi) for the sake of testing as that would not make much sense and would be kludgy at best and wouldn't scale well as more such methods make into the interface.

I think your problems likely stem from having interfaces that are too large. Interfaces in Go are more flexible than in many other languages since there is no "implements" keyword that binds a struct to a particular interface. This means that you can have many more specialised interfaces that your struct may automatically satisfy.

One consequence of this is that In Go it is usually preferable to define the interface where it is consumed instead of where the implementations are. This tends to make interfaces smaller, having just the methods needed for the consumer. It also implies that testing via the interface is the wrong approach.

 

This is a bit counterintuitive. Maybe this is how golang expects developers to write, but it is generally preferable to think of an interface as a contract that must be satisfied by any struct implementing it. For e.g., io.Writer interface expects any "writable" to implement the "Write" method and so, can be passed to any method that expects an io.Writer. This is pretty much I am trying to do. I want any struct to be able to implement the ResourceManager interface and thus can be passed to any method that accepts the ResourceManager interface.

First write your tests against the struct to verify that getting one or all resources work.

Right here, is my problem. Let's assume that I do write such a struct. However, it is not obvious to me how I can test GetAllResources when it internally invokes GetResource. One solution that I found was to have package level functions get and getAll and when mocking getAll, replace the get function with a mock. This is still a bit kludgy, but for now, I don't see a better alternative.

 

Break the interface into smaller specialised ones and move them next to the consuming code. In your case you may have an endpoint that uses a GetResource method, so you define that interface. Another endpoint may need just the GetAllResources method: another interface. You can then test your endpoints by supplying a test struct that implements the smaller interface needed.

I think the solution you propose above is for a problem that is different that I asked. My question was how to test the ResourceManagerImpl struct.

 

-- Ian