What is the best way to test for-select loop?

[Yicheng Qin]

This for-select loop pattern is recommended in Go(http://talks.golang.org/2013/advconc.slide#24):

```

func (s *sub) loop() {

    ... declare mutable state ...

    for {

        ... set up channels for cases ...

        select {

        case <-c1:

            ... read/write state ...

        case c2 <- x:

            ... read/write state ...

        case y := <-c3:

            ... read/write state ...

        }

    }

}

```

But we find it is hard to write test for this loop. For example, I want to test the behavior under `case <-c1`. I send the signal through `c1`, and want to check the state after the loop has handled the case and go back to select. But it is hard to tell when it is a good point to do the check. Here are several solutions that we have gone so far:

1. add c1_finish channel in the loop to notify that c1 case is done.

This is ugly because it needs to add code in loop just for debug.

2. sleep some time

This may increase test time out of control.

3. check goroutine status through runtime.Stack()

Its format is not fixed, and you cannot figure out what is the target goroutine easily.

Do you have any suggestions about how to test for-select loop best?

[Yifan Gu]

3 sounds super hacky to me. Can we poll the expected state?

- Yifan

[Peter Bourgon]

The state you change for `case <-c1` should be inspectable by some
other `case` statement. If your object looks like

func (s *loop) {
val := "uninitialized"
for {
select {
case val = <-s.c1:
case s.c2 <- val:
}
}
}

func (s *sub) write(val string) {
s.c1 <- val
}

func (s *sub) read() string {
return <-s.c2
}


Your test can look like

val := "hello"
s.write(val)
runtime.Gosched()
if want, have := val, s.read(); want != have {
t.Errorf("want %q, have %q", want, have)

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[ca...@doxsey.net]

It's difficult to provide an example without the full context, but in general I think you may be testing at too low of a level.

For example, if I were testing a program that used a TCP Listener / Conn pattern, I would do so by starting the server in a goroutine, making actual connections to it, and testing the results. It might have a select case:

    var DefaultTimeout = time.Second * 5

    func someFunc() {

      for {

        c := make(chan result)

        go doWork(c)

        select {

          case r := <-c:

          case <-time.After(DefaultTimeout):

        }

      }

    }

For testing the first select case I would have it just do a regular request / response cycle. For the timeout case I would set the DefaultTimeout to something very small, make the request and confirm that whatever I expect to happen on timeout actually happened. 

You can then rely on the code coverage tool to make sure each path is covered by some test. Even though the structure of your tests won't look anything like the structure of your code it is indeed testing everything. In general the goal is to cover the functionality not the incidental details of implementation.

Another thing to keep in mind with this is I would break up the actual implementation code (what you do for each case) from the glue code (your for / select loop). You can test the implementation in a much more straightforward way (given X function F does Y), and then only have a few tests for the code as a whole.

If you have a separate integration test suite, you may find you don't even need this outer test. Curling URLs may expose any errors you made at that level.

[Yicheng Qin]

@Peter : runtime.Gosched() is not a safe way. The fault is that if the process is running on a multiple GOMAXPROCS case, runtime.Gosched() may fail to yield because the other goroutine has processor, and continue executing before the other goroutine does its work.

> You can then rely on the code coverage tool to make sure each path is covered by some test. Even though the structure of your tests won't look anything like the structure of your code it is indeed testing everything. In general the goal is to cover the functionality not the incidental details of implementation.

> Another thing to keep in mind with this is I would break up the actual implementation code (what you do for each case) from the glue code (your for / select loop). You can test the implementation in a much more straightforward way (given X function F does Y), and then only have a few tests for the code as a whole.

This is a reasonable solution. It focuses more on high-level functionality and split the actual implementation for better readability and testing.

However, sometimes it is harder to cover all cases use these two ways IMO, e.g., the loop does some logic inside the loop, or the select statement is really big. One example:

```

func some(tickc <-chan time.Time) {

  var heartbeatc <-chan time.Time

  for {

    select {

    case <-activate:

      heartbeatc = tickc

    case <-deactivate:

      heartbeatc = nil

    case <-heartbeatc:

      (do something)

    }

  }

}

```

It includes some logic inside the loop, so it is hard to test them without starting the loop. On the other hand, it is hard to tell whether the deactivate works immediately considering select selects channel randomly.

[Jesper Louis Andersen]

On Fri, Jun 19, 2015 at 11:30 PM, Yicheng Qin <yiche...@coreos.com> wrote:

For example, I want to test the behavior under `case <-c1`. I send the signal through `c1`, and want to check the state after the loop has handled the case and go back to select.

One way around this is to have a way to observe the state of the data the goroutine is manipulating. Either by means of another channel, or by access directly to the data in some way. If you also have a way to canonicalize the data, then you can observe if your change had the desired effect.

Alternatively, you can observe the behavior of the system without observing its internal state. That is, you define a model which describes how the system should transition and what it should respond. Rather than performing a verification of the internal state, you verify it's later responses. The problem here is that you need a way to simplify the test if it fails because you don't know what poke put the system into a wrong state.

The latter way can be had with advanced variants of 'testing/quick', which to my knowledge isn't supported that well in Go. 

--

J.