testing code that uses ioutil.ReadDir?

[K Richard Pixley]

I have some code.  It uses ioutil.ReadDir which returns a possible error.

My testing is at 99% code coverage because I can't figure out a way to
set up a situation in which os.Open succeeds on a directory, but
ioutil.ReadDir fails.

I can get to 100% code coverage if I throw away the err rather than
testing for it and reporting it but that seems a bit... well...
disingenuous.  I've looked down the calls to the kernel call
getdirentries but the only errno I can see being relevant to a go
program is EIO and I can't think of a way to force that to happen on a
real file system.

I can imagine a fake file system created specifically for this purpose,
but that seems an awfully long way around the barn.

How can I get to 100% code coverage?  Or should I just give up on
finding a way to cover that last, single line of error handling code?

[Ian Lance Taylor]

I'm not proud. Here is an example program for which os.Open(dirname)
succeeds but ioutil.ReadDir(dirname) fails. You may have to adjust
pathMax and nameMax for your system. Whether you actually want to use
this technique is left to you.

Ian

[Tom Payne]

Testing code that uses the os and ioutil packages is tricky. I created

  https://pkg.go.dev/github.com/twpayne/go-vfs

  https://pkg.go.dev/github.com/twpayne/go-vfs/vfst

to make it easier.

Key features:

- Everything eventually calls the underlying os/ioutil function, so you get real behavior, not fake (and possibly incorrect) mock behavior.

- Makes it really easy to create, populate, and tear down temporary directories for hermetic tests.

It's best demonstrated by example:

  https://pkg.go.dev/github.com/twpayne/go-vfs/vfst?tab=doc#example-NewTestFS

Using vfs.FS you can create instances that fail in whatever way you want, for example:

package main

import (

"os"

"testing"

"github.com/twpayne/go-vfs"

"github.com/twpayne/go-vfs/vfst"

)

// a readDirFailer wraps a vfs.FS but all calls to ReadDir fail with

// os.ErrPermission.

type readDirFailer struct {

vfs.FS

}

func (readDirFailer) ReadDir(dirname string) ([]os.FileInfo, error) {

return nil, os.ErrPermission

}

// testReadDirFailer tests that calling fs.ReadDir returns an error.

func testReadDirFailer(t *testing.T, fs vfs.FS) {

if _, err := fs.ReadDir("/"); err == nil {

t.Errorf("expected error from ReadDir")

}

}

func TestX(t *testing.T) {

fs, cleanup, err := vfst.NewTestFS(nil)

if err != nil {

t.Fatal(err)

}

defer cleanup()

testReadDirFailer(t, readDirFailer{FS: fs})

}

HTH,

Tom

[Kevin Malachowski]

Is there a particular reason you want 100% code coverage?

Not trying to start a flame war: given limited time and effort, unless the rest of my projects had 100% coverage already, I would personally spend my time working with a package with lower coverage, or just fixing known bugs. If your codebase has no bugs and otherwise full coverage... I have to say I'm jealous :)

In any case, one way to test your application's handling of an error from that function is to fake it out during unit tests (simple example, let me know if you want a more thorough one: https://play.golang.org/p/uKGFLYVlQxz). Of course this does not test the "integration" of your application code and ioutil.ReadDir, but IMHO trying to get extremely high code coverage in non-unit tests is really a time sink.

[Rob Pike]

I am the author of the test coverage tool, and I do not believe that 100% code coverage is the right goal. Coverage is a proxy for testing quality, but not a guarantee of it. Getting to 100% coverage is a better indicator of chasing metrics than of actually writing good tests. I'm happy if my coverage gets into the 90% range, but then I try to write a lot of if statements protecting against the impossible. 

If you really do need to verify that your code works when the impossible happens, you can do that. Say you have code like this:

err := neverFails()

if err != nil {

   zounds()

   return err

}

and you want to verify that zounds gets called when the impossible failure occurs. That's a reasonable thing to want to do, and you can test for that by writing instead:

err := neverFails()

if failureTrigger || err != nil {

   zounds()

   return err

}

Then you declare the variable failureTrigger and in the test write:

func TestWhatIfNeverIsToday(t *testing.T) {

   failureTrigger = true

   defer func() { failureTrigger = false }

   call code....

   test that zounds is invoked.

}

-rob

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1908d64d-fb3f-4d73-9302-b215c49dac36%40googlegroups.com.

[K Richard Pixley]

On 4/12/20 6:55 AM, Kevin Malachowski wrote:
> [External Email. Be cautious of content]

>
>
> Is there a particular reason you want 100% code coverage?

In this case, I'm writing code that will be the introduction of go to a
team of which I am now a fresh member.  100% code coverage opens eyes. 
And the fact that it is even plausible for go packages is, I suspect, a
selling point to management.

But I agree that it can easily be a time sink.  I haven't done it before
so I'm still trying to overdo it before settle into what I will
eventually decide is reasonable.

Thanks.

[K Richard Pixley]

I like this solution better than Ian's, thank you.

If I can't eliminate an error, or fake it, add a new one.  :).

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAOXNBZRz_7%3DQpWtXv%2BE1VYHvRS_SjNb%2B8-1MXOeB7Xk8r_rb3Q%40mail.gmail.com.

[K Richard Pixley]

On 4/10/20 4:15 PM, Ian Lance Taylor wrote:
> I'm not proud. Here is an example program for which os.Open(dirname)
> succeeds but ioutil.ReadDir(dirname) fails. You may have to adjust
> pathMax and nameMax for your system. Whether you actually want to use
> this technique is left to you.

Thank you.  I hadn't thought of exploiting pathname limits.

I'll probably use Rob's solution presented a couple messages down, but I
appreciate hearing this solution too.

[Amnon Baron Cohen]

"Coverage is a proxy for testing quality, but not a guarantee of it. 
Getting to 100% coverage is a better indicator of chasing metrics than of actually writing good tests"

Wise words indeed.
I'm going to print out this quote and frame it.
Thanks Rob! 

[Robert Engels]

I don’t think littering your code with state variable and branches just to test is in anyway a good (or sustainable approach) approach.  I have never seen any large scale project that did this - but could be my ignorance. 

On Apr 13, 2020, at 12:13 PM, 'K Richard Pixley' via golang-nuts <golan...@googlegroups.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/920f13fe-48ed-bbcc-926c-9351877b4180%40juniper.net.

[K Richard Pixley]

If they end up "littering", then I'll agree.  So far, I just need the
one... hold my beer...