[ANN] unused - check your code for unused constants, variables, functions, types and fields

[Dominik Honnef]

Hi,

I want to announce a tool I just released, called 'unused'[1]. It reports
unused identifiers in your code – for example it complains about
unused unexported functions.

There's a complete list of checks[2] in the README[3].

You can install unused via `go get honnef.co/go/unused/cmd/unused`.

The tool is somewhat similar to Rémy Oudompheng's deadcode.go[4],
however my tool uses go/types instead of go/ast and is thus able to
provide more accurate information. It has less false negatives,
handles struct fields and scopes correctly and even checks if methods
are important for interfaces or not. The downside to using go/types is
that it is slower. However, it can still check the entire standard
library in ~4 seconds, and single packages in under a second, putting
it into the same class as errcheck.

Also the tool accepts packages and wildcards as arguments, instead of
directories, and provides flags for selectively turning off checks.

I hope the tool is useful to you. Please let me know if there are any
issues!

[1]: https://github.com/dominikh/go-unused
[2]: https://github.com/dominikh/go-unused#what-counts-as-usedunused
[3]: https://github.com/dominikh/go-unused/blob/master/README.md
[4]: https://github.com/remyoudompheng/go-misc/blob/master/deadcode/deadcode.go
--
Dominik Honnef

[Jan Mercl]

On Sun, Mar 13, 2016 at 11:45 PM Dominik Honnef <dom...@honnef.co> wrote:

> I want to announce a tool I just released, called 'unused'[1]. 

Thanks for making this. I already used it and I'm happy with the results[0].

  [0]: https://github.com/cznic/cc/commit/c36d47f6f4c66ccd6ac744a7f2ad0ddf42c96a5e

--

-j

[Caleb Spare]

Thanks Dominik! I too used it to remove dead code from some of my
packages within about 10 seconds of installing it. It does exactly
what I expect.

You mention running it on the standard library and in the readme you
give an example showing dead functions in cmd/go. Are you thinking of
sending any CLs cleaning up dead code?

-Caleb

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Dmitri Shuralyov]

Thank you for making this! This is great. Another reliable and useful Go tool added to my disposal.

[Dominik Honnef]

On Sun, Mar 13, 2016 at 06:50:53PM -0700, Caleb Spare wrote:
> Thanks Dominik! I too used it to remove dead code from some of my
> packages within about 10 seconds of installing it. It does exactly
> what I expect.

Great!

>
> You mention running it on the standard library and in the readme you
> give an example showing dead functions in cmd/go. Are you thinking of
> sending any CLs cleaning up dead code?

I'm not sure. I always feel a bit bad about sending such maintenance
CLs. Feels like it causes a lot of work to the Go team for little
reward.

--
Dominik Honnef

[Dave Cheney]

Never feel bad about leaving the camp ground cleaner than you found it.

[wheelcomplex]

Great! Thanks.

[Dave Cheney]

[unused aa057c0] all: remove unused public and private symbols

 104 files changed, 32 insertions(+), 897 deletions(-)

And i've barely scratched the surface. Thanks Dominik!

[Tamás Gulácsi]

 Thanks for this great tool!

:gthomas@waterhouse: ~/src/github.com/tgulacsi/agostle

$ git commit -a

[master 5719c91] Remove unused code, thanks to honnef.co/go/unused/cmd/unused

 9 files changed, 0 insertions(+), 125 deletions(-)

No code has no bugs!

[Dominik Honnef]

On Mon, Mar 14, 2016 at 02:01:23AM -0700, Dave Cheney wrote:
>
>
> [unused aa057c0] all: remove unused public and private symbols
>
> 104 files changed, 32 insertions(+), 897 deletions(-)
>
>
> And i've barely scratched the surface. Thanks Dominik!

Wow, impressive numbers.

--
Dominik Honnef

[willie...@gmail.com]

Can you use it with a go program vs. a specific package?

[Dominik Honnef]

On Mon, Mar 14, 2016 at 06:15:13AM -0700, willie...@gmail.com wrote:
> Can you use it with a go program vs. a specific package?

The tool accepts the same syntax for packages that `go build` does.
`unused foo` will check package foo, `unused foo bar` will check
packages foo and bar, and `unused foo/...` will check all packages
under foo.

The analysis itself only looks at individual packages, however, as it
limits itself to unexported (or semi-unexported, i.e. tests)
identifiers. It doesn't do whole-program analysis to find unused
exported identifiers.
--
Dominik Honnef

[bep]

Very useful! Thanks.

bep

[Nigel Vickers]

Hallo Dominik,

I am obviously doing something wrong , 1.6 via godeb on ubuntu 10.04. Have I missed something about go/types?

sysop@godev16:~/go/src$ go get honnef.co/go/unused/cmd/unused

# honnef.co/go/unused

honnef.co/go/unused/unused.go:97: cannot use c.pkg.Info.TypeOf(t) (type "golang.org/x/tools/go/types".Type) as type "go/types".Type in return argument:

        "golang.org/x/tools/go/types".Type does not implement "go/types".Type (wrong type for Underlying method)

                have Underlying() "golang.org/x/tools/go/types".Type

                want Underlying() "go/types".Type

honnef.co/go/unused/unused.go:100: impossible type assertion:

        *"go/types".Named does not implement "golang.org/x/tools/go/types".Type (wrong type for Underlying method)

<snip much of the same>

[Dominik Honnef]

Hi,

On Mon, Mar 14, 2016 at 10:54:03AM -0700, Nigel Vickers wrote:
> Hallo Dominik,
> I am obviously doing something wrong , 1.6 via godeb on ubuntu 10.04. Have
> I missed something about go/types?
>
> sysop@godev16:~/go/src$ go get honnef.co/go/unused/cmd/unused
> # honnef.co/go/unused
> honnef.co/go/unused/unused.go:97: cannot use c.pkg.Info.TypeOf(t) (type
> "golang.org/x/tools/go/types".Type) as type "go/types".Type in return
> argument:
> "golang.org/x/tools/go/types".Type does not implement
> "go/types".Type (wrong type for Underlying method)
> have Underlying() "golang.org/x/tools/go/types".Type
> want Underlying() "go/types".Type
> honnef.co/go/unused/unused.go:100: impossible type assertion:
> *"go/types".Named does not implement
> "golang.org/x/tools/go/types".Type (wrong type for Underlying method)
> <snip much of the same>

Use go get -u honnef.co/go/unused/cmd/unused to update your Go
packages. You have an oudated version of a dependency
(golang.org/x/tools/go/loader by the looks of it).

--
Dominik Honnef

[Nico]

On 14/03/16 17:54, Nigel Vickers wrote:
> go get honnef.co/go/unused/cmd/unused

Try instead:

go get -u honnef.co/go/unused/cmd/unused

[Nigel Vickers]

Brilliant!

sysop@godev16:~/go/src$ go version

go version go1.6 linux/amd64

sysop@godev16:~/go/src$ go get -u  honnef.co/go/unused/cmd/unused

sysop@godev16:~/go/src$ unused lib

/home/sysop/go/src/lib/buildsplash.go:103:6: makeMiddle is unused

/home/sysop/go/src/lib/buildsplash.go:118:6: makeRegButton is unused

/home/sysop/go/src/lib/buildsplash.go:165:6: controller is unused

/home/sysop/go/src/lib/rhewocomps.go:2319:6: sendDBid is unused

/home/sysop/go/src/lib/rhewocomps.go:2324:6: receiveDBid is unused

[Manlio Perillo]

Il giorno domenica 13 marzo 2016 23:45:46 UTC+1, Dominik Honnef ha scritto:

Hi,

I want to announce a tool I just released, called 'unused'[1]. It reports
unused identifiers in your code – for example it complains about
unused unexported functions.

There's a complete list of checks[2] in the README[3].

You can install unused via `go get honnef.co/go/unused/cmd/unused`.

Thanks for the tool.

It seems that there is a memory leak, since when I run:

    unused ...

the program eats all my RAM:

    /home/manlio/.local/lib/go/src/launchpad.net/gozk/zookeeper/zk.go:15:23: fatal error: zookeeper.h: No such file or directory

    compilation terminated.

    /home/manlio/.local/lib/go/src/github.com/google/gopacket/pfring/pfring.go:13:20: fatal error: pfring.h: No such file or directory

    compilation terminated.

    zsh: killed     unused ...

It is possible that the leak was caused by the two compilation errors.

When I run unused over the standard library, there are no problems.

Manlio 

[Dominik Honnef]

On Mon, Mar 14, 2016 at 12:16:16PM -0700, Manlio Perillo wrote:
> Il giorno domenica 13 marzo 2016 23:45:46 UTC+1, Dominik Honnef ha scritto:
> >
> > Hi,
> >
> > I want to announce a tool I just released, called 'unused'[1]. It reports
> > unused identifiers in your code – for example it complains about
> > unused unexported functions.
> >
> > There's a complete list of checks[2] in the README[3].
> >
> > You can install unused via `go get honnef.co/go/unused/cmd/unused`

> > <http://honnef.co/go/unused/cmd/unused>.

> >
> >
> Thanks for the tool.
>
> It seems that there is a memory leak, since when I run:
> unused ...
> the program eats all my RAM:
>
> /home/manlio/.local/lib/go/src/launchpad.net/gozk/zookeeper/zk.go:15:23:
> fatal error: zookeeper.h: No such file or directory
> compilation terminated.
>
> /home/manlio/.local/lib/go/src/github.com/google/gopacket/pfring/pfring.go:13:20:
> fatal error: pfring.h: No such file or directory
> compilation terminated.
> zsh: killed unused ...
>
> It is possible that the leak was caused by the two compilation errors.
>
> When I run unused over the standard library, there are no problems.

How much (available) RAM do you have, and how many packages are in
your GOPATH? (go list ... | wc -l)

--
Dominik Honnef

[Dominik Honnef]

It's not really a leak per se. The issue is that we're constructing a
single program consisting of all the packages you pass on the command
line. I've run some tests locally, and noticed ~700 MB usage for the
standard library, and ~3500 MB for ..., which are 1969 packages.

The alternative is to construct and discard one program per argument.
The issue is that this is orders of magnitude slower because we have
to type-check the same dependencies over and over again.

Though when I've tried the "one program per package" approach, I saw
much higher memory usage, so there's probably some data being retained
somewhere. But I'm not sure it's relevant to our current approach,
which does use all the data.

I'll experiment some more and see if this can be improved.

--
Dominik Honnef

[Manlio Perillo]

Il giorno lunedì 14 marzo 2016 20:18:57 UTC+1, Dominik Honnef ha scritto:

> [...]

> > You can install unused via `go get honnef.co/go/unused/cmd/unused`
> > <http://honnef.co/go/unused/cmd/unused>.
> >
> >
> Thanks for the tool.
>
> It seems that there is a memory leak, since when I run:
>     unused ...
> the program eats all my RAM:
>    
> /home/manlio/.local/lib/go/src/launchpad.net/gozk/zookeeper/zk.go:15:23:
> fatal error: zookeeper.h: No such file or directory
>     compilation terminated.
>    
> /home/manlio/.local/lib/go/src/github.com/google/gopacket/pfring/pfring.go:13:20:
> fatal error: pfring.h: No such file or directory
>     compilation terminated.
>     zsh: killed     unused ...
>
> It is possible that the leak was caused by the two compilation errors.
>
> When I run unused over the standard library, there are no problems.

How much (available) RAM do you have, and how many packages are in
your GOPATH? (go list ... | wc -l)

My total RAM is 8GB, with usually about 3-4GB already taken by Chromium.

The total number of packages in my GOPATH is 1940.

Thanks  Manlio

[Dan Kortschak]

I get this output when running in github.com/gonum/graph.

~/Development/src/honnef.co/go/unused $ git rev-parse HEAD
15fa1da06ea10734192ed051dd43c1b9d29306e5

$ go version
go version go1.6 linux/amd64

$ unused github.com/gonum/graph/...
panic: interface conversion: ast.Expr is *ast.Ident, not
*ast.KeyValueExpr

goroutine 1 [running]:
panic(0x68e1c0, 0xc8233023c0)
/home/daniel/Development/go/src/runtime/panic.go:464 +0x3e6
honnef.co/go/unused.(*Checker).Visit(0xc82000b200, 0x7fe2dca24f70,
0xc823c8ce80, 0x0, 0x0)
/home/daniel/Development/src/honnef.co/go/unused/unused.go:132 +0x719
go/ast.Walk(0x7fe2dca24e88, 0xc82000b200, 0x7fe2dca24f70, 0xc823c8ce80)
/home/daniel/Development/go/src/go/ast/walk.go:52 +0x56
go/ast.Walk(0x7fe2dca24e88, 0xc82000b200, 0x7fe2dc9d7a50, 0xc824bf9db0)
/home/daniel/Development/go/src/go/ast/walk.go:284 +0x2e0b
go/ast.walkStmtList(0x7fe2dca24e88, 0xc82000b200, 0xc822818c00, 0xb,
0x10)
/home/daniel/Development/go/src/go/ast/walk.go:32 +0xd5
go/ast.Walk(0x7fe2dca24e88, 0xc82000b200, 0x7fe2dc9d7a80, 0xc82338ec60)
/home/daniel/Development/go/src/go/ast/walk.go:224 +0x3e1e
go/ast.Walk(0x7fe2dca24e88, 0xc82000b200, 0x7fe2dca25180, 0xc82338efc0)
/home/daniel/Development/go/src/go/ast/walk.go:344 +0xb46
go/ast.walkDeclList(0x7fe2dca24e88, 0xc82000b200, 0xc8225a7200, 0x12,
0x20)
/home/daniel/Development/go/src/go/ast/walk.go:38 +0xd5
go/ast.Walk(0x7fe2dca24e88, 0xc82000b200, 0x7fe2dc748118, 0xc822e54f80)
/home/daniel/Development/go/src/go/ast/walk.go:353 +0x29b9
honnef.co/go/unused.(*Checker).Check(0xc82000b200, 0xc820110000, 0x10,
0x10, 0x0, 0x0, 0x0, 0x0, 0x0)
/home/daniel/Development/src/honnef.co/go/unused/unused.go:236 +0xa29
main.main()
/home/daniel/Development/src/honnef.co/go/unused/cmd/unused/main.go:64
+0x236

[Caleb Spare]

Do you also have outdated dependencies, like someone earlier on this
thread? go get -u might help.

I get this output:

$ unused github.com/gonum/graph/...
2016/03/14 16:44:55 couldn't load packages due to errors:
github.com/gonum/graph/graphs/gen, github.com/gonum/graph/network,
github.com/gonum/graph/community and 3 more

-Caleb

[Dominik Honnef]

On Tue, Mar 15, 2016 at 10:09:44AM +1030, Dan Kortschak wrote:
> I get this output when running in github.com/gonum/graph.
>
> ~/Development/src/honnef.co/go/unused $ git rev-parse HEAD
> 15fa1da06ea10734192ed051dd43c1b9d29306e5
>
> $ go version
> go version go1.6 linux/amd64
>
> $ unused github.com/gonum/graph/...
> panic: interface conversion: ast.Expr is *ast.Ident, not
> *ast.KeyValueExpr

I've filed https://github.com/dominikh/go-unused/issues/16

--
Dominik Honnef

[Dan Kortschak]

:) Yeah, it helped me find those - they are fixed (we changed the import
paths in stat recently and I hadn't fixed them in graph).

If you redo the get -u you should find the same error I had. I've
narrowed it down to github.com/gonum/graph/internal/set.

[Caleb Spare]

Never mind, I needed to fetch gonum test dependencies:

$ go get -t -u github.com/gonum/graph/...

I see the crash now.

[webus...@gmail.com]

Is there any particular reason a whole-program analysis is not done. I have a rather large program I'm working on with several sub packages. I would love to see what dead code is in there (exported/not exported). Also, I would just want to care about my program and not all the other packages that I import. 

[Dominik Honnef]

On Wed, Mar 16, 2016 at 06:57:04AM -0700, webus...@gmail.com wrote:
> Is there any particular reason a whole-program analysis is not done. I have
> a rather large program I'm working on with several sub packages. I would
> love to see what dead code is in there (exported/not exported). Also, I
> would just want to care about my program and not all the other packages
> that I import.
>

It's on the todo list, but not currently a high-priority item. When I
started writing the tool, it simply wasn't a focus/didn't occur to me.

I'm not sure how much work would be involved in supporting
whole-program analysis and if it needs any fundamentally different
assumptions that we're currently not considering.

There's an open issue[1] tracking this feature request. You could
subscribe to that if you want to track its progress. I'll probably try
to implement it after I've tackled some of the more important
limitations of the tool.

[1]: https://github.com/dominikh/go-unused/issues/22


--
Dominik Honnef

[Val]

Great tool, Dominik!

Do you mean the 3500MB are mostly used by your program in-memory data, or by the go toolchain?
Did you consider a file-based or db-based persistence to reduce memory usage, and would you appreciate PRs in that direction?

Cheers,
Val

[Dominik Honnef]

On Wed, Mar 16, 2016 at 08:48:52AM -0700, Val wrote:
> Great tool, Dominik!
>
> Do you mean the 3500MB are mostly used by your program in-memory data, or
> by the go toolchain?
> Did you consider a file-based or db-based persistence to reduce memory
> usage, and would you appreciate PRs in that direction?

The memory usage is primarily in go/types, when it type-checks the
code. It's not really something we can avoid at the moment.

And no, I'm not interested in any form of persistence. The tool should
be stateless. Besides, it's unlikely to help. To get the information
we need, we have to type-check the whole program. There's nothing to
really persist there that would reduce the memory usage.

>
> Cheers,
> Val
>
> On Monday, March 14, 2016 at 8:38:04 PM UTC+1, Dominik Honnef wrote:
>
> >
> > It's not really a leak per se. The issue is that we're constructing a
> > single program consisting of all the packages you pass on the command
> > line. I've run some tests locally, and noticed ~700 MB usage for the
> > standard library, and ~3500 MB for ..., which are 1969 packages.
> >
> > The alternative is to construct and discard one program per argument.
> > The issue is that this is orders of magnitude slower because we have
> > to type-check the same dependencies over and over again.
> >
> > Though when I've tried the "one program per package" approach, I saw
> > much higher memory usage, so there's probably some data being retained
> > somewhere. But I'm not sure it's relevant to our current approach,
> > which does use all the data.
> >
> > I'll experiment some more and see if this can be improved.
> >
> > --
> > Dominik Honnef
> >
>

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
Dominik Honnef

[Dominik Honnef]

Hi,

I've just pushed a series of improvements that should improve the
accuracy of results (less false positives and negatives), detect
unused code even if it's used by other unused code, and handle broken
packages better by not failing completely and instead checking the
remaining packages separately.

Oh, and you can pass build tags now.
--
Dominik Honnef