What are the best practices for go build tools?

[cpu...@gmail.com]

Hello experts,

I'm currently using //go:generate with external and module-internal tools. All external tools are documented in tools.go:

// +build tools

package main

import (

_ "github.com/golang/mock/mockgen"

_ "github.com/mjibson/esc"

)

However, the existing setup has a couple of drawbacks:

1) I cannot do go install tools.go:

tools.go:6:2: import "github.com/golang/mock/mockgen" is a program, not an importable package

tools.go:7:2: import "github.com/mjibson/esc" is a program, not an importable package

instead I need to duplicate each tool installation in the Makefile

2) The internal tools are currently not installed in invoked during build:

//go:generate go run ../cmd/tools/decorate.go ...

 

This fails during cross-build:

GOOS=linux GOARCH=arm make

go install github.com/mjibson/esc

go install github.com/golang/mock/mockgen

Generating embedded assets

go generate ./...

...

fork/exec /var/folders/pm/q2g_c0vn1hb414nqwdcsv4fm0000gn/T/go-build741389459/b001/exe/decorate: exec format error

charger/evsewifi.go:57: running "go": exit status 1

fork/exec /var/folders/pm/q2g_c0vn1hb414nqwdcsv4fm0000gn/T/go-build892585922/b001/exe/decorate: exec format error

meter/meter.go:16: running "go": exit status 1

I've looked at https://golang.org/cmd/go/#hdr-Build_constraints but couldn't find a description of best practices or even mention of +built tools.

Much appreciated,

Andi

[Paul Jolly]

Hi Andi,

> 1) I cannot do go install tools.go:

go install takes main packages as arguments, so no, this will not
work. The tools.go file (like the example you present) is simply a way
of declaring a dependency on a main package through a file that will
never be built (hence the +build tools constraint). Even though this
file is, in normal operation, ignored by all build commands (go build,
go install etc) it _is_ seen by go mod tidy (which ignores build
constraints). Hence the requirement is considered when collating the
main module's requirements.

> instead I need to duplicate each tool installation in the Makefile

If you want to install these commands then yes, this needs to happen
somewhere (could even be a go:generate step)

However, you can simply go run $mainpkg and this will work (see below)

> 2) The internal tools are currently not installed in invoked during build:

Per my suggestion above, this is my preferred mechanism. It avoids any
unnecessary steps of ensuring the correct GOBIN, PATH etc.

> //go:generate go run ../cmd/tools/decorate.go ...
>
> This fails during cross-build:
> GOOS=linux GOARCH=arm make
> go install github.com/mjibson/esc
> go install github.com/golang/mock/mockgen
> Generating embedded assets
> go generate ./...
> ...
> fork/exec /var/folders/pm/q2g_c0vn1hb414nqwdcsv4fm0000gn/T/go-build741389459/b001/exe/decorate: exec format error
> charger/evsewifi.go:57: running "go": exit status 1
> fork/exec /var/folders/pm/q2g_c0vn1hb414nqwdcsv4fm0000gn/T/go-build892585922/b001/exe/decorate: exec format error
> meter/meter.go:16: running "go": exit status 1
>
> I've looked at https://golang.org/cmd/go/#hdr-Build_constraints but couldn't find a description of best practices or even mention of +built tools.

I'm not sure how you are getting into this situation, because go run
will build a binary for the platform on which cmd/go is running.

Unless you have GOOS and GOARCH set, in which case the go run during
the go generate phase will be building for potentially a different
platform?


Paul

[cpu...@gmail.com]

Hi Paul!

Thank you for taking the time to reply.

On Friday, September 18, 2020 at 11:33:49 AM UTC+2 Paul Jolly wrote:

Hi Andi,

> 1) I cannot do go install tools.go:

go install takes main packages as arguments, so no, this will not
work. The tools.go file (like the example you present) is simply a way
of declaring a dependency on a main package through a file that will
never be built (hence the +build tools constraint). Even though this
file is, in normal operation, ignored by all build commands (go build,
go install etc) it _is_ seen by go mod tidy (which ignores build
constraints). Hence the requirement is considered when collating the
main module's requirements.

> instead I need to duplicate each tool installation in the Makefile

If you want to install these commands then yes, this needs to happen
somewhere (could even be a go:generate step)

However, you can simply go run $mainpkg and this will work (see below)

 

Understood. Unfortunately, that clashes with the simplistic approach to cross-building below.

> 2) The internal tools are currently not installed in invoked during build:

Per my suggestion above, this is my preferred mechanism. It avoids any
unnecessary steps of ensuring the correct GOBIN, PATH etc.

> //go:generate go run ../cmd/tools/decorate.go ...
>
> This fails during cross-build:
> GOOS=linux GOARCH=arm make
> go install github.com/mjibson/esc
> go install github.com/golang/mock/mockgen
> Generating embedded assets
> go generate ./...
> ...
> fork/exec /var/folders/pm/q2g_c0vn1hb414nqwdcsv4fm0000gn/T/go-build741389459/b001/exe/decorate: exec format error
> charger/evsewifi.go:57: running "go": exit status 1
> fork/exec /var/folders/pm/q2g_c0vn1hb414nqwdcsv4fm0000gn/T/go-build892585922/b001/exe/decorate: exec format error
> meter/meter.go:16: running "go": exit status 1
>
> I've looked at https://golang.org/cmd/go/#hdr-Build_constraints but couldn't find a description of best practices or even mention of +built tools.

I'm not sure how you are getting into this situation, because go run
will build a binary for the platform on which cmd/go is running. 

Unless you have GOOS and GOARCH set, in which case the go run during
the go generate phase will be building for potentially a different
platform?

Exactly. User tried  GOOS=linux GOARCH=arm make which ended up as above. This can obviously be worked around by (in my case) make assets && GORACH/GOARM make build but it's not obvious.

 

Paul

Regards,

Andi