Unix binary installations for Go

[rpo...@google.com]

Hi,

I am wondering what is the recommended way to package a Unix binary written in Go.

What I want, to adhere to standard Unix conventions:

- Something like "./configure --prefix=/opt/foo" that "does something" to set a final install location. It should also autodetect possible optional dependencies and set up "some" infrastructure to use or not use them (e.g. conditional compilation by enabling/disabling inclusion of a subpackage that self-registers via an init() function or by having multiple source files with different build tags defining the same interface). As we're talking about Go which already has "go get", this should have the option (probably enabled by default, but to be disabled by distribution package maintainers to make builds hermetic) to download missing packages.

- Something like "make". Well, that'd be "go build". Easy.

- Something like "make install" that copies generated executables to /opt/foo/bin, while defaulting to /usr/local/bin (and probably everything else to $GOPATH) but _will never compile_ so it's safe to run via sudo without potentially messing up the source tree with root-owned files. Too bad "go install" first requires the user to set up GOBIN...

- It should feel to the package's user just like one of the wellknown standard build systems like autoconf (e.g. waf and meson feel "sufficiently like this") or cmake (the weird one out there but it's becoming common enough that it can be assumed that users have seen it before). Why? I don't believe the mere user of a package is supposed to learn a different build system for each project.

- Of course stuff like cross compiling and installing into a staging location (e.g. via $DESTDIR) should be supported and easy.

- While at it, it should also be able to install config files into $prefix/etc, and make the install prefix "somehow known" to the compiled program so it can know where to read its config files from. And documentation goes into $prefix/doc, helper binaries into $prefix/libexec/somesubdir, etc...

Now I know I can probably build all this with my own shell script hackery; after all, "Makefile.in" only would have to be a simple wrapper with something like

DESTDIR =

all:

  go build

install:

  install binary $(DESTDIR)@bindir@/binary

However, I'd prefer something more "standardized".

Stuff I could already find:

https://github.com/mutse/go-cmake (seems some hackery but should work... just seems bad to have to copy the Go-specific parts into each specific project)

https://github.com/google/blueprint (maybe? can't quite tell if this also has a "make install" like functionality)

https://medium.com/palantir/g%C3%B6del-a-build-system-for-the-go-programming-language-9f7d2a4974e8 (has all but installing)

https://gist.github.com/dnishimura/2961173 (seems to try to solve the same problem, but far from done and assumes GOPATH, and no cross compilation)

https://sohlich.github.io/post/go_makefile/ (more of a tutorial how to somehow integrate with make than a solution, but still, good try; lacks "install" of course, and cross compilation is very... specific)

https://github.com/dreadl0ck/zeus (doesn't solve the problem at all, has same problem as raw make: you create your own command names and usage convention, do not want)

Rudolf

[Ronny Bangsund]

Which Unix/derivative? The great thing about them is that they have so many standards to choose from ;)

"go build" isn't sufficient for most of my projects, if I want proper distribution of binaries. I might use "go generate" to create some data/source a program depends on, and I always embed the version tag as returned by "git describe --tags <branch> --abbrev=0"  in the binaries with the -X flag. That requires a build script of some sort.

If you package with any common Linux distro's tools it's expected that the packager has a full build environment. It would of course be useful if you included a script to "go get" everything the build needs, and that can usually be done in pre-build hooks in these packaging systems.

If you're building server apps maybe you'd consider packaging as containers instead. If they're command line tools you're still going to have to consider the target OS and its conventions, but you can make the binary quite self-sufficient with per-platform source files or/and go generated settings, so that they look for or create a configuration file.

I don't think the packaging itself is a problem, as there are tools like dh-make-golang to massage Go into Debian-derivative packages. You just need to specify and dependencies like databases, discovery services and so on you require.

For my own stuff I make the programs themselves flexible enough that there aren't any hard requirements for configuration files and their locations, but Linux builds have a fallback to /etc/<appname>/ for server stuff, $HOME/.<appname>/ for CLI tools, and macOS builds use $HOME/Library/Application Support/<appname>/. I always leave in the option to specify an alternative path via -C or --config.

The config files define where to load all the other data and any logging options (/var/ and /log/ are possibilities there).

[Rudolf Polzer]

On Fri, Dec 14, 2018 at 10:44 AM Ronny Bangsund <ronny.b...@gmail.com> wrote:

Which Unix/derivative? The great thing about them is that they have so many standards to choose from ;)

Linux, and it should support the FHS :)

 

"go build" isn't sufficient for most of my projects, if I want proper distribution of binaries. I might use "go generate" to create some data/source a program depends on, and I always embed the version tag as returned by "git describe --tags <branch> --abbrev=0"  in the binaries with the -X flag. That requires a build script of some sort.

Yeah, that too.

 

If you package with any common Linux distro's tools it's expected that the packager has a full build environment. It would of course be useful if you included a script to "go get" everything the build needs, and that can usually be done in pre-build hooks in these packaging systems.

Right, but the thing is - distribution packagers may be OK to have to do some adjustments; I mainly want to target users who want to build and install the tool "just like any other tool". Basically, the user shouldn't notice that it's written in Go - it shouldn't be weird compared to anything else they compile and install.

 

If you're building server apps maybe you'd consider packaging as containers instead. If they're command line tools you're still going to have to consider the target OS and its conventions, but you can make the binary quite self-sufficient with per-platform source files or/and go generated settings, so that they look for or create a configuration file.

I don't think the packaging itself is a problem, as there are tools like dh-make-golang to massage Go into Debian-derivative packages. You just need to specify and dependencies like databases, discovery services and so on you require.

dh-make-golang is actually a good hint, I'll have to check what that does - whether that e.g. uses something like I want, or contains all logic in itself.

 

For my own stuff I make the programs themselves flexible enough that there aren't any hard requirements for configuration files and their locations, but Linux builds have a fallback to /etc/<appname>/ for server stuff, $HOME/.<appname>/ for CLI tools, and macOS builds use $HOME/Library/Application Support/<appname>/. I always leave in the option to specify an alternative path via -C or --config.

Even with that, I still want an install procedure that supports stuff like --prefix and installs there. Config files indeed are the smallest problem (and if I want, I can even rely on XDG for their locations rather than using the --prefix).

 

The config files define where to load all the other data and any logging options (/var/ and /log/ are possibilities there).

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/ycHR0LkdhL0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

No need to handle this message off-hours. E-Mail isn't realtime communication. IM if urgent.

Default Borgmon graphs to Dygraph: go/nebgua-dygraph-extension

No more "borgcfg production/borg/.../foo.borg update"! Just run uborgcfg! go/uborgcfg