Custom build tags with third-party libraries

[Bushnell, Thomas]

The module https://github.com/confluentinc/confluent-kafka-go takes advantage of a custom build tag to change its behavior. (Specifically, the “dynamic” build tag forces it to use a dynamic shared library -lrdkafka instead of a built-in one; this is needed sometimes because the built-in one does not support GSSAPI/Kerberos, while the dynamic one might [and on Linux distros, generally does].)

 

Suppose I am writing my own package foo which uses confluent-kafka-go under the hood to implement functionality, and I require the “dynamic” behavior from confluent-kafka-go. My package must then in turn tell its users that they must build their application with the “dynamic” build tag. This in turn propagates arbitrarily far; as well, it is easy for it to become stale (suppose Confluent changes the behavior of their module to no longer use this build tag—then all these transitive users will still have a “dynamic” build tag perhaps annoyingly hanging around for a long time).

 

Notice that the “dynamic” build tag goes together with linking with -lrdkafka, but that can be specified once, in the code close to the confluent-kafka-go package, rather than in an unbounded set of transitive reverse dependencies.

 

Relatedly, there is nothing to prevent some unrelated package from also changing its behavior based on the “dynamic” build tag, and applications in general may want the altered behavior from confluent-kafka-go but not from some unrelated package—which may even be obscure or entirely unknown.

 

I think there are several solutions to this difficulty; one might be a way to say in the go.mod file that a direct dependency should be built with a particular build tag, so that the application of the tag can be applied only to a portion of the build and not the whole. (This still leaves the possibility that a module might be used by two separate parts of a build, one with the tag and one without; this is fundamentally no different than a version conflict, but it might have some implications for other parts of the build system I haven’t considered.)

 

I think it’s obvious that it would be better for Confluent not to have built their tooling this way, but here we are. Any use of a non-standard custom build tag in a generally imported module has this problem, so I think it’s important to solve for the Go ecosystem; if the feature really is “don’t use this feature” then arguably the feature should be abandoned. If it’s to be kept as a feature, then there should be some way to use it safely.

 

Thomas

 

[Ian Lance Taylor]

Hi Thomas. I think the issue here is that build tags are basically a
global mechanism. They work OK as long as they are describing
something that applies across the entire program. Besides the standard
build tags, this includes popular semi-standardized tags like purego.
Package-specific build tags don't really work. I guess I'm saying
"don't use this feature," at least not in this way.

The standard library does have at least some package-specific build
tag, but at least it puts the package name in the build tag name, like
netgo, netcgo, timetzdata.

Given where we are today, my best suggestion for a package that
requires an imported package to use a build tag would be something
like

package mypackage

//go:build !dynamic

func init() {
log.Fatal("program must be built with -tags=dynamic")
}

I agree that that is really not satisfactory.

Ian

[Bushnell, Thomas]

Thanks Ian. This is more or less what I expected. Is there any hope for extending the current mechanism to provide a per-import or per-module-in-go.mod way to specify a build tag?

Thomas

-----Original Message-----
From: Ian Lance Taylor <ia...@golang.org>
Sent: Wednesday, February 19, 2025 5:36 PM
To: Bushnell, Thomas <Thomas....@deshaw.com>
Cc: golan...@googlegroups.com
Subject: Re: [go-nuts] Custom build tags with third-party libraries

This message was sent by an external party.

[Ian Lance Taylor]

On Thu, Feb 20, 2025 at 8:09 AM Bushnell, Thomas
<Thomas....@deshaw.com> wrote:
>
> Thanks Ian. This is more or less what I expected. Is there any hope for extending the current mechanism to provide a per-import or per-module-in-go.mod way to specify a build tag?

I would be concerned about the problem you pointed out: what should
happen if the same dependency appears multiple times, some with a
specified build tag and some without? There is no way to resolve that
conflict.

I don't see any problem with specifying build tags in the go.mod file
of the main package. Those tags would be used when building that
program. It would be a limited per-project GOFLAGS. But that wouldn't
help the case of your package foo, which requires a build tag to be
used when building a different package.

I'm not an expert in this area. Perhaps somebody else has an idea.

Ian

[robert engels]

I just ran into this today with a library of mine. A user has a PR to provide a "strict" mode that has an unfortunate performance impact. So the idea was to use a build tag to enable strict mode.

The problem is that he has a downstream project with no way to enforce the build tag being set.

Why can't go.mod in the import declaration allow the setting of build (additional) tags that the referenced module should be built with?

This seems like a trivial change with wide applicability. As it is now, we will end up creating methods like XXXStrict which is ugly as hell. I found several issues related to this (as far back at 2020 - all closed), and I am reminded again of the 'we don't need it, so you don't either' mentality of the Go team at times and it's incredibly frustrating when you run into it.

Maybe the Go team has a suggestion of how best to address this? As to the concerns of 'one import wanting it, another doesn't' - shouldn't that be treated the same as different version requirements? (i.e. the import build tags must match)

Thanks,

Robert

[Mike Schinkel]

I have been running into an equivalent thing, literally yesterday, but for GOEXPERIMENT=jsonv2.  

Clearly some experiments will build without or without the setting (like GreenTeaGC, I assume) but others like jsonv2 are specific to the source code and the source code cannot be compiled without them. 

For those experiments it seems like that indications should be allowed to be declared in go.mod.

-Mike

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/daa432b8-334c-4941-be4a-6eeeedc7b352n%40googlegroups.com.