Go modules replace statements referencing legacy codebases

[Jim Minter]

Hi,

Using Go 1.14, I'm working on a parent codebase which, in its go.mod
file, has a number of replace statements referencing legacy child
codebases which have not yet converted to Go modules.

At this stage the legacy child codebases handle versioning by branch,
rather than using semantic versioning, so my go.mod file has a number of
statements like these:

> replace github.com/foo/bar => github.com/foo/bar v0.0.0-2020xxxxxxxxxx-hhhhhhhhhhhh // should track branch baz
> replace github.com/foo/bar1 => github.com/foo/bar1 v0.0.0-2020yyyyyyyyyy-iiiiiiiiiiii // should track branch baz

My goal is to be confident that if I update dependencies in the parent
codebase (go get -u ./...), I'll be pulling in HEAD on the relevant
branch of all the legacy child dependencies.

My expectation is that this is a transitional use case because
eventually the child codebases will migrate to Go modules, but until
then, it's important to me to know I will pull in critical fixes from
the child dependencies if they happen.

The difficulty I've got is ensuring that the replace stanzas are valid
and up-to-date. I'm not finding a great way to do this with the Go
module tooling and I wonder if I'm missing anything. Here are the
feasible ugly alternatives I've found so far:

1. Child by child, abuse the fact that `go mod tidy` rightly or wrongly
seems to tolerate a single legacy branch reference in the file at a time
and rewrites it to the v0.0.0-2020xxxxxxxxxx-hhhhhhhhhhhh format:

> go mod edit -replace github.com/foo/bar=github.com/foo/bar@baz
> go mod tidy
> go mod edit -replace github.com/foo/bar1=github.com/foo/bar1@baz
> go mod tidy
> ...etc...
> go get -u ./...

2. A script to be run manually (or via go generate? yikes!) before
running go get -u ./..., containing abominations like this:

> go mod edit -replace github.com/foo/bar=$(go list -mod=mod -m github.com/foo/bar@baz | sed -e 's/ /@/')
> go mod edit -replace github.com/foo/bar1=$(go list -mod=mod -m github.com/foo/bar1@baz | sed -e 's/ /@/')
> ...etc...
> go get -u ./...

3. Alternative abominations like this:

> go mod edit -replace github.com/foo/bar=github.com/foo/bar $(curl -s https://proxy.golang.org/github.com/foo/bar/@v/baz.info | jq -r .Version)
> go mod edit -replace github.com/foo/bar1=github.com/foo/bar1 $(curl -s https://proxy.golang.org/github.com/foo/bar1/@v/baz.info | jq -r .Version)
> ...etc...
> go get -u ./...

I'm wondering: is there a better way? Should there be?

Many thanks!

Jim Minter

[seank...@gmail.com]

I think you should be able to do `go get ./... dep1@branchA dep2@branchB ...`

[Jim Minter]

Sadly not.  Doing this doesn't cause an error, but it also doesn't change the relevant replace directive in the go.mod file.

Jim

[roger peppe]

I don't quite understand why you're using replace directives here rather than just declaring the pseudo-version as a requirement.

Why wouldn't that work?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/922448f0-00cf-7f7e-efdc-50965aefc6d4%40minter.uk.

[Jim Minter]

Hi,

I don't fully understand why we have to use replace directives (I'm
still learning my way around go modules), but I have tried extensively
to remove them and I can't get go mod to honour an acceptable set of
versions in the requirement.

I am guessing at a few potential reasons why this situation may be; some
guesses may be wrong, and I do recognise that the cases are all horrible:

1- There are cases where packages in the imported libraries end in paths
like /v1 (but where this predates and has nothing to do with go
modules). I don't know if that can confuse the go module version
decision logic.

2- There are cases where the package repos have git tags in the form
vX.Y.Z which also predate and have nothing to do with go modules. I
don't know if that can confuse things either.

3- There are quite potentially errors in the defined package
inter-dependencies which are driving the go module decision logic to
resolve to use versions which are unacceptable to me; I have no idea.

4- There are also cases (I tried to omit this in my original mail in an
attempt to simplify) where an imported package is a fork where the
original package has gone away (e.g. bitbucket.org/ww/goautoneg) or
where a genuine fork has been made but neither the fork nor the code
that imports it has yet been updated with the new import paths.

All of this is pretty disgusting, but it's also completely outside my
control to get fixed quickly. I think it's a real world example of
complexities and technical debt involved in a large scale system
involving multiple separate teams which are themselves trying to migrate
to go modules and whose release timescales slow down getting fixes out
of the door (oh, for a monorepo!).

I guess I'm trying to work out the least worst way of working around
this sort of thing as a lowly package importer.

Cheers,

Jim