go mod conflict whith v2 pre-release

[Xiangrong Fang]

Hello,

I am developing version 2 of my hap package, which is a HTTP API framework. Version 2 is not ready yet, and the latest release ls v2.0.0-alpha.2, like so:

* 9319436 - bug fix, refined action error output (HEAD -> master, tag: v2.0.0-alpha.2, origin/master, origin/HEAD)
* 2b1046a - various improvements (20 hours ago)
* 464dcb7 - example: added source IP control and global logging (tag: v2.0.0-alpha.1)
* 94d6607 - reversed finalizer execution order
* 7604144 - added global actions and finalizers
... ...

The directory structure of the entire repo is:

  ├── example/

  ├── go.mod

  ├── LICENSE

  ├── README.md

  └── v2/

              ├── go.mod

              └── example/

The go.mod under the root directory is:

module go.xrfang.cn/hap

go 1.17

while v2/go.mod looks like:

module go.xrfang.cn/hap/v2
go 1.22.1

The problem now is go failed to get the v2 pre-release, go mod tidy under a project using hap/v2:

go: downloading go.xrfang.cn/hap/v2 v2.0.0-alpha.2
go: usermgr imports
        go.xrfang.cn/hap/v2: go.mod has non-.../v2 module path "go.xrfang.cn/hap" at revision v2.0.0-alpha.2
go: usermgr imports
        go.xrfang.cn/hap/v2/api: go.mod has non-.../v2 module path "go.xrfang.cn/hap" at revision v2.0.0-alpha.2
go: usermgr imports

... ...

Using go get:

$ GOPROXY=direct go get -u -v -x go.xrfang.cn/hap/v...@v2.0.0-alpha.2
# get https://go.xrfang.cn/hap/v2?go-get=1
# get https://go.xrfang.cn/hap/v2?go-get=1: 200 OK (0.196s)
get "go.xrfang.cn/hap/v2": found meta tag vcs.metaImport{Prefix:"go.xrfang.cn/hap/v2", VCS:"git", RepoRoot:"https://e.coding.net/xrfang/go/hap.git"} at //go.xrfang.cn/hap/v2?go-get=1
mkdir -p /home/xrfang/go/pkg/mod/cache/vcs # git3 https://e.coding.net/xrfang/go/hap.git
# lock /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9.lock
# /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9 for git3 https://e.coding.net/xrfang/go/hap.git
cd /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9; git tag -l
0.002s # cd /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9; git tag -l
cd /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' refs/tags/v2.0.0-alpha.2 --
0.003s # cd /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' refs/tags/v2.0.0-alpha.2 --
cd /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9; git cat-file blob 931943650fcb745bf3219ad4bf1ca93177047a5a:go.mod
0.002s # cd /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9; git cat-file blob 931943650fcb745bf3219ad4bf1ca93177047a5a:go.mod
go: go.xrfang.cn/hap/v...@v2.0.0-alpha.2: go.mod has non-.../v2 module path "go.xrfang.cn/hap" at revision v2.0.0-alpha.2

[Jason Phillips]

I believe the hosting provider is returning a meta tag for "go.xrfang.cn/hap/v2" when it shouldn't?

  > curl https://go.xrfang.cn/hap/v2?go-get=1

    <meta name="go-import" content="go.xrfang.cn/hap/v2 git https://e.coding.net/xrfang/go/hap.git">

The above meta tag says that the "go.xrfang.cn/hap/v2" module is at the root of the repository, but it's not. That meta tag would be correct if you completely replaced your v1 module with the v2 module. I think the meta tag needs to be the following for it to work for your current v2 module:

  <meta name="go-import" content="go.xrfang.cn/hap git https://e.coding.net/xrfang/go/hap.git">

Or, the hosting provider should return a 404 (with no meta tag) for "go.xrfang.cn/hap/v2".

Compare the output from Github for a module with a major version directory:

  > curl https://github.com/wailsapp/wails/v2?go-get=1

    Not Found

[Xiangrong Fang]

Hi Jason,

Acutally I wrote the go hosting service myself. According to your comments, I modified the output now it is:

$ curl https://go.xrfang.cn/hap/v2?go-get=1
<meta name="go-import" content="go.xrfang.cn/hap git https://e.coding.net/xrfang/go/hap.git">

But it still not working:

$ GOPROXY=direct go get -v -x -u go.xrfang.cn/hap/v...@v2.0.0-alpha.2
# get https://go.xrfang.cn/hap/v2?go-get=1
# get https://go.xrfang.cn/hap/v2?go-get=1: 200 OK (0.043s)
get "go.xrfang.cn/hap/v2": found meta tag vcs.metaImport{Prefix:"go.xrfang.cn/hap", VCS:"git", RepoRoot:"https://e.coding.net/xrfang/go/hap.git"} at //go.xrfang.cn/hap/v2?go-get=1
get "go.xrfang.cn/hap/v2": verifying non-authoritative meta tag
# get https://go.xrfang.cn/hap?go-get=1
# get https://go.xrfang.cn/hap?go-get=1: 200 OK (0.007s)

mkdir -p /home/xrfang/go/pkg/mod/cache/vcs # git3 https://e.coding.net/xrfang/go/hap.git
# lock /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9.lock
# /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9 for git3 https://e.coding.net/xrfang/go/hap.git
cd /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9; git tag -l

0.008s # cd /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9; git tag -l

cd /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' refs/tags/v2.0.0-alpha.2 --

0.007s # cd /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' refs/tags/v2.0.0-alpha.2 --

cd /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9; git cat-file blob 931943650fcb745bf3219ad4bf1ca93177047a5a:go.mod

0.001s # cd /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9; git cat-file blob 931943650fcb745bf3219ad4bf1ca93177047a5a:go.mod
cd /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9; git cat-file blob 931943650fcb745bf3219ad4bf1ca93177047a5a:v2/go.mod
0.001s # cd /home/xrfang/go/pkg/mod/cache/vcs/0f263e071aed73ec9ef118d38d27fc7cbbcc33f1c19bbac5423684b8b560e3f9; git cat-file blob 931943650fcb745bf3219ad4bf1ca93177047a5a:v2/go.mod
# get https://sum.golang.org/lookup/go.xrfang.cn/hap/v...@v2.0.0-alpha.2
# get https://sum.golang.org/lookup/go.xrfang.cn/hap/v...@v2.0.0-alpha.2: 404 Not Found (1.681s)
go: go.xrfang.cn/hap/v...@v2.0.0-alpha.2: verifying go.mod: go.xrfang.cn/hap/v...@v2.0.0-alpha.2/go.mod: reading https://sum.golang.org/lookup/go.xrfang.cn/hap/v...@v2.0.0-alpha.2: 404 Not Found
server response:
not found: go.xrfang.cn/hap/v...@v2.0.0-alpha.2: unrecognized import path "go.xrfang.cn/hap/v2": reading https://go.xrfang.cn/hap/v2?go-get=1: 404 Not Found
server response: not found

Jason Phillips <jasonrya...@gmail.com> 于2024年4月17日周三 20:18写道：

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/fabOo575rLQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/8f42f5cf-a69b-43c9-92e2-b72b8e1b90c8n%40googlegroups.com.

[Jason Phillips]

It works for me, now.

  >  go get go.xrfang.cn/hap/v...@v2.0.0-alpha.2
    go: downloading go.xrfang.cn/hap/v2 v2.0.0-alpha.2
    go: go.xrfang.cn/hap/v...@v2.0.0-alpha.2 requires go >= 1.22.1; switching to go1.22.2
    go: upgraded go 1.22.0 => 1.22.1
    go: added toolchain go1.22.2
    go: added go.xrfang.cn/hap/v2 v2.0.0-alpha.2

[Xiangrong Fang]

Yeah. After clean modcache.

Thanks!

Jason Phillips <jasonrya...@gmail.com> 于2024年4月17日周三 22:41写道：

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/726cfae3-bd87-4081-b1a9-bc4ef1bbce6fn%40googlegroups.com.

[Jolyon Direnko-Smith]

Is this the correct approach to major-v-bumping a module?  I had understood the "vN" slug in the module to be a "virtual" element, signalling the deliberate introduction of an upgrade to a new major version of a dependency.  i.e. it avoids (or at least makes less likely) the chance of inadvertently taking a dependency on a new version of a module already in use that introduces breaking changes (as usually signalled by a major version bump).

i.e. the module ref "host/module/path/v2" is resolved to the repository at "host/module/path".  The /vN suffix in an import ref must match the module id in the go.mod at the revision tagged by the version specified in the import. 

For pre-release versions of a new major version, "pre-release" code module exists in a development branch for that new version, with a version (tag) similar to "v2.0.0-alpha".  If someone wishes to import that to experiment with,  they need to import (e.g.) "host/module/path/v...@v2.0.0-alpha".  Having said that, if the intention is to maintain 1.x and 2.x versions as both "current" then I could see how this v2 folder approach might work, though I struggle then to imagine how one would make sense of tagged versions of the repo (i.e. the tagged "v2.0.0" also contains a "production" release of "v1.?.?")

What happens if if a project contains code where some packages use the v1 of an imported module while other packages use v2?

With modules using the "/vN" virtual suffix this remains straightforward (I think): each major version is held separately, at the referenced version for that major version, in the mod cache.  But if using major version folder, the cached mod for the later major version also contains some version of previous major versions.  And the cached mod of earlier major versions may contain pre-release versions of later major versions.  Maybe I just need to get my head around it properly, but this feels like it could get confusing (for the wetware - i.e. humans - if not the tooling).


This is a genuine question, not intended to criticise the approach but to understand it as - if "official" - it's something I did not come across when researching this issue in the one instance in the past where I needed to bump the major version of a module.

[Xiangrong Fang]

As a matter of fact, I gained a somewhat complete understanding of the whole concept of Go modules and the go-get facade mechanism through writing a simple pkgsvr (https://xrfang.coding.net/public/go/pkgsvr/git/files). However, before this issue came up, I hadn't noticed the significance of the <meta> tag. :-) 

Your reply introduced a new aspect: how does the go-get mechanism handle packages that are not tracked in the master branch of the version control system?

Jolyon Direnko-Smith <delt...@gmail.com> 于2024年4月18日周四 08:53写道：

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/4ff4a619-7dcc-4d31-a503-9616121c19a5n%40googlegroups.com.

[Jason Phillips]

@Jolyon

> Is this the correct approach to major-v-bumping a module?

There are two approaches supported by the Go tooling. This is one of them. Neither is more correct than the other, there are pros and cons to each. Using a major version directory has the benefit of working transparently with GOPATH mode, which may be desirable for some projects. But, it has the shortcoming, as you pointed out, of making things confusing since multiple major versions are maintained in the same source tree.

Here is what the Go mod reference has to say:

> Subdirectories with a major version suffix are major version subdirectories. They may be used to develop multiple major versions of a module on a single branch. This may be unnecessary when development of multiple major versions proceeds on separate branches. However, major version subdirectories have an important property: in GOPATH mode, package import paths exactly match directories under GOPATH/src. The go command provides minimal module compatibility in GOPATH mode (see Compatibility with non-module repositories), so major version subdirectories aren’t always necessary for compatibility with projects built in GOPATH mode. Older tools that don’t support minimal module compatibility may have problems though.

@Xiangrong

> Your reply introduced a new aspect: how does the go-get mechanism handle packages that are not tracked in the master branch of the version control system?

When pulling from source control, the go command uses semantic version tags to pull in the correct version. That means if a module A imports modules B/v1.w.x and B/v2.y.z, and module B uses "major version subdirectories", the v1.w.x and v2.y.z versions that are pulled in don't need to correspond to the same commit in source control. B/v1.w.x and B/v2.y.z are treated completely independently. It's worth noting that's no different than how modules which don't use "major version subdirectories" are treated.

[Jolyon Direnko-Smith]

@Jason

That's really helpful, thanks.  I/we adopted Go long after modules were introduced and have never had to deal with GOPATH mode, which probably explains why I didn't read the aspects relating to that (or wiped them from my memory subsequently - lol).