GO111MODULE=off go get deprecated?

[Jeffery Carr]

Has this been deprecated or maybe it is broken in debian sid, but:

bash$ GO111MODULE=off go get -v go.wit.com/apps/test
go: modules disabled by GO111MODULE=off; see 'go help modules'
basj$ go version
go version go1.22.0 linux/amd64

this doesn't work anymore. Also, 'go help modules' is less than helpful.

Is there some other way to download the actual sourcecode into ~/go/src/mything/ anymore?

This functionality was _PERFECT_ in my experience. I can't even imagine why it would be deprecated. How in the world are you all developing complicated software using GO without this working? Is this why everyone has gazillions of internal/ packages and massive git repos?

Thanks in advance for any advice,

jcarr

[Ian Lance Taylor]

On Sun, Mar 3, 2024 at 1:25 PM Jeffery Carr <basil...@gmail.com> wrote:
>
> Has this been deprecated or maybe it is broken in debian sid, but:
>
> bash$ GO111MODULE=off go get -v go.wit.com/apps/test
> go: modules disabled by GO111MODULE=off; see 'go help modules'
> basj$ go version
> go version go1.22.0 linux/amd64
>
> this doesn't work anymore. Also, 'go help modules' is less than helpful.
>
> Is there some other way to download the actual sourcecode into ~/go/src/mything/ anymore?
>

I think you may be seeing a bad error message. `go get
go.wit.com/apps/test` doesn't work whether you are in module mode or
not. With Go 1.21 I get

> GO111MODULE=off ~/go1.21/bin/go get -v go.wit.com/apps/test
package go.wit.com/apps/test: unrecognized import path
"go.wit.com/apps/test": parse https://go.wit.com/apps/test?go-get=1:
no go-import meta tags ()

I filed https://go.dev/issue/66080 for the odd difference in error messages.

> This functionality was _PERFECT_ in my experience. I can't even imagine why it would be deprecated. How in the world are you all developing complicated software using GO without this working? Is this why everyone has gazillions of internal/ packages and massive git repos?

Most people simply use modules with no internal packages and with git
repos that contain only their own code. That's kind of the point of
modules, in fact. When I want to see the source code of an imported
module, I build the package using `go build`, and then look in
$GOPATH/pkg/mod.

Ian

[Jeffery Carr]

I guess this boils down to:

GO111MODULE=off go get go.uber.org/zap

used to git clone into ~/go/src/go.uber.org/zap

How am I supposed to do that now? I assume I'm not expected to parse the HTML for the go-import and go-source lines!

jcarr

[Christian Stewart]

Hi Jeffery,

You can build large projects with many repos using Go modules.

Firstly, you can reference the other repos and use the latest "master"
version by writing "master" where the v.... verson is in the file,
then use "go mod tidy"

Second, you can use "go work" to create a workspace with multiple go
modules in it, so that you can develop with them without having to
constantly update the go.mod versions.

Referencing private repositories can be done with GOPRIVATE:

export GOPRIVATE=github.com/myorg

Also edit your ~/.gitconfig to use ssh for your organization private repos:

[url "ssh://g...@github.com/myorg/"]
insteadOf = https://github.com/myorg/

Hope that helps.
Christian

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/41de28be-a36b-4c2e-8daa-3bbf55cb5b3en%40googlegroups.com.

[Jeffery Carr]

I don't think I understand you.

How can I download go.uber.org/zap sources? I need to be able to do it using the "go" binary. I was able to do it before, now I can't. Is there some way to do

go download go.uber.org/zap and have it git clone?

I don't know how to resolve the git clone path.

[Jeffery Carr]

"You can build large projects with many repos using Go modules. "

Also, I forgot to add, lots of my repos are binaries, so module things don't apply. How do I tell someone to download go.wit.com/apps/helloworld? I can't now with a single GO111MODULE=off go get. Can we have 'go download' or can go get be reverted to the prior behavior. I don't understand what is wrong with the way things worked before. Does it break something for someone?

jcarr

[Kurtis Rader]

On Sun, Mar 3, 2024 at 5:05 PM Jeffery Carr <basil...@gmail.com> wrote:

"You can build large projects with many repos using Go modules. "

Also, I forgot to add, lots of my repos are binaries, so module things don't apply. How do I tell someone to download go.wit.com/apps/helloworld? I can't now with a single GO111MODULE=off go get. Can we have 'go download' or can go get be reverted to the prior behavior. I don't understand what is wrong with the way things worked before. Does it break something for someone?

> git clone https://git.wit.org/gui/helloworld

Cloning into 'helloworld'...
remote: Enumerating objects: 55, done.
remote: Counting objects: 100% (55/55), done.
remote: Compressing objects: 100% (53/53), done.
remote: Total 55 (delta 23), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (55/55), 18.66 KiB | 398.00 KiB/s, done.
Resolving deltas: 100% (23/23), done.

> ls helloworld

LICENSE  Makefile  debugger.go  main.go

-- 

Kurtis Rader

Caretaker of the exceptional canines Junior and Hank

[Jeffery Carr]

Yes, I do understand the go-source/go-import part, but  am I missing something? I don't understand how you would find the go-source other than parsing the HTML

I mean 'go get' used to do that for me, do I now have to write a program to do that? I don't understand why that is some bad idea. Yes, we can look at the curl output, but I need it to be programatic (I have lots of these repos so typing them in by hand is not really an option).

Thanks,

jcarr

[Jeffery Carr]

I guess what I would add, is that it seems fair to say that the go-import / go-source method of versioning is part of how GO works. It's part of the GO API so to speak If I have to parse that output to clone the source repo myself, then if GO changes, I have to change my code and it seems like it's better if the standard is defined by GO in the first place. Either way, I have to do it now (parse the HTML) and it would have been nice to have a warning before I updated this morning to 1.22 and had it fail that the GO111MODULE go get behavior was going to be deprecated in this release.

jcarr

[Bryan C. Mills]

Per https://go.dev/doc/go1.22#go-command:

> go get is no longer supported outside of a module in the legacy GOPATH mode (that is, with GO111MODULE=off). Other build commands, such as go build and go test, will continue to work indefinitely for legacy GOPATH programs.

This was discussed in https://go.dev/issue/60915, which simultaneously proposed to preserve `go build` and `go test` and to finish removing `go get` in GOPATH mode. (That proposal was a change to the plan stated in https://go.dev/blog/go116-module-changes, which said: “We plan to drop support for GOPATH mode in Go 1.17.” That clearly did not happen as planned in Go 1.17.)

The simplest way to download Go dependencies for code review and debugging is to run `go mod vendor` in your module or (newly added) `go work vendor` in your workspace. Also note that `go mod download -json` and `go list -m -json` now include origin information in most cases:

~$ go list -json -m go.uber.org/z...@v1.27.0
{
        "Path": "go.uber.org/zap",
        "Version": "v1.27.0",
        "Time": "2024-02-20T20:55:06Z",
        "GoMod": "/usr/local/google/home/bcmills/pkg/mod/cache/download/go.uber.org/zap/@v/v1.27.0.mod",
        "GoVersion": "1.19",
        "Origin": {
                "VCS": "git",
                "URL": "https://github.com/uber-go/zap",
                "Hash": "fcf8ee58669e358bbd6460bef5c2ee7a53c0803a",
                "Ref": "refs/tags/v1.27.0"
        }
}

[Jeffery Carr]

On Monday, March 4, 2024 at 8:44:14 AM UTC-6 Bryan C. Mills wrote:

Per https://go.dev/doc/go1.22#go-command:

> go get is no longer supported outside of a module in the legacy GOPATH mode (that is, with GO111MODULE=off). Other build commands, such as go build and go test, will continue to work indefinitely for legacy GOPATH programs.

This was discussed in https://go.dev/issue/60915, which simultaneously proposed to preserve `go build` and `go test` and to finish removing `go get` in GOPATH mode. (That proposal was a change to the plan stated in https://go.dev/blog/go116-module-changes, which said: “We plan to drop support for GOPATH mode in Go 1.17.” That clearly did not happen as planned in Go 1.17.)

Thank you for the helpful link to 60915.

GO111MODULE=off go get <go-import-path> I used because it was a very easy way to git clone my programs and all the packages in the correct go import paths.

If my go.work file is:

go 1.22.0

use (
mygit.test.org/apps/myapp
mygit.test.org/lib/myapppackage1

myothergitserver.org/lib/myotherfunthing2

friendsproject.org/someotherpackage3

)

How do I git clone the repos? I just don't seem to understand what I'm supposed to do. 

I made the go path mapping to the git clone paths, so obviously I can write something to do this, but using 'go get' before to do the git clone was really easy! Especially since it got all the dependencies also.

Again, thanks for the help,

jcarr

BTW,

go list -json -m go.uber.org/zap@latest

@latest doesn't show the URL so one can't really get the current URL sent from the go-import/go-source lines.  Actually, it doesn't seem like "go list" hits the external authoritative server at all and maybe only looks at the local ~/go/pkg/ files.

 

The simplest way to download Go dependencies for code review and debugging is to run `go mod vendor` in your module or (newly added) `go work vendor` in your workspace. Also note that `go mod download -json` and `go list -m -json` now include origin information in most cases:

~$ go list -json -m go.uber.org/z...@v1.27.0

...

[Bryan C. Mills]

On Mon, Mar 4, 2024 at 8:52 PM Jeffery Carr <basil...@gmail.com> wrote:

On Monday, March 4, 2024 at 8:44:14 AM UTC-6 Bryan C. Mills wrote:

Per https://go.dev/doc/go1.22#go-command:

> go get is no longer supported outside of a module in the legacy GOPATH mode (that is, with GO111MODULE=off). Other build commands, such as go build and go test, will continue to work indefinitely for legacy GOPATH programs.

This was discussed in https://go.dev/issue/60915, which simultaneously proposed to preserve `go build` and `go test` and to finish removing `go get` in GOPATH mode. (That proposal was a change to the plan stated in https://go.dev/blog/go116-module-changes, which said: “We plan to drop support for GOPATH mode in Go 1.17.” That clearly did not happen as planned in Go 1.17.)

Thank you for the helpful link to 60915.

GO111MODULE=off go get <go-import-path> I used because it was a very easy way to git clone my programs and all the packages in the correct go import paths.

If my go.work file is:

go 1.22.0

use (
mygit.test.org/apps/myapp
mygit.test.org/lib/myapppackage1

myothergitserver.org/lib/myotherfunthing2

friendsproject.org/someotherpackage3

)

How do I git clone the repos? I just don't seem to understand what I'm supposed to do. 

Part of the intent with modules is that you do not need to do a full `git clone` of each dependency — you should really only need to do that if you need to author upstream changes yourself, in which case you can use `git clone` directly (instead of having the `go` command do that).

I made the go path mapping to the git clone paths, so obviously I can write something to do this, but using 'go get' before to do the git clone was really easy! Especially since it got all the dependencies also.

Again, thanks for the help,

jcarr

BTW,

go list -json -m go.uber.org/zap@latest

@latest doesn't show the URL so one can't really get the current URL sent from the go-import/go-source lines.  Actually, it doesn't seem like "go list" hits the external authoritative server at all and maybe only looks at the local ~/go/pkg/ files.

The Go proxy doesn't necessary serve origin information for previously-fetched modules, but you can force the use of the upstream git server using `GOPROXY=direct`:

$ go clean -modcache
$ GOPROXY=direct go list -json -m go.uber.org/zap@latest

{
        "Path": "go.uber.org/zap",
        "Version": "v1.27.0",

        "Query": "latest",

        "Time": "2024-02-20T20:55:06Z",

        "GoMod": "/tmp/tmp.9jWBoIG4kn/.gopath/pkg/mod/cache/download/go.uber.org/zap/@v/v1.27.0.mod",
        "GoVersion": "1.19",
        "Origin": {

                "VCS": "git",
                "URL": "https://github.com/uber-go/zap",

                "Hash": "fcf8ee58669e358bbd6460bef5c2ee7a53c0803a",
                "TagSum": "t1:nyhl6oz//aQZM1r8apo5oy5HcHnpycepirvxAqo5MIE=",
                "Ref": "refs/tags/v1.27.0"
        }
}

 

 

The simplest way to download Go dependencies for code review and debugging is to run `go mod vendor` in your module or (newly added) `go work vendor` in your workspace. Also note that `go mod download -json` and `go list -m -json` now include origin information in most cases:

~$ go list -json -m go.uber.org/z...@v1.27.0
...

 

                "VCS": "git",
                "URL": "https://github.com/uber-go/zap",

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/mxP3SKA5QoM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/17d6e3dd-4687-4474-b2cd-f44aa702f436n%40googlegroups.com.

[Jeffery Carr]

On Tuesday, March 5, 2024 at 9:29:58 AM UTC-6 Bryan C. Mills wrote:

Part of the intent with modules is that you do not need to do a full `git clone` of each dependency — you should really only need to do that if you need to author upstream changes yourself,

Yes, I'm authoring upstream changes myself on my own code as I tinker. Nowadays, I usually just make 1 repo per package. That was easier before as go get would recursively clone all of them.

 

in which case you can use `git clone` directly (instead of having the `go` command do that).

Okay, no problem, I'll make something to automate that, so I'm all good. Others might not be so happy with go version 1.22 since it is not a simple replacement method.

For users like me, having the prior release have fmt.Println("Use of GO111MODULE=off" will be deprecated in 1.22") might have prompted slackers like me that procrastinate until the last minute to attempt to submit patches or fix their things!

The Go proxy doesn't necessary serve origin information for previously-fetched modules, but you can force the use of the upstream git server using `GOPROXY=direct`:

$ go clean -modcache
$ GOPROXY=direct go list -json -m go.uber.org/zap@latest

Someone on irc was also cool and pointed to internal/vcs and friends. It would be ideal to use the same code go uses to resolve the git clone path.

As you pointed out the use of the go.work file,  I'll have to make something that works the old way and the new way. I've only been using a go.work file for running the fantastically helpful pkgsite. Kudos to everyone that's put that marvelousness together.

I wonder, can the go.sum and go.mod files go in .git/ ? If not, I'll experiment with that too.

[Jeffery Carr]

On Sunday, March 3, 2024 at 5:46:13 PM UTC-6 Christian Stewart wrote:

Firstly, you can reference the other repos and use the latest "master"
version by writing "master" where the v.... verson is in the file,
then use "go mod tidy"

Second, you can use "go work" to create a workspace with multiple go
modules in it, so that you can develop with them without having to
constantly update the go.mod versions.

Yes, the second way is better for me, but I couldn't figure out how that was better than just using the compiler when I'm at the beginning and I don't even know what I'm doing yet.

[Christian Stewart]

I suggest you start with "go mod vendor" (which still allows Go To
Definition in your editor and quick temporary modifications) and when
you find a repo you need to modify, clone it into your workspace and
add to go.work

[Jan Mercl]

On Sun, Mar 3, 2024 at 10:25 PM Jeffery Carr <basil...@gmail.com> wrote:

> Has this been deprecated or maybe it is broken in debian sid, but:
>
> bash$ GO111MODULE=off go get -v go.wit.com/apps/test
> go: modules disabled by GO111MODULE=off; see 'go help modules'
> basj$ go version
> go version go1.22.0 linux/amd64
>
> this doesn't work anymore. Also, 'go help modules' is less than helpful.
>
> Is there some other way to download the actual sourcecode into ~/go/src/mything/ anymore?

One can get a somehow similar effect using 'auto' instead of 'off':

jnml@e5-1650:~/tmp/get$ ls -la
total 8
drwxr-xr-x 2 jnml jnml 4096 Mar  7 11:18 .
drwxr-xr-x 6 jnml jnml 4096 Mar  7 11:18 ..
jnml@e5-1650:~/tmp/get$ GOPATH=$(pwd) GO111MODULE=off go get -v modernc.org/sqlite      

go: modules disabled by GO111MODULE=off; see 'go help modules'

jnml@e5-1650:~/tmp/get$ GOPATH=$(pwd) GO111MODULE=auto go get -v modernc.org/sqlite
go: downloading modernc.org/sqlite v1.29.2
go: downloading modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6
go: downloading golang.org/x/sys v0.16.0
go: downloading modernc.org/libc v1.41.0
go: downloading modernc.org/token v1.1.0
go: downloading modernc.org/mathutil v1.6.0
go: downloading github.com/dustin/go-humanize v1.0.1
go: downloading github.com/hashicorp/golang-lru/v2 v2.0.7
go: downloading modernc.org/strutil v1.2.0
go: downloading github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec
go: downloading github.com/google/uuid v1.3.0
go: downloading github.com/ncruces/go-strftime v0.1.9
go: downloading github.com/mattn/go-isatty v0.0.16
go: downloading modernc.org/memory v1.7.2
jnml@e5-1650:~/tmp/get$ ls pkg/mod/
cache  github.com  golang.org  modernc.org
jnml@e5-1650:~/tmp/get$  go version
go version go1.22.0 linux/amd64
jnml@e5-1650:~/tmp/get$ 

HTH

[roger peppe]

It's not quite what you're after, but FWIW the gohack command (github.com/rogpeppe/gohack) knows enough to download arbitrary Go modules including the VCS checkout. You might find it useful, I guess.

For example:

% gohack get -vcs golang.org/x/mod

creating golang.org/x/m...@v0.16.0
golang.org/x/mod => /home/rogpeppe/gohack/golang.org/x/mod

% cd /home/rogpeppe/gohack/golang.org/x/mod

% git log -1

commit 766dc5df63e3e3e5cd6b1682f522a01c99723beb (HEAD, tag: v0.16.0)
Author: Sam Thanawalla <samtha...@google.com>
Date:   Fri Feb 2 20:14:19 2024 +0000

... etc

Now that workspaces are a thing, it might be nice for it to add it to the current go.work

file rather than to go.mod, I guess.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/8a3d883a-039e-4a95-8fc4-ff9fc0b874c2n%40googlegroups.com.

[Jeffery Carr]

On Mon, Mar 11, 2024 at 10:05 AM roger peppe <rogp...@gmail.com> wrote:

It's not quite what you're after, but FWIW the gohack command (github.com/rogpeppe/gohack) knows enough to download arbitrary Go modules including the VCS checkout. You might find it useful, I guess.

Okay, yes, I'm only 5 years late to the party then. You're Readme has exactly the rationale I was thinking.

It appears the VCS code you used has since been moved inside the compiler and is internal/

Now that workspaces are a thing, it might be nice for it to add it to the current go.work

file rather than to go.mod, I guess.

I took a stab at that. go install go.wit.com/apps/go-clone@latest

Maybe the VCS shouldn't completely be internal/ ? Maybe there is a GO function that returns the git clone path from the go import line?

[Jeffery Carr]

On Thu, Mar 7, 2024 at 4:28 AM Jan Mercl <0xj...@gmail.com> wrote:

 

jnml@e5-1650:~/tmp/get$ GOPATH=$(pwd) GO111MODULE=auto go get -v modernc.org/sqlite

 

jnml@e5-1650:~/tmp/get$ ls pkg/mod/
cache  github.com  golang.org  modernc.org

Interesting, it does indeed pull the pkg/mod, but in this case, I need the old ~/go/src/ not ~/go/pkg/mod.

I have to git clone the repositories. I ended up making a simple tool that works for what I need it for (go.wit.com/apps/go-clone)

"go-clone --work modernc.org/sqlite" almost also worked where all the go.sum requirements are pretty clean. It's still a hack job though really.

[Jeffery Carr]

On Tuesday, March 5, 2024 at 9:29:58 AM UTC-6 Bryan C. Mills wrote:

 

The simplest way to download Go dependencies for code review and debugging is to run `go mod vendor` in your module or (newly added) `go work vendor` in your workspace. Also note that `go mod download -json` and `go list -m -json` now include origin information in most cases:

~$ go list -json -m go.uber.org/z...@v1.27.0
...

 

                "VCS": "git",
                "URL": "https://github.com/uber-go/zap",

In switching to this, I had to pull the Module struct from src/cmd/go/internal/list/list.go and Origin from src/cmd/go/internal/modfetch/codehost/codehost.go

While doing that and looking at the code, there are lots of interesting and good things in internal/ that can't be used. While I think I understand the reasons for internal/, I wonder if the GO compiler might be more immune to those normal reasons. In general: it's sage advice to never make the compiler people unhappy. Usually then lots of strange things start not working!

In that regard, s/internal/workshop/ would denote things not yet stable. The compiler really only needing an example to demonstrate the usage

package notinternal

import { internal/notthis )

func CanCallThis() {

      notthis.ButNotThis()

}

That's my 2 cents there. It's fun to work with such an advanced language.

[Christian Stewart]

Hi all,

What is the best way to download a module and all dependencies (via
Git clone) to a tree similar to old GOPATH?

I want to make a backup of all dependencies and this is painstaking to
do manually.

Is there a tool for this?

Your go.wit.com URLs are not accessible for me with connection failed.

Thanks,
Christian Stewart

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CA%2BHr9gpAmN%3DcmnN5G3HkObM1TRtBpGVq_%3Dmqu3_A8nyugyU0tA%40mail.gmail.com.

[Ian Lance Taylor]

On Sat, Mar 23, 2024 at 12:20 PM Jeffery Carr <basil...@gmail.com> wrote:
>
> While doing that and looking at the code, there are lots of interesting and good things in internal/ that can't be used. While I think I understand the reasons for internal/, I wonder if the GO compiler might be more immune to those normal reasons. In general: it's sage advice to never make the compiler people unhappy. Usually then lots of strange things start not working!
>
> In that regard, s/internal/workshop/ would denote things not yet stable. The compiler really only needing an example to demonstrate the usage

Our experience is that once something becomes publicly accessible, it
becomes very difficult to change.

It's already difficult to change runtime internal functions, because
packages out there reach into the runtime using //go:linkname
directives. This is clearly and obviously unsupported. Yet people do
it. That forces us into the uncomfortable position of breaking
existing working packages or advancing the runtime. It's not a great
position to be in. That's why we have rules like internal packages
that are difficult to break.

Ian