go get defaulting to proxy.golang.org and sum.golang.org

[Russ Cox]

Hi all,

I just submitted CL 178179, which enables the Go module mirror and checksum database by default in the go command. The full details are in the commit message, which I've included below. If you run into problems, please file issues. We fully expect there are issues we don't know about yet, and we want to find out what they are. 

Thanks.

Russ

cmd/go: default to GOPROXY=https://proxy.golang.org and GOSUMDB=sum.golang.org

This CL changes the default module download and module verification mechanisms
to use the Go module mirror and Go checksum database run by Google.
See https://proxy.golang.org/privacy for the services' privacy policy.
(Today, that URL is a redirect to Google's standard privacy policy,
which covers these services as well. If we publish a more specific
privacy policy just for these services, that URL will be updated to
display or redirect to it.)

See 'go help modules' and 'go help modules-auth' for details (added in this CL).

To disable the mirror and checksum database for non-public modules:

go env -w GONOPROXY=*.private.net,your.com/*
go env -w GONOSUMDB=*.private.net,your.com/*

(If you are using a private module proxy then you'd only do the second.)

If you run into problems with the behavior of the go command when using
the Go module mirror or the Go checksum database, please file issues at
https://golang.org/issue/new, so that we can address them for the
Go 1.13 release.

For #25530.

[Jason Riddle]

A few questions.

1. For the defaults GOPROXY=https://proxy.golang.org and GOSUMDB=sum.golang.org, why does GOPROXY include 'https://' but this is missing from GOSUMDB? This seems inconsistent.

2. If HTTP_PROXY and HTTPS_PROXY are set, will go module proxying respect and go through these endpoints.

3. Following up from https://github.com/golang/go/issues/31755, chinese users don't have access to proxy.golang.org. Does the above change affect these users?

4. If I wish to not use a module proxy at all, public or private, is it enough to set both GONOPROXY and GONOSUMDB to empty values? Ex.

go env -w GONOPROXY=
go env -w GONOSUMDB=

5. It looks like this change is missing from https://tip.golang.org/doc/go1.13. This will be added to this though correct?

[Brad Fitzpatrick]

On Thu, May 23, 2019 at 2:00 PM Jason Riddle <jallr...@gmail.com> wrote:

A few questions.

1. For the defaults GOPROXY=https://proxy.golang.org and GOSUMDB=sum.golang.org, why does GOPROXY include 'https://' but this is missing from GOSUMDB? This seems inconsistent.

They're different formats. GOPROXY is a URL.

GOSUMDB is https://tip.golang.org/cmd/go/#hdr-Module_authentication_failures ... search for "The GOSUMDB environment variable".

2. If HTTP_PROXY and HTTPS_PROXY are set, will go module proxying respect and go through these endpoints.

They should.

 

3. Following up from https://github.com/golang/go/issues/31755, chinese users don't have access to proxy.golang.org. Does the above change affect these users?

For now. See the reply from Russ at https://go-review.googlesource.com/c/go/+/178179/2#message-af124ab2851dc2122d5f41422c91faa339d27a67 ... "We realize that, and we know we have to do something for the actual release. We're looking into exactly how much and what we can do."

4. If I wish to not use a module proxy at all, public or private, is it enough to set both GONOPROXY and GONOSUMDB to empty values? Ex.

go env -w GONOPROXY=
go env -w GONOSUMDB=

I think you'd set GOPROXY=direct (https://tip.golang.org/cmd/go/#hdr-Module_downloading_and_verification).

And GOSUMDB=off (https://tip.golang.org/cmd/go/#hdr-Module_authentication_failures)

 

5. It looks like this change is missing from https://tip.golang.org/doc/go1.13. This will be added to this though correct?

Yes.

 

On Thursday, May 23, 2019 at 7:36:38 AM UTC-7, rsc wrote:

Hi all,

I just submitted CL 178179, which enables the Go module mirror and checksum database by default in the go command. The full details are in the commit message, which I've included below. If you run into problems, please file issues. We fully expect there are issues we don't know about yet, and we want to find out what they are. 

Thanks.

Russ

cmd/go: default to GOPROXY=https://proxy.golang.org and GOSUMDB=sum.golang.org

This CL changes the default module download and module verification mechanisms
to use the Go module mirror and Go checksum database run by Google.
See https://proxy.golang.org/privacy for the services' privacy policy.
(Today, that URL is a redirect to Google's standard privacy policy,
which covers these services as well. If we publish a more specific
privacy policy just for these services, that URL will be updated to
display or redirect to it.)

See 'go help modules' and 'go help modules-auth' for details (added in this CL).

To disable the mirror and checksum database for non-public modules:

go env -w GONOPROXY=*.private.net,your.com/*
go env -w GONOSUMDB=*.private.net,your.com/*

(If you are using a private module proxy then you'd only do the second.)

If you run into problems with the behavior of the go command when using
the Go module mirror or the Go checksum database, please file issues at
https://golang.org/issue/new, so that we can address them for the
Go 1.13 release.

For #25530.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/827108f4-3ef9-4f8e-ab08-ac195a79dc7a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[thepud...@gmail.com]

1. For the defaults GOPROXY=https://proxy.golang.org and GOSUMDB=sum.golang.org, why does GOPROXY include 'https://' but this is missing from GOSUMDB? This seems inconsistent.

FWIW, question 1 is tracked as https://github.com/golang/go/issues/32191.

Regards,

thepudds

To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.

[kushp...@gmail.com]

Hi Russ,

There was an issue to allow fallback to direct if proxy 404s. Would this work for non-public modules or do we have to go the route of setting `GONOPROXY` and `GONOSUMDB` ? Or did we ditch the doing GOPROXY=proxy,direct thing ?

Thanks!

On Thursday, May 23, 2019 at 7:36:38 AM UTC-7, rsc wrote:

[thepud...@gmail.com]

> There was an issue to allow fallback to direct if proxy 404s. Would this work for non-public modules 

> or do we have to go the route of setting `GONOPROXY` and `GONOSUMDB` ? 

> Or did we ditch the doing GOPROXY=proxy,direct thing ?

Hi,

I suspect you might be asking about https://golang.org/cl/173441, which was merged and allows GOPROXY=someproxy,direct.

As far as I understand it, that does not replace the need for GONOSUMDB, or at least not in all situations. This is discussed a bit more in this section of the proposal: https://go.googlesource.com/proposal/+/master/design/25530-sumdb.md#command-client

Regards,

thepudds

[Russ Cox]

Hi all,

Two mistakes so far in the default settings: (1) they weren't respected at all, so no one was getting the proxy by default, even though 'go env' made it look like they were, and (2) we omitted the fallback to direct if the mirror serves a 404. Both are fixed by CL 178720 (just submitted), which also makes the mirror and checksum database fetches visible in -x output.

On Thu, May 23, 2019 at 8:59 PM <kushp...@gmail.com> wrote:

There was an issue to allow fallback to direct if proxy 404s. Would this work for non-public modules or do we have to go the route of setting `GONOPROXY` and `GONOSUMDB` ? Or did we ditch the doing GOPROXY=proxy,direct thing ?

Thank you for pointing this out. You're right that we missed it, and the default is now https://proxy.golang.org,direct. The fallback to direct is most important for public modules that the mirror cannot legally serve.

For private modules, you need to set GONOSUMDB, or else any private module version not already in go.sum will fail to authenticate. You don't strictly need to set GONOPROXY as well (once the fallback works), but you probably want to: it will be faster not to ask the proxy for a module you know it won't serve, and you avoid sending that module path to the proxy at all.

That is, for the public mirror, you will typically want GONOSUMDB and GONOPROXY set the same. If you were using a private proxy, though, you'd only need GONOSUMDB. That's why they are different settings. We thought about having a combination setting, like maybe GOPRIVATE would imply GONOPROXY and GONOSUMDB, but it wouldn't let us remove either of the specialized ones, so then we'd just have three.

Best,

Russ