[go] cmd/go: don't update go.sum in 'go mod download' without args

781 views
Skip to first unread message

Jay Conrod (Gerrit)

unread,
May 10, 2021, 6:15:14 PM5/10/21
to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

Jay Conrod has uploaded this change for review.

View Change

cmd/go: don't update go.sum in 'go mod download' without args

'go mod download' without arguments is frequently used to populate the
module cache. It tends to fetch a lot of extra files (for modules in
the build list that aren't needed to build packages in the main
module). It's annoying when sums are written for these extra files.

'go mod download mod@version' will still write sums for specific
modules in the build list. 'go mod download all' still has the
previous behavior.

Fixes #45332

Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
---
M src/cmd/go/internal/modcmd/download.go
M src/cmd/go/testdata/script/mod_download.txt
2 files changed, 40 insertions(+), 7 deletions(-)

diff --git a/src/cmd/go/internal/modcmd/download.go b/src/cmd/go/internal/modcmd/download.go
index a6c6d91..83da697 100644
--- a/src/cmd/go/internal/modcmd/download.go
+++ b/src/cmd/go/internal/modcmd/download.go
@@ -86,8 +86,10 @@
if !modload.HasModRoot() && len(args) == 0 {
base.Fatalf("go mod download: no modules specified (see 'go help mod download')")
}
- if len(args) == 0 {
+ haveExplicitArgs := len(args) != 0
+ if !haveExplicitArgs {
args = []string{"all"}
+ modload.DisallowWriteGoMod() // see call to WriteGoMod below.
} else if modload.HasModRoot() {
modload.LoadModFile(ctx) // to fill Target
targetAtUpgrade := modload.Target.Path + "@upgrade"
@@ -185,8 +187,18 @@
base.ExitIfErrors()
}

- // Update go.mod and especially go.sum if needed.
- modload.WriteGoMod(ctx)
+ // If there were explicit arguments, update go.mod and especially go.sum.
+ // 'go mod download mod@version' is a useful way to add a sum without using
+ // 'go get mod@version', which may have other side effects. We print this in
+ // some error message hints.
+ //
+ // 'go mod download' without arguments is sometimes used to pre-populate the
+ // module cache, even though it may fetch more zip files than needed.
+ // In 1.16, it added sums for newly downloaded files, which was annoying.
+ // 'go mod download all' may still be used for this if needed.
+ if haveExplicitArgs {
+ modload.WriteGoMod(ctx)
+ }

// If there was an error matching some of the requested packages, emit it now
// (after we've written the checksums for the modules that were downloaded
diff --git a/src/cmd/go/testdata/script/mod_download.txt b/src/cmd/go/testdata/script/mod_download.txt
index 8a9faff..667ff63 100644
--- a/src/cmd/go/testdata/script/mod_download.txt
+++ b/src/cmd/go/testdata/script/mod_download.txt
@@ -107,13 +107,25 @@
! go mod download m@latest
stderr '^go mod download: m@latest: malformed module path "m": missing dot in first path element$'

-# download updates go.mod and populates go.sum
+# download without arguments does not update go.mod or go.sum.
cd update
+cp go.mod.orig go.mod
! exists go.sum
go mod download
+cmp go.mod.orig go.mod
+! exists go.sum
+
+# download with arguments (even "all") does updates go.mod and go.sum.
+go mod download rsc.io/sampler
+cmp go.mod.update go.mod
grep '^rsc.io/sampler v1.3.0 ' go.sum
-go list -m rsc.io/sampler
-stdout '^rsc.io/sampler v1.3.0$'
+cp go.mod.orig go.mod
+rm go.sum
+
+go mod download all
+cmp go.mod.update go.mod
+grep '^rsc.io/sampler v1.3.0 ' go.sum
+cd ..

# allow go mod download without go.mod
env GO111MODULE=auto
@@ -131,7 +143,7 @@
-- go.mod --
module m

--- update/go.mod --
+-- update/go.mod.orig --
module m

go 1.16
@@ -140,3 +152,12 @@
rsc.io/quote v1.5.2
rsc.io/sampler v1.2.1 // older version than in build list
)
+-- update/go.mod.update --
+module m
+
+go 1.16
+
+require (
+ rsc.io/quote v1.5.2
+ rsc.io/sampler v1.3.0 // older version than in build list
+)

To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

Gerrit-Project: go
Gerrit-Branch: master
Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
Gerrit-Change-Number: 318629
Gerrit-PatchSet: 1
Gerrit-Owner: Jay Conrod <jayc...@google.com>
Gerrit-MessageType: newchange

Jay Conrod (Gerrit)

unread,
May 10, 2021, 6:15:28 PM5/10/21
to goph...@pubsubhelper.golang.org, Bryan C. Mills, Michael Matloob, golang-co...@googlegroups.com

Attention is currently required from: Bryan C. Mills, Michael Matloob.

Patch set 1:Run-TryBot +1Trust +1

View Change

    To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

    Gerrit-Project: go
    Gerrit-Branch: master
    Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
    Gerrit-Change-Number: 318629
    Gerrit-PatchSet: 1
    Gerrit-Owner: Jay Conrod <jayc...@google.com>
    Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
    Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
    Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
    Gerrit-Attention: Bryan C. Mills <bcm...@google.com>
    Gerrit-Attention: Michael Matloob <mat...@golang.org>
    Gerrit-Comment-Date: Mon, 10 May 2021 22:15:24 +0000
    Gerrit-HasComments: No
    Gerrit-Has-Labels: Yes
    Gerrit-MessageType: comment

    Jay Conrod (Gerrit)

    unread,
    May 11, 2021, 10:21:14 AM5/11/21
    to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

    Attention is currently required from: Bryan C. Mills, Jay Conrod, Michael Matloob.

    Jay Conrod uploaded patch set #2 to this change.

    View Change

    cmd/go: don't update go.sum in 'go mod download' without args

    'go mod download' without arguments is frequently used to populate the
    module cache. It tends to fetch a lot of extra files (for modules in
    the build list that aren't needed to build packages in the main
    module). It's annoying when sums are written for these extra files.

    'go mod download mod@version' will still write sums for specific
    modules in the build list. 'go mod download all' still has the
    previous behavior.

    Fixes #45332

    Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
    ---
    M src/cmd/go/internal/modcmd/download.go
    M src/cmd/go/testdata/script/mod_download.txt
    M src/cmd/go/testdata/script/mod_get_trailing_slash.txt
    M src/cmd/go/testdata/script/mod_query.txt
    M src/cmd/go/testdata/script/mod_retract.txt
    5 files changed, 55 insertions(+), 15 deletions(-)

    To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

    Gerrit-Project: go
    Gerrit-Branch: master
    Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
    Gerrit-Change-Number: 318629
    Gerrit-PatchSet: 2
    Gerrit-Owner: Jay Conrod <jayc...@google.com>
    Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
    Gerrit-Reviewer: Go Bot <go...@golang.org>
    Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
    Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
    Gerrit-Attention: Bryan C. Mills <bcm...@google.com>
    Gerrit-Attention: Jay Conrod <jayc...@google.com>
    Gerrit-Attention: Michael Matloob <mat...@golang.org>
    Gerrit-MessageType: newpatchset

    Jay Conrod (Gerrit)

    unread,
    May 11, 2021, 10:21:31 AM5/11/21
    to goph...@pubsubhelper.golang.org, Go Bot, Bryan C. Mills, Michael Matloob, golang-co...@googlegroups.com

    Attention is currently required from: Bryan C. Mills, Michael Matloob.

    Patch set 2:Run-TryBot +1Trust +1

    View Change

      To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

      Gerrit-Project: go
      Gerrit-Branch: master
      Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
      Gerrit-Change-Number: 318629
      Gerrit-PatchSet: 2
      Gerrit-Owner: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
      Gerrit-Reviewer: Go Bot <go...@golang.org>
      Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
      Gerrit-Attention: Bryan C. Mills <bcm...@google.com>
      Gerrit-Attention: Michael Matloob <mat...@golang.org>
      Gerrit-Comment-Date: Tue, 11 May 2021 14:21:24 +0000

      Jay Conrod (Gerrit)

      unread,
      May 11, 2021, 11:46:16 AM5/11/21
      to goph...@pubsubhelper.golang.org, Go Bot, Bryan C. Mills, Michael Matloob, golang-co...@googlegroups.com

      Attention is currently required from: Bryan C. Mills, Michael Matloob.

      View Change

      1 comment:

      • Patchset:

        • Patch Set #2:

          1 of 25 TryBots failed. […]

          CL 318811 should fix this.

      To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

      Gerrit-Project: go
      Gerrit-Branch: master
      Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
      Gerrit-Change-Number: 318629
      Gerrit-PatchSet: 2
      Gerrit-Owner: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
      Gerrit-Reviewer: Go Bot <go...@golang.org>
      Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
      Gerrit-Attention: Bryan C. Mills <bcm...@google.com>
      Gerrit-Attention: Michael Matloob <mat...@golang.org>
      Gerrit-Comment-Date: Tue, 11 May 2021 15:46:10 +0000
      Gerrit-HasComments: Yes
      Gerrit-Has-Labels: No
      Comment-In-Reply-To: Go Bot <go...@golang.org>
      Gerrit-MessageType: comment

      Bryan C. Mills (Gerrit)

      unread,
      May 11, 2021, 4:26:10 PM5/11/21
      to Jay Conrod, goph...@pubsubhelper.golang.org, Bryan C. Mills, Go Bot, Michael Matloob, golang-co...@googlegroups.com

      Attention is currently required from: Jay Conrod, Michael Matloob.

      Patch set 2:Code-Review +2

      View Change

      2 comments:

      • File src/cmd/go/internal/modcmd/download.go:

        • Patch Set #2, Line 92: modload.DisallowWriteGoMod() // see call to WriteGoMod below.

          (not sure)

          Would it make sense to have 'go mod download' write checksums for the subset of go.mod files that are needed for the module graph, even if it isn't writing checksums for the downloaded modules themselves?

          In some sense we *know* that the go.mod files for the module graph are necessary, because they're independent of the package import graph.

          On the other hand, if we're about to build in a lazy module, we don't necessarily even need the go.sum files for every explicit requirement — we only strictly need sums for the dependencies whose requirements will be spot-checked by the subsequent build operation...

      • File src/cmd/go/testdata/script/mod_download.txt:

      To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

      Gerrit-Project: go
      Gerrit-Branch: master
      Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
      Gerrit-Change-Number: 318629
      Gerrit-PatchSet: 2
      Gerrit-Owner: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
      Gerrit-Reviewer: Go Bot <go...@golang.org>
      Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
      Gerrit-Attention: Jay Conrod <jayc...@google.com>
      Gerrit-Attention: Michael Matloob <mat...@golang.org>
      Gerrit-Comment-Date: Tue, 11 May 2021 20:26:05 +0000
      Gerrit-HasComments: Yes
      Gerrit-Has-Labels: Yes
      Gerrit-MessageType: comment

      Bryan C. Mills (Gerrit)

      unread,
      May 11, 2021, 4:28:18 PM5/11/21
      to Jay Conrod, goph...@pubsubhelper.golang.org, Bryan C. Mills, Go Bot, Michael Matloob, golang-co...@googlegroups.com

      Attention is currently required from: Jay Conrod, Michael Matloob.

      View Change

      1 comment:

      • Patchset:

        • Patch Set #2:

          This change should probably have a release note too — users who started running `go mod download` for Go 1.16 may need to update scripts to add the `all` argument.

      To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

      Gerrit-Project: go
      Gerrit-Branch: master
      Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
      Gerrit-Change-Number: 318629
      Gerrit-PatchSet: 2
      Gerrit-Owner: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
      Gerrit-Reviewer: Go Bot <go...@golang.org>
      Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
      Gerrit-Attention: Jay Conrod <jayc...@google.com>
      Gerrit-Attention: Michael Matloob <mat...@golang.org>
      Gerrit-Comment-Date: Tue, 11 May 2021 20:28:14 +0000
      Gerrit-HasComments: Yes
      Gerrit-Has-Labels: No
      Gerrit-MessageType: comment

      Jay Conrod (Gerrit)

      unread,
      May 12, 2021, 4:34:59 PM5/12/21
      to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

      Attention is currently required from: Jay Conrod, Michael Matloob.

      Jay Conrod uploaded patch set #3 to this change.

      View Change

      cmd/go: don't update go.sum in 'go mod download' without args

      'go mod download' without arguments is frequently used to populate the
      module cache. It tends to fetch a lot of extra files (for modules in
      the build list that aren't needed to build packages in the main
      module). It's annoying when sums are written for these extra files.

      'go mod download mod@version' will still write sums for specific
      modules in the build list. 'go mod download all' still has the
      previous behavior.

      Fixes #45332

      Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
      ---
      M doc/go1.17.html

      M src/cmd/go/internal/modcmd/download.go
      M src/cmd/go/testdata/script/mod_download.txt
      M src/cmd/go/testdata/script/mod_get_trailing_slash.txt
      M src/cmd/go/testdata/script/mod_query.txt
      M src/cmd/go/testdata/script/mod_retract.txt
      6 files changed, 69 insertions(+), 15 deletions(-)

      To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

      Gerrit-Project: go
      Gerrit-Branch: master
      Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
      Gerrit-Change-Number: 318629
      Gerrit-PatchSet: 3
      Gerrit-Owner: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
      Gerrit-Reviewer: Go Bot <go...@golang.org>
      Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
      Gerrit-Attention: Jay Conrod <jayc...@google.com>
      Gerrit-Attention: Michael Matloob <mat...@golang.org>
      Gerrit-MessageType: newpatchset

      Jay Conrod (Gerrit)

      unread,
      May 12, 2021, 4:35:45 PM5/12/21
      to goph...@pubsubhelper.golang.org, Bryan C. Mills, Go Bot, Michael Matloob, golang-co...@googlegroups.com

      Attention is currently required from: Bryan C. Mills, Michael Matloob.

      Patch set 3:Run-TryBot +1Trust +1

      View Change

      3 comments:

      • Patchset:

        • Patch Set #2:

          This change should probably have a release note too — users who started running `go mod download` fo […]

          Done

      • File src/cmd/go/internal/modcmd/download.go:

        • (not sure) […]

          I'm not really sure of the right behavior either.

          Saying that 'go mod download' doesn't update go.mod or go.sum is nice and simple. I'd even be tempted to go further to make it an error if a sum was missing for a .mod file needed to calculate the build list.

      • File src/cmd/go/testdata/script/mod_download.txt:

        • Done

      To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

      Gerrit-Project: go
      Gerrit-Branch: master
      Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
      Gerrit-Change-Number: 318629
      Gerrit-PatchSet: 3
      Gerrit-Owner: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
      Gerrit-Reviewer: Go Bot <go...@golang.org>
      Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
      Gerrit-Attention: Bryan C. Mills <bcm...@google.com>
      Gerrit-Attention: Michael Matloob <mat...@golang.org>
      Gerrit-Comment-Date: Wed, 12 May 2021 20:35:40 +0000
      Gerrit-HasComments: Yes
      Gerrit-Has-Labels: Yes
      Comment-In-Reply-To: Bryan C. Mills <bcm...@google.com>
      Gerrit-MessageType: comment

      Bryan C. Mills (Gerrit)

      unread,
      May 12, 2021, 5:30:57 PM5/12/21
      to Jay Conrod, goph...@pubsubhelper.golang.org, Go Bot, Bryan C. Mills, Michael Matloob, golang-co...@googlegroups.com

      Attention is currently required from: Jay Conrod, Michael Matloob.

      Patch set 3:Code-Review +2

      View Change

      1 comment:

      • File doc/go1.17.html:

        • Patch Set #3, Line 172:

          In Go 1.16, <code>download</code> saved all new sums for modules
          in the build list, which caused problems for some users.

          (not sure)

          Should we backport this change to Go 1.16, so that users still on 1.16 can skip the associated problems?

      To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

      Gerrit-Project: go
      Gerrit-Branch: master
      Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
      Gerrit-Change-Number: 318629
      Gerrit-PatchSet: 3
      Gerrit-Owner: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
      Gerrit-Reviewer: Go Bot <go...@golang.org>
      Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
      Gerrit-Attention: Jay Conrod <jayc...@google.com>
      Gerrit-Attention: Michael Matloob <mat...@golang.org>
      Gerrit-Comment-Date: Wed, 12 May 2021 21:30:51 +0000
      Gerrit-HasComments: Yes
      Gerrit-Has-Labels: Yes
      Gerrit-MessageType: comment

      Jay Conrod (Gerrit)

      unread,
      May 17, 2021, 3:40:24 PM5/17/21
      to goph...@pubsubhelper.golang.org, golang-co...@googlegroups.com

      Attention is currently required from: Jay Conrod, Michael Matloob.

      Jay Conrod uploaded patch set #4 to this change.

      View Change

      cmd/go: in 'go mod download' without args, don't save module zip sums


      'go mod download' without arguments is frequently used to populate the
      module cache. It tends to fetch a lot of extra files (for modules in
      the build list that aren't needed to build packages in the main
      module). It's annoying when sums are written for these extra files.

      'go mod download mod@version' will still write sums for specific
      modules in the build list. 'go mod download all' still has the
      previous behavior.

      For now, all invocations of 'go mod download' still update go.mod and
      go.sum with changes needed to load the build list (1.15 behavior).


      Fixes #45332

      Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
      ---
      M doc/go1.17.html
      M src/cmd/go/internal/modcmd/download.go
      M src/cmd/go/testdata/mod/rsc.io_sampler_v1.2.1.txt

      M src/cmd/go/testdata/script/mod_download.txt
      M src/cmd/go/testdata/script/mod_get_trailing_slash.txt
      M src/cmd/go/testdata/script/mod_query.txt
      M src/cmd/go/testdata/script/mod_retract.txt
      7 files changed, 85 insertions(+), 17 deletions(-)

      To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

      Gerrit-Project: go
      Gerrit-Branch: master
      Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
      Gerrit-Change-Number: 318629
      Gerrit-PatchSet: 4
      Gerrit-Owner: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
      Gerrit-Reviewer: Go Bot <go...@golang.org>
      Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
      Gerrit-Attention: Jay Conrod <jayc...@google.com>
      Gerrit-Attention: Michael Matloob <mat...@golang.org>
      Gerrit-MessageType: newpatchset

      Jay Conrod (Gerrit)

      unread,
      May 17, 2021, 3:47:44 PM5/17/21
      to goph...@pubsubhelper.golang.org, Go Bot, Bryan C. Mills, Michael Matloob, golang-co...@googlegroups.com

      Attention is currently required from: Bryan C. Mills, Jay Conrod, Michael Matloob.

      Jay Conrod removed a vote from this change.

      View Change

      Removed Code-Review+2 by Bryan C. Mills <bcm...@google.com>

      To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

      Gerrit-Project: go
      Gerrit-Branch: master
      Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
      Gerrit-Change-Number: 318629
      Gerrit-PatchSet: 4
      Gerrit-Owner: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
      Gerrit-Reviewer: Go Bot <go...@golang.org>
      Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
      Gerrit-Attention: Bryan C. Mills <bcm...@google.com>
      Gerrit-Attention: Jay Conrod <jayc...@google.com>
      Gerrit-Attention: Michael Matloob <mat...@golang.org>
      Gerrit-MessageType: deleteVote

      Jay Conrod (Gerrit)

      unread,
      May 17, 2021, 3:48:03 PM5/17/21
      to goph...@pubsubhelper.golang.org, Go Bot, Bryan C. Mills, Michael Matloob, golang-co...@googlegroups.com

      Attention is currently required from: Bryan C. Mills, Michael Matloob.

      Patch set 4:Trust +1

      View Change

      3 comments:

      • Patchset:

        • Patch Set #4:

          This changed a bit. Please take another look if you don't mind.

      • File doc/go1.17.html:

        • Patch Set #3, Line 172:

          In Go 1.16, <code>download</code> saved all new sums for modules
          in the build list, which caused problems for some users.

        • (not sure) […]

          Yes, opened backport issue #46214.

      • File src/cmd/go/internal/modcmd/download.go:

        • I'm not really sure of the right behavior either. […]

          Reverted to 1.15 behavior, per out-of-band discussion.

      To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

      Gerrit-Project: go
      Gerrit-Branch: master
      Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
      Gerrit-Change-Number: 318629
      Gerrit-PatchSet: 4
      Gerrit-Owner: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
      Gerrit-Reviewer: Go Bot <go...@golang.org>
      Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
      Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
      Gerrit-Attention: Bryan C. Mills <bcm...@google.com>
      Gerrit-Attention: Michael Matloob <mat...@golang.org>
      Gerrit-Comment-Date: Mon, 17 May 2021 19:47:58 +0000
      Gerrit-HasComments: Yes
      Gerrit-Has-Labels: Yes
      Comment-In-Reply-To: Bryan C. Mills <bcm...@google.com>
      Comment-In-Reply-To: Jay Conrod <jayc...@google.com>
      Gerrit-MessageType: comment

      Bryan C. Mills (Gerrit)

      unread,
      May 21, 2021, 1:23:04 PM5/21/21
      to Jay Conrod, goph...@pubsubhelper.golang.org, Go Bot, Bryan C. Mills, Michael Matloob, golang-co...@googlegroups.com

      Attention is currently required from: Jay Conrod, Michael Matloob.

      Patch set 4:Code-Review +2

      View Change

        To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

        Gerrit-Project: go
        Gerrit-Branch: master
        Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
        Gerrit-Change-Number: 318629
        Gerrit-PatchSet: 4
        Gerrit-Owner: Jay Conrod <jayc...@google.com>
        Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
        Gerrit-Reviewer: Go Bot <go...@golang.org>
        Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
        Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
        Gerrit-Attention: Jay Conrod <jayc...@google.com>
        Gerrit-Attention: Michael Matloob <mat...@golang.org>
        Gerrit-Comment-Date: Fri, 21 May 2021 17:22:59 +0000

        Jay Conrod (Gerrit)

        unread,
        May 21, 2021, 1:49:02 PM5/21/21
        to goph...@pubsubhelper.golang.org, Go Bot, Bryan C. Mills, Michael Matloob, golang-co...@googlegroups.com

        Attention is currently required from: Michael Matloob.

        View Change

        1 comment:

        To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

        Gerrit-Project: go
        Gerrit-Branch: master
        Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
        Gerrit-Change-Number: 318629
        Gerrit-PatchSet: 4
        Gerrit-Owner: Jay Conrod <jayc...@google.com>
        Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
        Gerrit-Reviewer: Go Bot <go...@golang.org>
        Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
        Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
        Gerrit-Attention: Michael Matloob <mat...@golang.org>
        Gerrit-Comment-Date: Fri, 21 May 2021 17:48:58 +0000
        Gerrit-HasComments: Yes
        Gerrit-Has-Labels: No
        Comment-In-Reply-To: Go Bot <go...@golang.org>

        Jay Conrod (Gerrit)

        unread,
        May 21, 2021, 1:49:06 PM5/21/21
        to goph...@pubsubhelper.golang.org, golang-...@googlegroups.com, Go Bot, Bryan C. Mills, Michael Matloob, golang-co...@googlegroups.com

        Jay Conrod submitted this change.

        View Change

        Approvals: Bryan C. Mills: Looks good to me, approved Jay Conrod: Trusted
        cmd/go: in 'go mod download' without args, don't save module zip sums

        'go mod download' without arguments is frequently used to populate the
        module cache. It tends to fetch a lot of extra files (for modules in
        the build list that aren't needed to build packages in the main
        module). It's annoying when sums are written for these extra files.

        'go mod download mod@version' will still write sums for specific
        modules in the build list. 'go mod download all' still has the
        previous behavior.

        For now, all invocations of 'go mod download' still update go.mod and
        go.sum with changes needed to load the build list (1.15 behavior).

        Fixes #45332

        Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
        Reviewed-on: https://go-review.googlesource.com/c/go/+/318629
        Trust: Jay Conrod <jayc...@google.com>
        Reviewed-by: Bryan C. Mills <bcm...@google.com>

        ---
        M doc/go1.17.html
        M src/cmd/go/internal/modcmd/download.go
        M src/cmd/go/testdata/mod/rsc.io_sampler_v1.2.1.txt
        M src/cmd/go/testdata/script/mod_download.txt
        M src/cmd/go/testdata/script/mod_get_trailing_slash.txt
        M src/cmd/go/testdata/script/mod_query.txt
        M src/cmd/go/testdata/script/mod_retract.txt
        7 files changed, 85 insertions(+), 17 deletions(-)

        diff --git a/doc/go1.17.html b/doc/go1.17.html
        index 3534f7b..f00c649 100644
        --- a/doc/go1.17.html
        +++ b/doc/go1.17.html
        @@ -186,6 +186,17 @@
        password-protected SSH keys.
        </p>

        +<h4 id="go-mod-download"><code>go</code> <code>mod</code> <code>download</code></h4>
        +
        +<p><!-- golang.org/issue/45332 -->
        + When <code>go</code> <code>mod</code> <code>download</code> is invoked without
        + arguments, it will no longer save sums for downloaded module content to
        + <code>go.sum</code>. It may still make changes to <code>go.mod</code> and
        + <code>go.sum</code> needed to load the build list. This is the same as the
        + behavior in Go 1.15. To save sums for all modules, use <code>go</code>
        + <code>mod</code> <code>download</code> <code>all</code>.
        +</p>
        +
        <p><!-- CL 249759 -->
        TODO: <a href="https://golang.org/cl/249759">https://golang.org/cl/249759</a>: cmd/cover: replace code using optimized golang.org/x/tools/cover
        </p>
        diff --git a/src/cmd/go/internal/modcmd/download.go b/src/cmd/go/internal/modcmd/download.go
        index a6c6d91..42b06db 100644
        --- a/src/cmd/go/internal/modcmd/download.go
        +++ b/src/cmd/go/internal/modcmd/download.go
        @@ -86,9 +86,11 @@

        if !modload.HasModRoot() && len(args) == 0 {
        base.Fatalf("go mod download: no modules specified (see 'go help mod download')")
        }
        - if len(args) == 0 {
        +	haveExplicitArgs := len(args) > 0

        + if !haveExplicitArgs {
        args = []string{"all"}
        -	} else if modload.HasModRoot() {
        + }
        + if modload.HasModRoot() {

        modload.LoadModFile(ctx) // to fill Target
        targetAtUpgrade := modload.Target.Path + "@upgrade"
         		targetAtPatch := modload.Target.Path + "@patch"
        @@ -135,6 +137,18 @@
        type token struct{}
        sem := make(chan token, runtime.GOMAXPROCS(0))
        infos, infosErr := modload.ListModules(ctx, args, 0)
        + if !haveExplicitArgs {
        + // 'go mod download' is sometimes run without arguments to pre-populate
        + // the module cache. It may fetch modules that aren't needed to build
        + // packages in the main mdoule. This is usually not intended, so don't save
        + // sums for downloaded modules (golang.org/issue/45332).
        + // TODO(golang.org/issue/45551): For now, save sums needed to load the
        + // build list (same as 1.15 behavior). In the future, report an error if
        + // go.mod or go.sum need to be updated after loading the build list.
        + modload.WriteGoMod(ctx)
        + modload.DisallowWriteGoMod()
        + }
        +
        for _, info := range infos {
        if info.Replace != nil {
        info = info.Replace
        @@ -185,8 +199,15 @@

        base.ExitIfErrors()
        }

        - // Update go.mod and especially go.sum if needed.
        - modload.WriteGoMod(ctx)
        + // If there were explicit arguments, update go.mod and especially go.sum.
        + // 'go mod download mod@version' is a useful way to add a sum without using
        + // 'go get mod@version', which may have other side effects. We print this in
        + // some error message hints.
        + //
        +	// Don't save sums for 'go mod download' without arguments; see comment above.

        + if haveExplicitArgs {
        + modload.WriteGoMod(ctx)
        + }

        // If there was an error matching some of the requested packages, emit it now
        // (after we've written the checksums for the modules that were downloaded
        diff --git a/src/cmd/go/testdata/mod/rsc.io_sampler_v1.2.1.txt b/src/cmd/go/testdata/mod/rsc.io_sampler_v1.2.1.txt
        index 00b71bf..7982ccc 100644
        --- a/src/cmd/go/testdata/mod/rsc.io_sampler_v1.2.1.txt
        +++ b/src/cmd/go/testdata/mod/rsc.io_sampler_v1.2.1.txt
        @@ -5,7 +5,7 @@

        require "golang.org/x/text" v0.0.0-20170915032832-14c0d48ead0c
        -- .info --
        -{"Version":"v1.2.1","Name":"cac3af4f8a0ab40054fa6f8d423108a63a1255bb","Short":"cac3af4f8a0a","Time":"2018-02-13T18:16:22Z"}EOF
        +{"Version":"v1.2.1","Name":"cac3af4f8a0ab40054fa6f8d423108a63a1255bb","Short":"cac3af4f8a0a","Time":"2018-02-13T18:16:22Z"}
        -- hello.go --
        // Copyright 2018 The Go Authors. All rights reserved.
        // Use of this source code is governed by a BSD-style
        diff --git a/src/cmd/go/testdata/script/mod_download.txt b/src/cmd/go/testdata/script/mod_download.txt
        index 8a9faff..ad640b4 100644
        --- a/src/cmd/go/testdata/script/mod_download.txt
        +++ b/src/cmd/go/testdata/script/mod_download.txt
        @@ -107,13 +107,28 @@

        ! go mod download m@latest
        stderr '^go mod download: m@latest: malformed module path "m": missing dot in first path element$'

        -# download updates go.mod and populates go.sum
        +# download without arguments updates go.mod and go.sum after loading the
        +# build list, but does not save sums for downloaded zips.

        cd update
        +cp go.mod.orig go.mod
        ! exists go.sum
        go mod download
        +cmp go.mod.update go.mod
        +cmp go.sum.update go.sum

        +cp go.mod.orig go.mod
        +rm go.sum
        +
        +# download with arguments (even "all") does update go.mod and go.sum.

        +go mod download rsc.io/sampler
        +cmp go.mod.update go.mod
        grep '^rsc.io/sampler v1.3.0 ' go.sum
        -go list -m rsc.io/sampler
        -stdout '^rsc.io/sampler v1.3.0$'
        +cp go.mod.orig go.mod
        +rm go.sum
        +
        +go mod download all
        +cmp go.mod.update go.mod
        +grep '^rsc.io/sampler v1.3.0 ' go.sum
        +cd ..

        # allow go mod download without go.mod
        env GO111MODULE=auto
        @@ -131,7 +146,7 @@

        -- go.mod --
        module m

        --- update/go.mod --
        +-- update/go.mod.orig --
        module m

        go 1.16
        @@ -140,3 +155,17 @@

        rsc.io/quote v1.5.2
        rsc.io/sampler v1.2.1 // older version than in build list
        )
        +-- update/go.mod.update --
        +module m
        +
        +go 1.16
        +
        +require (
        + rsc.io/quote v1.5.2
        + rsc.io/sampler v1.3.0 // older version than in build list
        +)
        +-- update/go.sum.update --
        +golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
        +rsc.io/quote v1.5.2/go.mod h1:LzX7hefJvL54yjefDEDHNONDjII0t9xZLPXsUe+TKr0=
        +rsc.io/sampler v1.2.1/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
        +rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
        diff --git a/src/cmd/go/testdata/script/mod_get_trailing_slash.txt b/src/cmd/go/testdata/script/mod_get_trailing_slash.txt
        index 3b38d8b..c536693 100644
        --- a/src/cmd/go/testdata/script/mod_get_trailing_slash.txt
        +++ b/src/cmd/go/testdata/script/mod_get_trailing_slash.txt
        @@ -1,6 +1,3 @@
        -# Populate go.sum
        -go mod download
        -
        # go list should succeed to load a package ending with ".go" if the path does
        # not correspond to an existing local file. Listing a pattern ending with
        # ".go/" should try to list a package regardless of whether a file exists at the
        @@ -31,3 +28,10 @@
        go 1.13

        require example.com/dotgo.go v1.0.0
        +-- go.sum --
        +example.com/dotgo.go v1.0.0 h1:XKJfs0V8x2PvY2tX8bJBCEbCDLnt15ma2onwhVpew/I=
        +example.com/dotgo.go v1.0.0/go.mod h1:Qi6z/X3AC5vHiuMt6HF2ICx3KhIBGrMdrA7YoPDKqR0=
        +-- use.go --
        +package use
        +
        +import _ "example.com/dotgo.go"
        diff --git a/src/cmd/go/testdata/script/mod_query.txt b/src/cmd/go/testdata/script/mod_query.txt
        index e101857..a75f86e 100644
        --- a/src/cmd/go/testdata/script/mod_query.txt
        +++ b/src/cmd/go/testdata/script/mod_query.txt
        @@ -1,9 +1,7 @@
        env GO111MODULE=on

        -# Populate go.sum.
        # TODO(golang.org/issue/41297): we shouldn't need go.sum. None of the commands
        # below depend on the build list.
        -go mod download

        go list -m -versions rsc.io/quote
        stdout '^rsc.io/quote v1.0.0 v1.1.0 v1.2.0 v1.2.1 v1.3.0 v1.4.0 v1.5.0 v1.5.1 v1.5.2 v1.5.3-pre1$'
        @@ -36,6 +34,9 @@
        module x
        require rsc.io/quote v1.0.0

        +-- go.sum --
        +rsc.io/quote v1.0.0 h1:kQ3IZQzPTiDJxSZI98YaWgxFEhlNdYASHvh+MplbViw=
        +rsc.io/quote v1.0.0/go.mod h1:v83Ri/njykPcgJltBc/gEkJTmjTsNgtO1Y7vyIK1CQA=
        -- use.go --
        package use

        diff --git a/src/cmd/go/testdata/script/mod_retract.txt b/src/cmd/go/testdata/script/mod_retract.txt
        index a52e05b..4f95ece 100644
        --- a/src/cmd/go/testdata/script/mod_retract.txt
        +++ b/src/cmd/go/testdata/script/mod_retract.txt
        @@ -1,8 +1,5 @@
        cp go.mod go.mod.orig

        -# Populate go.sum.
        -go mod download
        -
        # 'go list pkg' does not report an error when a retracted version is used.
        go list -e -f '{{if .Error}}{{.Error}}{{end}}' ./use
        ! stdout .
        @@ -32,6 +29,11 @@

        require example.com/retract v1.0.0-bad

        +-- go.sum --
        +example.com/retract v1.0.0-bad h1:liAW69rbtjY67x2CcNzat668L/w+YGgNX3lhJsWIJis=
        +example.com/retract v1.0.0-bad/go.mod h1:0DvGGofJ9hr1q63cBrOY/jSY52OwhRGA0K47NE80I5Y=
        +example.com/retract/self/prev v1.1.0 h1:0/8I/GTG+1eJTFeDQ/fUbgrMsVHHyKhh3Z8DSZp1fuA=
        +example.com/retract/self/prev v1.1.0/go.mod h1:xl2EcklWuZZHVtHWcpzfSJQmnzAGpKZYpA/Wto7SZN4=
        -- use/use.go --
        package use


        To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

        Gerrit-Project: go
        Gerrit-Branch: master
        Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
        Gerrit-Change-Number: 318629
        Gerrit-PatchSet: 5
        Gerrit-Owner: Jay Conrod <jayc...@google.com>
        Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
        Gerrit-Reviewer: Go Bot <go...@golang.org>
        Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
        Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
        Gerrit-MessageType: merged

        Bryan C. Mills (Gerrit)

        unread,
        May 21, 2021, 4:12:50 PM5/21/21
        to Jay Conrod, goph...@pubsubhelper.golang.org, Go Bot, Bryan C. Mills, Michael Matloob, golang-co...@googlegroups.com

        Attention is currently required from: Jay Conrod.

        View Change

        2 comments:

        To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

        Gerrit-Project: go
        Gerrit-Branch: master
        Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
        Gerrit-Change-Number: 318629
        Gerrit-PatchSet: 5
        Gerrit-Owner: Jay Conrod <jayc...@google.com>
        Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
        Gerrit-Reviewer: Go Bot <go...@golang.org>
        Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
        Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
        Gerrit-Attention: Jay Conrod <jayc...@google.com>
        Gerrit-Comment-Date: Fri, 21 May 2021 20:12:46 +0000
        Gerrit-HasComments: Yes
        Gerrit-Has-Labels: No
        Gerrit-MessageType: comment

        Bryan C. Mills (Gerrit)

        unread,
        May 21, 2021, 4:29:23 PM5/21/21
        to Jay Conrod, goph...@pubsubhelper.golang.org, Go Bot, Bryan C. Mills, Michael Matloob, golang-co...@googlegroups.com

        Attention is currently required from: Jay Conrod.

        View Change

        1 comment:

        • Patchset:

          • Patch Set #5:

            It looks like this also causes TestScript/mod_replace to no longer report a directory for the downloaded `not-rsc.io/quote` module (presumably because it is missing a checksum). I'll mail a fix.

        To view, visit change 318629. To unsubscribe, or for help writing mail filters, visit settings.

        Gerrit-Project: go
        Gerrit-Branch: master
        Gerrit-Change-Id: I9e17d18a7466ac7271a0e1a2b663f6b3cb168c97
        Gerrit-Change-Number: 318629
        Gerrit-PatchSet: 5
        Gerrit-Owner: Jay Conrod <jayc...@google.com>
        Gerrit-Reviewer: Bryan C. Mills <bcm...@google.com>
        Gerrit-Reviewer: Go Bot <go...@golang.org>
        Gerrit-Reviewer: Jay Conrod <jayc...@google.com>
        Gerrit-Reviewer: Michael Matloob <mat...@golang.org>
        Gerrit-Attention: Jay Conrod <jayc...@google.com>
        Gerrit-Comment-Date: Fri, 21 May 2021 20:29:19 +0000
        Reply all
        Reply to author
        Forward
        0 new messages