go.sum security error

[Igor Chubin]

When I generate `go.sum` with go 1.16, and try to build it with go of a different version (1.13 in my case), I get `SECURITY ERROR`:

```

verifying github.com/tredoe/osu...@v1.1.1/go.mod: checksum mismatch

downloaded: h1:fx79htI3WZA9Ep4jphLFq06l3iRDimfOWTrkKOz+OAA=

go.sum:     h1:wHEjPMepmXQXkZhf9H4sQcCtmC45KuFo5VR97zG9/dY=

SECURITY ERROR

This download does NOT match an earlier download recorded in go.sum.

The bits may have been replaced on the origin server, or an attacker may

have intercepted the download attempt.

For more information, see 'go help module-auth'.

```

Then I fix (remove the entry and run `go mod tidy`) `go.sum` and try to build it again. It works with 1.13, but the problem appears then with 1.16.

So there should be some incompatibility between Go 1.13 and 1.16 (not sure exactly when it was introduced, so don't know about 1.14 and 1.15).

Currently, as a workaround, I added this to my build scripts:

```

sed -i /osutil/d go.sum \

&& go mod download github.com/tredoe/osutil

```

but it is not a real solution, of course.

How am I supposed to fix this problem?

[Ian Lance Taylor]

We no longer support Go 1.13.

You can probably work around this problem temporarily and insecurely
by setting the GONOSUMDB environment variable. See the mentions of
GONOSUMDB at https://pkg.go.dev/cmd/go.

Ian

[Jay Conrod]

This doesn't seem like a problem with Go versions. The security error is correct. It looks like the module author tagged v1.1.1 with this go.mod file then changed the tag to point to a different commit with this file.

The file on proxy.golang.org is hashed and included in the checksum database. It looks like the hash there is h1:fx79htI3WZA9Ep4jphLFq06l3iRDimfOWTrkKOz+OAA=. That's the correct one to put in go.sum.

The incorrect version may still be in your module cache. You can remove it with `go clean -modcache` (though this will remove everything else there, too).

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAOyqgcV56QDp1TXTaNsr%2B1UezWmoMbYRhk8iN58bDRzJq83xkA%40mail.gmail.com.

[Igor Chubin]

Thank you for your answers!

This is definitely not in the cache, because the problem exists everywhere,
including new containers and new cloud instances.

I can test it with 1.14 and 1.15 too; I don't think that the problem is specific
for 1.13 only.

You say, that the security error is correct: but how can it be then it is detected
by only one of the Go versions and is ignored by the other?

[Sean Liao]

Where did you install `go` from and what's the output of `go env` for both versions?

[Jay Conrod]

I think the problem is in go.sum. If it already contains an incorrect sum for a module version, the go command will report a security error when downloading that version (if the download has a different sum) or when using that version (if the cached version had a different sum that appeared to be valid at the time).

I'd suggest the following:

• Make sure GOPRIVATE is set correctly (if you depend on any private modules). For example, 'go env -w GOPRIVATE=github.com/my-private-org'.
• Make sure GOSUMDB and GONOSUMDB are not set.
• Clear the module cache with 'go clean -modcache' or temporarily set it to an empty directory with 'export GOMODCACHE=$HOME/tmpmodcache'.
• Manually delete lines from go.sum for publicly available modules (not matched by GOPRIVATE).
• Run 'go mod tidy' to re-fetch modules, re-validate sums, and re-populate go.sum.

When the go command downloads a module, by default it fetches a .mod and .zip file from proxy.golang.org. It computes a hash for each of those. If that hash is present in go.sum and it doesn't match, the go command will report a security error and delete the downloaded file. If the hash is not present in go.sum, the go command checks that against sum.golang.org, which you can think of as one big go.sum file for all public modules. sum.golang.org hashes the contents of proxy.golang.org, so this should work. The go command will then add the hash to go.sum.

That procedure is pretty much unchanged since Go 1.13, when the checksum database was introduced.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/ddcee94d-e09b-46b7-bdfb-50a2d832af09n%40googlegroups.com.

[Jay Conrod]

Ideally `go mod verify` would help in this situation, but it only compares the contents of go.sum against the module cache, and if they're consistent with each other but not the outside world, it won't report an error. I've opened #47752 for this.