v1.13: Altered go.mod file in project after updating vim-go binaries

[Michael Ellis]

I've got an existing Go project that lives under GOPATH, i.e. ~/go/src/github.com/...

I just updated to go 1.13 (from 1.12.1)  and rebuilt the project without changing any sources. The build succeeded and Go created a go.mod (and go.sum) file that containing my third party dependencies (there are only two of them).  Ok, that's nice, I thought. I added and committed go.mod and go.sum.

When I started editing sources (with NeoVim), I noticed that vim-go was no longer working, so I ran :GoUpdateBinaries.  It most succeeded with one complaint among the 2 dozen or so packages it updated.  It's still not working well in the editor, but that's not the point of this post.

I wrote some new code in one of my project files, running go vet from the command line several times until there were no errors.  When I looked at git status, I saw that the go.mod and go.sum files had changed.  My go.mod now contains lines for all vim-go dependencies!  WTF? None of those repos have anything to do with my project code.

Has anyone else encountered this? 

FWIW, here's what the require section now looks like.  The lines I've bolded are the prior content.

9fans.net/go v0.0.2 // indirect

github.com/alecthomas/gometalinter v3.0.0+incompatible // indirect

github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4 // indirect

github.com/cosiner/argv v0.0.1 // indirect

github.com/davidrjenni/reftools v0.0.0-20190827201643-0605d60846fb // indirect

github.com/fatih/gomodifytags v1.0.1 // indirect

github.com/fatih/motion v1.0.0 // indirect

github.com/go-delve/delve v1.3.1 // indirect

github.com/google/shlex v0.0.0-20181106134648-c34317bd91bf // indirect

github.com/josharian/impl v0.0.0-20190715203526-f0d59e96e372 // indirect

github.com/jstemmer/gotags v1.4.1 // indirect

github.com/keegancsmith/rpc v1.1.0 // indirect

github.com/kisielk/errcheck v1.2.0 // indirect

github.com/klauspost/asmfmt v1.2.0 // indirect

github.com/konsorten/go-windows-terminal-sequences v1.0.2 // indirect

github.com/koron/iferr v0.0.0-20180615142939-bb332a3b1d91 // indirect

github.com/mattn/go-colorable v0.1.2 // indirect

github.com/mattn/go-isatty v0.0.9 // indirect

github.com/mattn/go-runewidth v0.0.4 // indirect

github.com/mdempsky/gocode v0.0.0-20190203001940-7fb65232883f // indirect

github.com/peterh/liner v1.1.0 // indirect

github.com/robfig/cron v1.2.0

github.com/rogpeppe/godef v1.1.1 // indirect

github.com/sirupsen/logrus v1.4.2 // indirect

github.com/spf13/cobra v0.0.5 // indirect

github.com/stamblerre/gocode v0.0.0-20190327203809-810592086997 // indirect

github.com/zmb3/gogetdoc v0.0.0-20190228002656-b37376c5da6a // indirect

go.starlark.net v0.0.0-20190820173200-988906f77f65 // indirect

golang.org/x/arch v0.0.0-20190815191158-8a70ba74b3a1 // indirect

golang.org/x/crypto v0.0.0-20190829043050-9756ffdc2472 // indirect

golang.org/x/lint v0.0.0-20190409202823-959b441ac422 // indirect

golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297

golang.org/x/sys v0.0.0-20190904154756-749cb33beabd // indirect

golang.org/x/tools v0.0.0-20190904211325-a4fdb3a8b281 // indirect

gopkg.in/alecthomas/kingpin.v3-unstable v3.0.0-20180810215634-df19058c872c // indirect

honnef.co/go/tools v0.0.1-2019.2.3 // indirect

[mihaib...@gmail.com]

On Thursday, September 5, 2019 at 8:48:11 PM UTC+3, Michael Ellis wrote:

I ran :GoUpdateBinaries.

 

When I looked at git status, I saw that the go.mod and go.sum files had changed.  My go.mod now contains lines for all vim-go dependencies!

 

Running 'go get example.com/some/module' inside a Go module (the go.mod file is in the current directory or a parent directory) seems to add example.com/some/module to go.mod as an "// indirect" dependency.

Also 'go get -u modA" will try to update modA's dependencies and add them also as "// indirect" dependencies to your go.mod.

More details in https://github.com/golang/go/wiki/Modules

At first glance, it seems that GoUpdateBinaries calls GoInstallBinaries which constructs a 'go get' command, optionally with a -u flag:

https://github.com/fatih/vim-go/blob/master/plugin/go.vim#L117

https://github.com/fatih/vim-go/blob/master/plugin/go.vim#L191

Running 'go get ...' inside a module can add indirect dependencies to go.mod.

Mihai

[Jan Mercl]

On Thu, Sep 5, 2019 at 8:18 PM <mihaib...@gmail.com> wrote:

> Running 'go get ...' inside a module can add indirect dependencies to go.mod.

I'm surprised. There's probably some rationale behind `go get` having
such side effects. Can anyone please shed some light on this?

[Michael Ellis]

I second Jan's request for light-shedding.  It's one thing to add real indirect dependencies. I can sort of understand that.  Can't think of a sensible reason for adding dependencies that are unrelated to the code in the module. 

[Mihai Borobocea]

On Thu, Sep 5, 2019 at 9:22 PM Jan Mercl <0xj...@gmail.com> wrote:
>
> > Running 'go get ...' inside a module can add indirect dependencies to go.mod.
>
> I'm surprised. There's probably some rationale behind `go get` having
> such side effects. Can anyone please shed some light on this?

$ go help module-get
The 'go get' command changes behavior depending on whether the
go command is running in module-aware mode or legacy GOPATH mode.
…
Get resolves and adds dependencies to the current development module
and then builds and installs them.

If you are inside a module (there is a go.mod file in your current dir
or a parent dir) you can just type
$ go help get
But the output of this command changes depending on your current dir
(describes either module-aware or legacy GOPATH behavior).

Other resources are https://tip.golang.org/doc/go1.13#modules (with a
subsection on go get), https://github.com/golang/go/wiki/Modules and
this series of blog posts https://blog.golang.org/using-go-modules
These are what I've looked at to get a sense of modules and stop using
GOPATH now that go 1.13 is out.

I posted this related message with a minimal example of module
dependencies and thoughts/questions on how to update my module's
direct dependencies without mentioning indirect dependencies:
https://groups.google.com/forum/#!topic/golang-nuts/FrWNhWsLDVY

My impression is that Go modules and dependency management are well
thought through and the commands have well defined behavior. But it is
not a small topic/scenario, so getting a sense of how things work
takes some effort and experimentation.

Mihai

[Jan Mercl]

On Thu, Sep 5, 2019 at 8:51 PM Mihai Borobocea <mihaib...@gmail.com> wrote:

> $ go help module-get
> The 'go get' command changes behavior depending on whether the
> go command is running in module-aware mode or legacy GOPATH mode.
> …
> Get resolves and adds dependencies to the current development module
> and then builds and installs them.

Thanks for the pointer. I think that behavior is not a good idea. Yes,
it's convenient that when working on a module and realizing some
dependency is missing, `go get` handles updating `go.mod`. However at
the same time this breaks the workflow of possibly a big number of
people. The OP's :GoUpdateBinaries in vim-go is just one example. I
don't know how many people use `vim_go`, but the project has 10k+
stars. Developer's scripts/make files/go generate "scripts" etc, all
of that may legitimately invoke `go get` for things the module _is
not_ actually dependent on.

`go get` should do just one thing and do it well. Automagically
adjusting `go.mod` if one is found in the current directory (or in any
of its parents) when outside $GOPATH is IMO neither of that. Or it
should be enabled by a flag, like -um (update go.mod) or this is maybe
a job for some `go mod foo` command, not `go get`.

[Mihai Borobocea]

On Thu, Sep 5, 2019 at 10:18 PM Jan Mercl <0xj...@gmail.com> wrote:
> `go get` should do just one thing and do it well. Automagically
> adjusting `go.mod` if one is found in the current directory (or in any
> of its parents) when outside $GOPATH is IMO neither of that. Or it
> should be enabled by a flag, like -um (update go.mod) or this is maybe
> a job for some `go mod foo` command, not `go get`.

A-ha, I think I understand the problem: 'go get' can be used for two
unrelated purposes:
1) to add/update dependencies in the current Go module
2) to install a program globally: 'go get golang.org/x/lint/golint'
produces ~/go/bin/golint

One normally wants to perform (2) from anywhere: like 'apt-get install
...' (or perhaps 'npm install -g ...'), it doesn't matter what the
current directory is. But if the current directory happens to be
inside a module, the user inadvertently also performed (1).

If this is the case, it seems an issue worth addressing.
However, I have only a basic understanding of go commands and modules.
I might miss important details.

[Michael Ellis]

> current directory is. But if the current directory happens to be
> inside a module, the user inadvertently also performed (1).

Thanks, Mihai. That sounds like a consistent explanation.  I'm going to try creating a new junk project to see if I can reproduce the problem. Unfortunately, I'm going to be traveling for the next 2 days and won't have time to try it until Sunday.  For the time being I've simply done a git checkout to retrieve the prior go.mod and go.sum.

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/-vUHGd5zlPI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAMXRBmi_Gpa9_DrmDXP8NKNP7YR2xi-ZSxuRGKcdqHbidZytcQ%40mail.gmail.com.

[Dan Kortschak]

I also.

We have to add additional mess to our build scripts when we get testing
dependencies that are not part of our distribution to avoid
contaminating the go.{mod,sum} in the repo root.

This has repeatedly been a source of irritation and frustration.

Dan

On Thu, 2019-09-05 at 11:36 -0700, Michael Ellis wrote:
> I second Jan's request for light-shedding. It's one thing to add
> real
> indirect dependencies. I can sort of understand that. Can't think of
> a
> sensible reason for adding dependencies that are unrelated to the
> code in
> the module.
>
> On Thursday, September 5, 2019 at 2:23:11 PM UTC-4, Jan Mercl wrote:
> >

> > On Thu, Sep 5, 2019 at 8:18 PM <mihaib...@gmail.com <javascript:>>

[t hepudds]

FWIW,  I think this issue tracks a similar request:

 

  #30515 cmd/go: offer a consistent "global install" command

  https://github.com/golang/go/issues/30515

  

It was tagged as a release-blocker for Go 1.13, but I think time ran out.

One outcome considered there was a new argument to 'go get' such as 'go get -b', where '-b' might be for 'binary' or 'bare' and would just install a binary without updating the current go.mod.  A less likely outcome might be re-purposing 'go install' to do the same thing.

That issue could be a good place to comment if you have a specific idea, etc.

Regards,

thepudds