Ignore /vendor by default in polyglot repository

[Jacob Vosmaer]

Hi,

In our organization we have a main code repository that contains components written in different programming languages. One of these components is written in Go. I would like to be able to have a single go.mod file for the whole repository, in the root directory. This doesn't work smoothly as of Go 1.16 because we have a /vendor directory in this repository, which has nothing to do with our Go code.

Because /vendor exists, commands like 'go build' and 'go test' complain about the vendor directory being out of sync. I am aware that there are workarounds with command line flags (such as -mod=mod) and that one can avoid typing these over and over by setting GOFLAGS.

However, this makes for a poor developer experience because every new contributor has to run in this problem locally and then modify their local development environment. It would be much more ergonomic if there was a way to tell the go toolchain to ignore /vendor based on a file in the repository. Because then everybody who clones the repository and tries to run 'go test' or 'go build' gets the correct default behavior.

I could have sworn I found an issue about this a while ago but I can't find it anymore, and I also don't see this topic in the golang-nuts archive. 

Is there another workaround I'm overlooking? Should I open an issue for this?

Thanks,

Jacob Vosmaer

[Manlio Perillo]

One possible solution is to have a GOENV file in the root directory of a repository.

This similar to git where, by default, the config file is first read from  .git/config, then from ~/.gitconfig and finally from /etc/gitconfig.

The problem is how to handle a write to GOENV; probably the local env file should be ignored.

Regards

Manlio 

[Jacob Vosmaer]

On Thu, Jun 3, 2021 at 6:35 PM Manlio Perillo <manlio....@gmail.com> wrote:
> One possible solution is to have a GOENV file in the root directory of a repository.

Thanks Manlio. Is that something that is possible now, or a feature
suggestion? From what I can tell it's not possible now.

What is possible already, now that I think about it, is for us to ask
our contributors to run 'go env -w GOFLAGS=-mod=mod'. It's not
something I would recommend in general because by the time you run
into a problem because of that global value, you typically have
forgotten you put it there.

I think my ideal solution would be more specific to vendoring; for
example a sort of pragma in vendor/modules.txt that tells the
toolchain to act as if there is no vendoring after all. But I don't
know the topic well enough to understand the implications of that.

Jacob

[Manlio Perillo]

On Friday, June 4, 2021 at 6:24:28 PM UTC+2 Jacob Vosmaer wrote:

On Thu, Jun 3, 2021 at 6:35 PM Manlio Perillo <manlio....@gmail.com> wrote:
> One possible solution is to have a GOENV file in the root directory of a repository.
Thanks Manlio. Is that something that is possible now, or a feature
suggestion? From what I can tell it's not possible now.

It was only a suggestion.

I don't know if the Go team is willing to add additional complexity to the go command.

 

What is possible already, now that I think about it, is for us to ask
our contributors to run 'go env -w GOFLAGS=-mod=mod'. It's not
something I would recommend in general because by the time you run
into a problem because of that global value, you typically have
forgotten you put it there.

Have you thought about using a Makefile?  Or a make.bash/test.bash script?

I think my ideal solution would be more specific to vendoring; for
example a sort of pragma in vendor/modules.txt that tells the
toolchain to act as if there is no vendoring after all. But I don't
know the topic well enough to understand the implications of that.

One possible solution that is possible now is to set the go version in the go.mod file to 1.13, so that the vendor directory is ignored:

https://github.com/golang/go/blob/2e59cc5fb4/src/cmd/go/internal/modload/init.go#L699

However it will prevent the use of recent features, like the embed package.

 

Manlio 

[Brian Candler]

I don't suppose you could move all the go code down a level, say into a "go" subdirectory of the repo, including the go.mod file?

[Amnon]

How about renaming your vendor directory to something else?

[Jacob Vosmaer]

On Fri, Jun 4, 2021 at 11:53 PM Amnon <amn...@gmail.com> wrote:
>
> How about renaming your vendor directory to something else?

That is not really an option because it is not "my" vendor directory.
For example, there is a lot of Ruby code in this repository, which
some people install with 'bundle install --deployment' which creates a
'/vendor/ruby' directory.

> On Friday, 4 June 2021 at 21:44:33 UTC+1 Brian Candler wrote:
>>
>> I don't suppose you could move all the go code down a level, say into a "go" subdirectory of the repo, including the go.mod file?

That is more or less the situation we're in now and I want to move
towards a top-level go.mod for the sake of automatic dependency
scanner tools (license detection, vulnerability scanners).

>> On Friday, 4 June 2021 at 17:44:37 UTC+1 manlio....@gmail.com wrote:
>>>
>>> On Friday, June 4, 2021 at 6:24:28 PM UTC+2 Jacob Vosmaer wrote:
>>>>
>>>> What is possible already, now that I think about it, is for us to ask
>>>> our contributors to run 'go env -w GOFLAGS=-mod=mod'. It's not
>>>> something I would recommend in general because by the time you run
>>>> into a problem because of that global value, you typically have
>>>> forgotten you put it there.
>>>>
>>>
>>> Have you thought about using a Makefile? Or a make.bash/test.bash script?

We already have a Makefile for the Go code. That is fine for building
the software, but not great for development because the Go code I'm
talking about now has 34 packages and you don't always want to run 'go
test' for all 34 of them.

The moment you need fine grained control I think you need to use the
go tool chain.

If there was a local version of `go env`, as you suggested earlier,
then we could bootstrap that from our Makefile.

>>>
>>> One possible solution that is possible now is to set the go version in the go.mod file to 1.13, so that the vendor directory is ignored:
>>> https://github.com/golang/go/blob/2e59cc5fb4/src/cmd/go/internal/modload/init.go#L699
>>>
>>> However it will prevent the use of recent features, like the embed package.

Thanks, that would indeed work, but it's not tenable in the long run
because someone will want to use new Go language features at some
point.

Too bad, it doesn't sound like there are any good options for me.