"go test" fails after "go install"

[Travis Johnson]

I noticed that "go test" wasn't saving intermediate .a files, so I ran "go install", which did, and now "go test" won't run unless I pass "-a", instead it fails with an error like this:

/usr/local/Cellar/go/1.6/libexec/pkg/tool/darwin_amd64/link: cannot open file /usr/local/Cellar/go/1.6/libexec/pkg/darwin_amd64/gopkg.in/yaml.v2.a: open /usr/local/Cellar/go/1.6/libexec/pkg/darwin_amd64/gopkg.in/yaml.v2.a: no such file or directory

 

If I pass "-a" then it works as expected. The same happens on linux (with a different pkg/path obviously).

It's not specific to the package in question, I've made an MVP with the same problem here (including a dockerfile to recreate the problem): https://github.com/conslo/badlink, where it actually fails to find itself:

/usr/local/go/pkg/tool/linux_amd64/link: cannot open file /usr/local/go/pkg/linux_amd64/github.com/conslo/badlink.a: open /usr/local/go/pkg/linux_amd64/github.com/conslo/badlink.a: no such file or directory

The .o file it's looking for does however exist in my $GOPATH:

~ ❯❯❯ find $GOPATH -name 'badlink.a'

/Users/xxx/gowork/pkg/darwin_amd64/github.com/conslo/badlink.a

~ ❯❯❯

Though obviously it's not looking there, only at my $GOROOT (I think).

I'm on OSX 10.11.3, with Go 1.6. Though I encounter the same problem with the Golang 1.6 docker container running Wheezy.

So two questions: Is there a better way to get intermediate .a files to be kept around (making builds quicker), and is this a bug of some sort or am I doing something totally wrong?

I'm hesitant to believe the latter, because whatever's going on it seems odd that a go command would break "go test".

[Dave Cheney]

Somehow you've installed some go code that is not part of the standard lib into /usr/local/Cellar/go/1.6.

The best way to fix this will be to nuke that go installation and redo it using the mechanisms homebrew provides.

I don't know how you've managed to repro this in a linux container, but your go install should not be writeable by a normal user, so if at any point you reach for the sudo hammer, that is the point where things have gone wrong.

[Travis Johnson]

With homebrew /usr/local/* is writeable (homebrew doesn't run as root) could that be causing the problem? Also docker runs as root in containers by default, so that makes sense. I've reinstalled go a few times already via homebrew but if it simply "being writable by the current user" causes the problem...

[Travis Johnson]

Also, I do not appear to have not stdlib code in /usr/local/Cellar/go/1.6:

https://gist.github.com/conslo/06aff9edf76619e5d413

And did I interpret you correctly: it is a requirement that when building packages that GOROOT is not writeable by the user? Or is it "the user must not be root". Either way I'm not quite groking.

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/OAQzIysNSuU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

-Travis

617-549-3856

[Dave Cheney]

Well, both and neither. You shouldn't put your own code in GOROOT, and it shouldn't be writable.

[Dave Cheney]

On Saturday, 5 March 2016 14:42:04 UTC+11, Travis Johnson wrote:

With homebrew /usr/local/* is writeable (homebrew doesn't run as root) could that be causing the problem?

Probably not, but that's gross on the part of homebrew.

 

Also docker runs as root in containers by default, so that makes sense. I've reinstalled go a few times already via homebrew but if it simply "being writable by the current user" causes the problem...

It's not so much about "who is root", but simple insurance, you're code goes in your $HOME, owned by you, writable by you. Stuff you didn't write, like your Go install, shouldn't (in general, there are always exceptions, but this isn't one of them) be writeable by you.

[Dave Cheney]

Can you get the whole output, including the command you ran, and put it in a gist or something. Thanks.

On Saturday, 5 March 2016 14:25:13 UTC+11, Travis Johnson wrote:

[Travis Johnson]

On Saturday, 5 March 2016 14:42:04 UTC+11, Travis Johnson wrote:

With homebrew /usr/local/* is writeable (homebrew doesn't run as root) could that be causing the problem?

Probably not, but that's gross on the part of homebrew.

I kinda like a package system that doesn't require root :p. Gives me some assurance it's not going to screw with anything important to my system stability. 

It's not so much about "who is root", but simple insurance, you're code goes in your $HOME, owned by you, writable by you. Stuff you didn't write, like your Go install, shouldn't (in general, there are always exceptions, but this isn't one of them) be writeable by you.

Yea I code in my $GOPATH, which is in my home dir. But user-owned $GOROOT's aren't uncommon for me (kinda like vendoring my Go installation). I not infrequently extract a go installation in my home dir and add it to my path (and set $GOROOT, and a different $GOPATH), and start writing something to solve a problem (which is awesome because when I'm done I'm left with a static binary, but don't need to add "install go" to the system's state). Is this a workflow that's not supported? I can't think of any reason a writeable $GOROOT should cause things to break (especially when it's still writeable), though doing anything in $GOROOT isn't to be done (at least for me).

On Friday, March 4, 2016 at 7:58:57 PM UTC-8, Dave Cheney wrote:

Can you get the whole output, including the command you ran, and put it in a gist or something. Thanks.

Sure:  https://gist.github.com/conslo/e44793399eb0bca95e83

[Dave Cheney]

On Saturday, 5 March 2016 15:16:50 UTC+11, Travis Johnson wrote:

On Saturday, 5 March 2016 14:42:04 UTC+11, Travis Johnson wrote:

With homebrew /usr/local/* is writeable (homebrew doesn't run as root) could that be causing the problem?

Probably not, but that's gross on the part of homebrew.

I kinda like a package system that doesn't require root :p. Gives me some assurance it's not going to screw with anything important to my system stability. 

It's not so much about "who is root", but simple insurance, you're code goes in your $HOME, owned by you, writable by you. Stuff you didn't write, like your Go install, shouldn't (in general, there are always exceptions, but this isn't one of them) be writeable by you.

Yea I code in my $GOPATH, which is in my home dir. But user-owned $GOROOT's aren't uncommon for me (kinda like vendoring my Go installation). I not infrequently extract a go installation in my home dir and add it to my path (and set $GOROOT, and a different $GOPATH), and start writing something to solve a problem (which is awesome because when I'm done I'm left with a static binary, but don't need to add "install go" to the system's state). Is this a workflow that's not supported?

It is supported.

 

I can't think of any reason a writeable $GOROOT should cause things to break (especially when it's still writeable), though doing anything in $GOROOT isn't to be done (at least for me).

Your initial post showed something was trying to overwrite a package that wasn't part of the stdlib inside GOROOT. Although it _is_ possible to put extra code into GOROOT, it's unexpected, and you should try to find another way to do it.

[Travis Johnson]

Your initial post showed something was trying to overwrite a package that wasn't part of the stdlib inside GOROOT. Although it _is_ possible to put extra code into GOROOT, it's unexpected, and you should try to find another way to do it.

Did it? The error says:

no such file or directory

To me it seems like it's *expecting* a package not part of the stdlib to be inside GOROOT, but it isn't.

I can prove this a bit further:

bash-4.3$ echo $GOPATH

/Users/tjohnson/gowork

bash-4.3$ pwd

/Users/tjohnson/gowork/src/github.com/conslo/badlink

bash-4.3$ go test

# testmain

/usr/local/Cellar/go/1.6/libexec/pkg/tool/darwin_amd64/link: cannot open file /usr/local/Cellar/go/1.6/libexec/pkg/darwin_amd64/github.com/conslo/badlink.a: open /usr/local/Cellar/go/1.6/libexec/pkg/darwin_amd64/github.com/conslo/badlink.a: no such file or directory

FAIL github.com/conslo/badlink [build failed]

bash-4.3$ stat /usr/local/Cellar/go/1.6/libexec/pkg/darwin_amd64/github.com

stat: /usr/local/Cellar/go/1.6/libexec/pkg/darwin_amd64/github.com: stat: No such file or directory

bash-4.3$

That code does *not* exist in GOROOT, at all. And I am operating out of my GOPATH.

[Travis Johnson]

I've also just had a friend attempt this on his computer (also a mac), with the same results. It's just "go get" "cd to package" "go install" "go test":

https://gist.github.com/davinche/a91c30b68bdd3140ef54

Also, I downgraded to 1.5.3, and now I don't have this problem, so it seems like a regression:

https://gist.github.com/conslo/7511ca5a2b5b2ca2cacc

[Dave Cheney]

Can you try again with go test -x, and the go install -x (you said that fixed the problem temporarily)

Can you please also include the full output of go env.

Thanks

[Travis Johnson]

On Friday, March 4, 2016 at 9:06:25 PM UTC-8, Dave Cheney wrote:

Can you try again with go test -x, and the go install -x (you said that fixed the problem temporarily)

Can you please also include the full output of go env.

 

It's "go test -a" that still works, and it's not temporary, test without "-a" continues to fail, but "go test -a" always works. I'll give you both:

"go install -x" (on a clean env), followed by a failing "go test -x": https://gist.github.com/conslo/e31dd8b4f8aeac772906

A succeeding "go test -a -x" right after the same setup as above: https://gist.github.com/conslo/70d13ae0ce2d08a9576d

[Travis Johnson]

I can confirm this also fails on ubuntu (mint, actually) with a clean Go 1.6 install. It works on this system with 1.5.3.

Again it is simply attempting to "go test" after running "go install". My example repository is quite minimal (16 lines of code, including whitespace and closing logic) if you take a look: https://github.com/conslo/badlink

$ go get github.com/conslo/badlink
$ cd $GOPATH/src/github.com/conslo/badlink
$ go install
$ go test

Since I've managed to isolate this and easily reproduce on multiple systems, I think I'm going to simply make an issue on github, or perhaps post on golang-dev first.

[Dave Cheney]

If you delete line 6 of badlink_test.go, does that fix the problem ?

[Travis Johnson]

It does! In fact if I use the full package path instead of "." it works fine as well. (but I like my relative import for tests :(, makes me feel like I'm saying "I am testing myself").

Now I'm noticing "go install -a" doesn't seem to like $GOROOT not being writeable (as it is such on my linux box), seems to be trying to recompile stdlib packages (that I'm not even using), but that looks like a different problem so I'll poke at that separately.

Well, now I have a way to make things work (thank you), but this still feels like a bug.

[Dave Cheney]

I think the statement

import "."

Should be a compiler error, please consider logging a bug, golang.org/issue/new

[Travis Johnson]

Makes sense to me. I did think it a bit odd that the dot import worked (considering "relative imports" have been rejected repeatedly: https://github.com/golang/go/issues/3515#issuecomment-66066361)

But then I don't know what precisely  rsc means by "limited support" here: https://github.com/golang/go/issues/6147#issuecomment-94483157

Possibly related?:

https://github.com/golang/go/issues/14609

I'll make an issue, thanks!