symlinks in GOPATH

[Nick Craig-Wood]

symlinks in GOPATH don't seem to be working as I expect.

rm -rf /tmp/go /tmp/mycode
mkdir /tmp/mycode
mkdir -p /tmp/go/src/
ln -s /tmp/mycode /tmp/go/src/
cat <<EOF >/tmp/mycode/main.go
package main
import "fmt"
func main() { fmt.Println("Hello from mycode") }
EOF

export GOPATH=/tmp/go
cd /tmp/go/src/mycode
go build
/tmp/go/src/mycode/mycode

echo "installing mycode"
go install mycode && echo "OK"

echo "local directory install"
go install && echo "OK"

This prints

Hello from mycode
installing mycode
OK
local directory install
go install: no install location for directory /tmp/mycode outside GOPATH

So having a symlink in GOPATH kind of works. It confuses other things
like go tool cover -html, and goimports.

Is this a bug, or just a not supported way of working with GOPATH ?
https://golang.org/doc/code.html is silent on the issue!

--
Nick Craig-Wood <ni...@craig-wood.com> -- http://www.craig-wood.com/nick

[mgutz]

Ran into same issue. Makes it difficult to symlink forks of projects.

[James Bardin]

When working with directories outside GOPATH, I always make certain that they are symlinks into GOPATH, never the reverse. I've found I can always arrange my workspace to fit this layout.

On Thursday, September 11, 2014 10:52:24 AM UTC-4, mgutz wrote:

Ran into same issue. Makes it difficult to symlink forks of projects.

Using a symlink for a fork without rewriting the imports could be convenient though.

 

[Keith Rarick]

> Ran into same issue. Makes it difficult to symlink forks of projects.

It's better to stick with the original import path, and not bother
with a symlink. You can put a git remote for every remote fork
in the same repo under the one import path.

[Dave Cheney]

It's not supported because symlinks create aliases of packages. The go tool goes out of it's way to ignore symlinks.

[Nick Craig-Wood]

On 12/09/14 02:23, Dave Cheney wrote:
> On Thursday, 11 September 2014 23:13:55 UTC+10, Nick Craig-Wood wrote:
> So having a symlink in GOPATH kind of works. It confuses other things
> like go tool cover -html, and goimports.
>
> Is this a bug, or just a not supported way of working with GOPATH ?

> https://golang.org/doc/code.html <https://golang.org/doc/code.html>

> is silent on the issue!
>
> It's not supported because symlinks create aliases of packages. The go
> tool goes out of it's way to ignore symlinks.

:-(

I'm trying to work out a way of working with some go code checked into a
sub directory in a much bigger respository. The symlink from GOPATH is
the best I've come up with but it doesn't work fully. Most frustrating!

I can use a mount --bind instead of a symlink which works, but is pretty
untidy. Depending on where I'm editing the files I can have goimports
working or version control but not both!

[Benjamin Measures]

> I'm trying to work out a way of working with some go code checked into a
sub directory in a much bigger respository. The symlink from GOPATH

A little OT but can't you just add/append it to your GOPATH?

[Nick Craig-Wood]

I think that is the answer - thanks! I don't know why I didn't think of
that before! I thought of putting it at the start of the GOPATH which
splats pkg, bin and go get stuff into your source tree but not at the end.

The setup now looks like this: I have my go workspace at ~/go say and I
append to it the path to part of the bigger repository, eg

GOPATH=~/go:~/BigProject/SubProject/go

within the go directory I have

src/dir/code.go

The only inelegance seems to be checking in a few levels of empty
directories (go, src and dir) which I can live with.

Thanks!

[Stephen Day]

We generally use symlinks into our GOPATHs to support compatibility with existing repo layouts. For example, I have a checkout directory structure that has existed way before we ever used Go. Symlinks are then used from the GOPATH into that directory structure to allow us to build packages with local changes. We could develop everything in GOPATH, but then we'd have to instruct every one to do it the "Go Way", which has varying success. We could reverse it, per James' instruction, but the goal is to make it easier for adoption.

We've only really noticed problems around the usage of godoc.

Could alternative solutions to the approach of ignoring symlinks in the go tool be found to specific problems? IMHO, the go tool shouldn't really have opinions about the underlying structure of the file system. What are the real problems with aliased packages?

[Dave Cheney]

On 16 Sep 2014 04:58, "Stephen Day" <stev...@gmail.com> wrote:
>
> We generally use symlinks into our GOPATHs to support compatibility with existing repo layouts. For example, I have a checkout directory structure that has existed way before we ever used Go. Symlinks are then used from the GOPATH into that directory structure to allow us to build packages with local changes. We could develop everything in GOPATH, but then we'd have to instruct every one to do it the "Go Way", which has varying success. We could reverse it, per James' instruction, but the goal is to make it easier for adoption.
>
> We've only really noticed problems around the usage of godoc.
>
> Could alternative solutions to the approach of ignoring symlinks in the go tool be found to specific problems? IMHO, the go tool shouldn't really have opinions about the underlying structure of the file system.

But it does, its an opinionated tool.

What are the real problems with aliased packages?
>

Aliased packages mean you have two versions of the same source with different names, so you have to independent paths for package I initialisation.

> --
> You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/f5ZYztyHK5I/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[aro...@gmail.com]

Unfortunately, multiple GOPATHs have somewhat surprising characteristic that code in a separate path is sometimes compiled:

https://code.google.com/p/go/issues/detail?id=3749

This caused severe problems for us when we decided to have a third_party path for all external code.  When we updated the third_party code, not all developers got the new version when they synced because they had already compiled the code once.

- Augusto

[Dave Cheney]

Yup. I think more people are coming around to the idea of a single GOPATH entry.

[aro...@gmail.com]

Coming around to it?  Personally I can't think of any meaningfully correct way to use multiple GOPATHs due to this issue.  Using "-a" to reinstall everything only works if the root go code is user-modifiable, and then is absurdly inefficient.  This feels like a broken feature, and I felt like we were forced into a single GOPATH.

Hrm...

Seems like having a build flag that ensures freshness across GOPATHs would be ideal for me.  Like a "-f" or "-u" flag (fresh or updated).

I would have really liked to cleanly separate third_party code from our code, and aside from the recompilation issue it had worked perfectly for us.

- Augusto

Relevant:

  https://code.google.com/p/go/issues/detail?id=6424

  https://code.google.com/p/go/issues/detail?id=8290

[Benjamin Measures]

On Tuesday, 16 September 2014 04:27:25 UTC+1, aro...@gmail.com wrote:

Personally I can't think of any meaningfully correct way to use multiple GOPATHs due to this issue.

Oh it's not /that/ bad.

Reading the code [1] relating to issue #3749 [2],

// [...] If a package p is not in the same tree as any package name on the command-line, assume it is up-to-date [...] it effectively makes each tree listed in $GOPATH a separate compilation world. [...]
if p.Root != "" && !topRoot[p.Root] {

So put a "dummy" package called "thirdparty" in your third-party GOPATH, and install with:

go install mypackage thirdparty

This causes the packages of the third-party tree to be included in the build roots. Problem solved, no drama.

As a corollary, this behaviour enables interesting workflows where you /don't/ want certain packages rescanned each build. 

[1] https://code.google.com/p/go/source/browse/src/cmd/go/pkg.go#741

[2] https://code.google.com/p/go/issues/detail?id=3749