On default GOPATH behavior

[Dustin Sallings]

I've seen this discussion a few times and it keeps biting new users
(or at least me when they come asking me for help).

Does anyone have any strong opinions on changing the default GOPATH
behavior to prevent people who don't set it from shooting themselves in
the foot?

It seems rarely desirable to clobber GOROOT and mix downloaded
packages with system packages.

Having it default to a ~/.go or ~/go or similar, or perhaps just error
might be slightly smoother for the new users who do a few go gets, then
set GOPATH and then are very confused when some things are there and
others aren't.

--
dustin

[Jan Mercl]

On Thu, Dec 13, 2012 at 9:01 PM, Dustin Sallings <dsal...@gmail.com> wrote:
> Having it default to a ~/.go or ~/go or similar, or perhaps just error
> might be slightly smoother for the new users who do a few go gets, then
> set GOPATH and then are very confused when some things are there and
> others aren't.

Setting it to such values is IMO not a good option. But your second
idea was, IIRC, already discussed at the dev list recently. Maybe it's
already in tip?

-j

[Dave Cheney]

Hello,

I share your pain and proposed an alternative solution in 4186.

We can't change go install because it is used by the build process, but hopefully make go get fail without a GOPATH set and the improvements in 4079 should make it harder for people to shoot themselves in the foot.

Cheers

Dave

> --
>
>

[Patrick Mylund Nielsen]

Indeed. Of all the issues I can think of that have been mentioned in #go-nuts, this comes up the most often by far, usually because the variable is assumed to be exported but isn't. It's better now that 'go env' shows GOPATH, but if it were to fail when GOPATH isn't set, that would help a lot of newcomers.

--

[Kevin Gillette]

The GOPATH structure (including src and pkg) was one of the growing-pain points I encountered, yeah. I think the documentation just needs to be much more prominent. When I was learning Go, about a year ago, I only figured it out after I stumbled upon the then-equivalent to `go help gopath` -- it still seems users are left in the dust if they start with the source distribution (which I suspect many will do, since many of us dislike binary dists).

[Jan Mercl]

On Thu, Dec 13, 2012 at 11:30 PM, Kevin Gillette
<extempor...@gmail.com> wrote:
> The GOPATH structure (including src and pkg) was one of the growing-pain
> points I encountered, yeah. I think the documentation just needs to be much
> more prominent. When I was learning Go, about a year ago, I only figured it
> out after I stumbled upon the then-equivalent to `go help gopath` -- it
> still seems users are left in the dust if they start with the source
> distribution (which I suspect many will do, since many of us dislike binary
> dists).

You're right, GOPATH is never mentioned in [Installing Go from
source](http://golang.org/doc/install/source). It probably should be.
Something like "Don't forget to export a sane
[GOPATH](http://golang.org/cmd/go/#GOPATH_environment_variable) before
using the [go tool](http://golang.org/cmd/go/) " could be enough.

-j

[Dustin Sallings]

Jan Mercl <0xj...@gmail.com> writes:

> You're right, GOPATH is never mentioned in [Installing Go from
> source](http://golang.org/doc/install/source). It probably should be.
> Something like "Don't forget to export a sane
> [GOPATH](http://golang.org/cmd/go/#GOPATH_environment_variable) before
> using the [go tool](http://golang.org/cmd/go/) " could be enough.

A common way users come in is by finding a project that looks cool.
They see the instructions on the README that say

Install go.

Type "go get github.com/awesomecoder/awesomeproject"

Enjoy!

Then they type "brew install go" followed by the go get and then they
realize how awesome go is. If everything goes well, they spend the rest
of their time confused since some project source is in their GOPATH and
some isn't and some updates and some doesn't and it won't refetch,
etc...

I'd just like to stop telling people, "Well, you should've done this
first" as I clean up after them and try to apologize for the confusion
and tell them it'll all be worth it once it's set up correctly.

I like the idea error-with-a-story idea and a commandline override.
Let the brand new user know that we care about him, but he's not
completed the setup yet.

--
dustin

[minux]

I think the main reason CL that refuses to "go get" when GOPATH is not set

is rejected is because it's occasionally useful to able to "go get" to GOROOT.

(this mechanism is good, and we shouldn't enforce a particular policy [go get must

install to GOPATH] with the go command)

I'm willing to do the documentation fixes. What do you suggest?

[Dustin Sallings]

minux <minu...@gmail.com> writes:

> I think the main reason CL that refuses to "go get" when GOPATH is not
> set is rejected is because it's occasionally useful to able to "go
> get" to GOROOT. (this mechanism is good, and we shouldn't enforce a
> particular policy [go get must install to GOPATH] with the go command)

That means that an experienced user may want it to happen while an
inexperienced user certainly doesn't.

IMO, adding a flag that declares it OK to fallback to GOROOT or
something might solve that, but at that point I don't see why you
wouldn't just set GOPATH to GOROOT to get that effect.

When does a user want to install something in GOROOT and declaring
that by not specifying a GOPATH?

> I'm willing to do the documentation fixes. What do you suggest?

I don't think anyone would read the documentation. If they read the
documentation carefully enough, I don't think I'd have to clean up after
them as much.

Before someone runs her first python script, she doesn't read the
python docs to try to understand PYTHONPATH and make sure it's set up
correctly. Just run it and see what happens. What happens with go by
default is a mess is made which leads to confusion and eventually
enlightenment and cleanup, but it seems that that path is longer than
necessary.

--
dustin

[minux]

On Fri, Dec 14, 2012 at 5:41 PM, Dustin Sallings <dsal...@gmail.com> wrote:

minux <minu...@gmail.com> writes:

> I think the main reason CL that refuses to "go get" when GOPATH is not
> set is rejected is because it's occasionally useful to able to "go
> get" to GOROOT.  (this mechanism is good, and we shouldn't enforce a
> particular policy [go get must install to GOPATH] with the go command)

  That means that an experienced user may want it to happen while an
inexperienced user certainly doesn't.

  IMO, adding a flag that declares it OK to fallback to GOROOT or
something might solve that, but at that point I don't see why you
wouldn't just set GOPATH to GOROOT to get that effect.

because setting GOPATH=GOROOT won't work, and in fact, it shouldn't work

because the organization of GOROOT is not the same as that of GOPATH.

(e.g. pkgs are placed under GOROOT/src/pkg, where they are placed uner

GOPATH/src)

> I'm willing to do the documentation fixes. What do you suggest?

  I don't think anyone would read the documentation.  If they read the
documentation carefully enough, I don't think I'd have to clean up after
them as much.

  Before someone runs her first python script, she doesn't read the
python docs to try to understand PYTHONPATH and make sure it's set up
correctly.  Just run it and see what happens.  What happens with go by
default is a mess is made which leads to confusion and eventually
enlightenment and cleanup, but it seems that that path is longer than
necessary.

Well, it seems this is a sad reality.

Do we have any alternatives other than make "go get" reject empty GOPATH?

give a warning? or give a warning when there are two copies of a package in

both GOROOT and GOPATH (the most confusing case)?

[minux]

On Fri, Dec 14, 2012 at 6:12 PM, <itmi...@gmail.com> wrote:

Maybe change "go get" to read up GOPATH and offer a choice of workspace to import to?

"

$ go get code.google.com/p/go.example/hello

Choose workspace to import to:

[1] $GOPATH/first

[2] $GOPATH/second

...

$ 2

$ installed to $GOPATH/second

"

please, no.

this is not DOS software.

what's more, it is not which workspace to install into that confuses people.

The main problem is when to do when users first "go get" something into their $GOROOT,

and latter they figure out GOPATH should be set, and redo the go get. But the copy in

$GOROOT will override the copy in $GOPATH but go get won't ever update the copy

in $GOROOT when GOPATH is set.

[minux]

On Fri, Dec 14, 2012 at 6:19 PM, <itmi...@gmail.com> wrote:

This will also make it obvious that you can create workspaces other than the default one.

As far as declaring a GOPATH, this is already covered in the documentation.

I've only used Go in Win and GNU/Linux and I never used any installers, I just simply created some conf files for setting the envvars after going through the documentation, the first time and reused them ever since. Maybe be some tools to do that would also help.

We can certainly do this in the binary installer (ask the user where do they want to store their

own packages) and set GOPATH automatically.

However, the problem is that a lot of Unix people compile from source.

Maybe we should mention GOPATH in the top level README file. 

[André Moraes]

> what's more, it is not which workspace to install into that confuses people.
> The main problem is when to do when users first "go get" something into
> their $GOROOT,
> and latter they figure out GOPATH should be set, and redo the go get. But
> the copy in
> $GOROOT will override the copy in $GOPATH but go get won't ever update the
> copy
> in $GOROOT when GOPATH is set.

Maybe adding a flag to go get.

go get --importRoot

Then, that code will move everything that aren't go-std-lib and put
into the first GOPATH workspace.
Of course the go tool will need to know what is stdlib and what isn't,
but only the top-level dir of the std-lib should be checked. I don't
think people will install things under io/my-really-nice-package.

>
> --
>
>



--
André Moraes
http://amoraes.info

[Dave Cheney]

> I think the main reason CL that refuses to "go get" when GOPATH is not set
> is rejected is because it's occasionally useful to able to "go get" to
> GOROOT.
> (this mechanism is good, and we shouldn't enforce a particular policy [go
> get must
> install to GOPATH] with the go command)

That is actually what I am proposing. I would like to change the go
tool to refuse to download packages via `go get` into anywhere other
than the first $GOPATH entry, and refuse to work at all if $GOPATH is
not set.

Dave

[Kyle Lemons]

So, I agree that advanced users might want it to put it in GOROOT, but I also support the notion that "go get" should fail if GOPATH is unset.  What if "go get" only will install in GOROOT if GOPATH is set, but empty?

Dave

--

[Kyle Lemons]

Oh, I just realized that os.Getenv treats empty and unset the same.  Probably the smart move, but unhelpful in this instance.  Alternate suggestions would be to use some magic string like "GOROOT" in GOPATH or to make an option to go get like -D/--destination (which can be checked as being identical to GOROOT).

[minux]

On Sat, Dec 15, 2012 at 12:41 AM, Kyle Lemons <kev...@google.com> wrote:

So, I agree that advanced users might want it to put it in GOROOT, but I also support the notion that "go get" should fail if GOPATH is unset.  What if "go get" only will install in GOROOT if GOPATH is set, but empty?

i think this is a good compromise however on some systems it's not possible to set a

environment variable to empty string.

[Kamil Kisiel]

How about an error if GOPATH is empty, and a flag for go get to be able to install in to GOROOT which bypasses GOPATH? 

eg: go get --root importpath

I think since this is only used in a minority of cases, it would be fine to use a flag for it. Would that break too many existing things? 

[André Moraes]

Or something lik:

go get --use-go-root

This forces the install to be under GOROOT and if the user don't
provide that flag, go get will abort if GOPATH is empty.

[Dave Cheney]

Or you can just use your dvcs of choice and do the checkout manually.

I cant think of a reason, apart from OS distro packaging, where you would need this function from the go tool. Can we turn the discussion to _why_ this may be needed, not _how_ it could be implemented?

--

[Volker Dobler]

I think the main reason CL that refuses to "go get" when GOPATH is not set

is rejected is because it's occasionally useful to able to "go get" to GOROOT.

Occasionally it would be useful to compile a Go program with unused

imports or unused variables; but it is forbidden because it is dangerous.

go get without GOPATH is also dangerous, it should be forbidden.

 

I'm willing to do the documentation fixes. What do you suggest?

Heavy use of the blink tag. Such documentation is always overlooked.

Something like André suggested

    GOPATH="" go get -toGOROOT somePkg  #  installs to GOROOT

but

     GOPATH="" go get somePkg  #  fails complaining about missing GOPATH

seems reasonable.

[Dave Cheney]

On Sat, Dec 15, 2012 at 11:06 AM, Volker Dobler
<dr.volke...@gmail.com> wrote:
> Occasionally it would be useful to compile a Go program with unused
> imports or unused variables; but it is forbidden because it is dangerous.
> go get without GOPATH is also dangerous, it should be forbidden.

Well said.

I'd like to move this discussion from the theoretical to the concrete
with this proposal

https://codereview.appspot.com/6941058

I have yet to hear a sound reason why using go get to install a
package into $GOROOT is more than just a historical artefact.

Dave

[Dave Cheney]

On Sat, Dec 15, 2012 at 1:22 PM, <itmi...@gmail.com> wrote:
> A new envvar: GOGETPATH. Without it, "go get" fails/complains.

Please, let not solve a problem that doesn't exist.

[Dave Cheney]

> "go get"-ing in the first of the GOPATHs it's still a mildly confusing
> concept. GOGETPATH-ing helps build a dedicated centralized local repo.
> Unless I'm missing something...

Please, lets not boil all the oceans at once. The subject of a
centralised package repo was not the OPs original question.

[Uli Kunitz]

I read the issue description. It appears that mentioning GOPATH in the first line is an error. I believe the following is correct:

Back in the day, before the Go 1.0 release, $GOROOT was mandatory for building
from source. Fast forward to now and $GOPATH is mandatory and $GOROOT is
optional, ...

[minux]

On Sat, Dec 15, 2012 at 8:06 AM, Volker Dobler <dr.volke...@gmail.com> wrote:

I think the main reason CL that refuses to "go get" when GOPATH is not set

is rejected is because it's occasionally useful to able to "go get" to GOROOT.

Occasionally it would be useful to compile a Go program with unused

imports or unused variables; but it is forbidden because it is dangerous.

go get without GOPATH is also dangerous, it should be forbidden.

i don't think unused imports or unused variables are dangerous.

we ban them simply because it is a bad programming practice and can slow

down the compilation speed.

I argue that those problems are not the same as the "go get" into $GOROOT.

Here I provide a concrete use case for that.

I have several Go installations (all hg repositories) for various release versions

and default branch in my $HOME because I might need to verify an issue using

multiple prior released versions of Go. And for this to work, I prefer to install temporary

packages into GOROOT (I will periodically clean up my various $GOROOT).

If I install them into $GOPATH, I must take care of separating their binary pkg.a into

different GOPATH for difference Go installations (note that we don't guarantee

binary package compatibility between releases and that's why we don't support

binary-only packages now). also note that I occasionally need to diagnose and

fix issues for older releases (mostly I only need to backport a fix).

Of course, I do have GOPATH for normal development.

[Matt Harden]

It seems to me that someone who wants to install to GOROOT can simply run

    GOPATH=$GOROOT go get thiscoolpackage

I suggest simply refusing to install if GOPATH is not set, and provide a useful error message in that case, that explains how to get back the historical behavior.

[minux]

On Sat, Dec 15, 2012 at 11:49 PM, Matt Harden <matt....@gmail.com> wrote:

It seems to me that someone who wants to install to GOROOT can simply run

    GOPATH=$GOROOT go get thiscoolpackage

no, this will be rejected by the go tool.

[minux]

however, i think a good compromise is to treat GOPATH='$GOROOT'

(literal $GOROOT) as a permit to install into $GOROOT.

It wil be hard to inadvertently set GOPATH to that, and I believe nobody

will create a directory named '$GOROOT' for GOPATH.

[Dave Cheney]

Thank you to all that have responded. To be clear, I'm not looking for
a way to hide, or protect via another flag to facility to go get into
$GOROOT, but simply to remove that feature. Please, lets not keep
discussing the former without engaging in its negative effects.

I am arguing, and I believe I can speak for Dustin and Kamil here
also, that the behaviour of go get for new Go programmers is
confusing, and leads to extremely hard to debug situations.

For example, most notices at least know about sudo as they would have
used it to install the binary tarball into /usr/local. It is logical
they would use the sudo hammer to go get other Go packages when faced
with a permission error talking about the path they just had to use
sudo on last time. EVEN if they had read the documentation and set
$GOPATH, go get will not work as expected as sudo masks most
environment variables.

llucky(~) % ~/go/bin/go env | grep GOPATH
GOPATH="/home/dfc"
lucky(~) % sudo ~/go/bin/go env | grep GOPATH
GOPATH=""

To Volker, the only person so far to come up with a possible use for
the _current_ behaviour of go get, I'm sorry, this change would make
it a little less convenient to achieve your goal, but certainly not
impossible. go get and mv into $GOROOT, or just using your dvcs tools
directly remain an option

Cheers

Dave

> --
>
>

[minux]

i totally agree that "go get"'s current behavior will confuse newcomers, but I still

not convinced that we must drop support for "go get" installing into $GOROOT.

(btw, is the behavior of the go command covered by the Go 1 compatibility promise?)

On Sun, Dec 16, 2012 at 6:28 AM, Dave Cheney <da...@cheney.net> wrote:

To Volker, the only person so far to come up with a possible use for
the _current_ behaviour of go get, I'm sorry, this change would make

what's the possible use for the current behavior of go get?

i can't find it in Volker's posts in this thread. 

[David Symonds]

On Sun, Dec 16, 2012 at 12:02 PM, minux <minu...@gmail.com> wrote:

> (btw, is the behavior of the go command covered by the Go 1 compatibility
> promise?)

No. It's explicitly excluded. http://golang.org/doc/go1compat.html#tools

[minux]

oh, it seems the restrictions on them is pretty weak. IIUC, it's possible to even

remove the "go get" feature in the course of Go 1.x?

[David Symonds]

On Sun, Dec 16, 2012 at 1:11 PM, minux <minu...@gmail.com> wrote:

> oh, it seems the restrictions on them is pretty weak. IIUC, it's possible to even
> remove the "go get" feature in the course of Go 1.x?

Possible, sure. Unlikely, though.