$GOPATH confusion seen everywhere, please help

[DisposaBoy]

I'm confused. Is there a specific question? (I won't comment on other project's *bugs*)

[peterGo]

See GOPATH and workspaces, How to Write Go Code.
http://golang.org/doc/code.html

Peter


On Saturday, May 4, 2013 10:55:05 PM UTC-4, faqin...@gmail.com wrote:

I looked around everywhere I could find, and I see two different interpretations of how the $GOPATH should work.

1.) The most common:

The $GOPATH provides us with a place to store common, shared third party libs on our system, and our own projects.

$GOPATH/src/ourcompany/ourproject1
$GOPATH/src/ourcompany/ourproject2
$GOPATH/src/othercompany/library1

This structure works perfectly with go's build tools and saves us from having a new copy of each lib source per project.

2.) Also around

Every project directory gets added to the $GOPATH. 

In project B, using lib X, which was previously installed in project A, we wouldn't see lib X in the src directory for project B, but as long as project A's directory is also in the $GOPATH, would all not work ok?

3.) Hybrid

We could add per-project directories to the $GOPATH, and also have a shared libs directory within the $GOPATH

Looking around dozens of major projects in the community, there is no consensus understanding on how this is supposed to work, and the docs don't quite clarify it. It says the $GOPATH is a workspace, but it doesn't say if that is intended as a per-project workspace.

This came up because the golang plugin for IntelliJ only works with pattern 2, and because it is not using the native import resolution tool (or $GOPATH), it breaks because it assumes that all code for a given project are within that precise project's directory. It thus requires a separate workspace per project, and the re-installation of third party libs within each project directory. 

[Andrew Gerrand]

#1 is the correct and supported way. The other methods are actively discouraged.

Andrew

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Dan Kortschak]

On Sun, 2013-05-05 at 09:19 -0400, Andrew Gerrand wrote:
> #1 is the correct and supported way. The other methods are actively
> discouraged.
>

I think a thing that leads to confusion here is that GOPATH is a list of
paths rather than a single location, but is in the documentation treated
as though it is a single locations, e.g.

For convenience, add the workspace's bin subdirectory to your
PATH:

$ export PATH=$PATH:$GOPATH/bin

This leaves people in a state where they see that it can be a list, that
it's idiomatically treated as a single item and with no explanation as
to why it should be a list at all.

Some clarification about the reason behind multiple workspaces would
probably help here - though I'm aware of the keep this document as short
as possible to prevent people ignoring it completely.

An example of things to clarify is: "In the case of a multiple item
GOPATH, what is the build target location?" I've always used a single
item GOPATH so this has never been anything other than an intellectual
curiosity (yes, not a complicated one to figure out). The answer is what
I would think is the most reasonable, but it's not clearly specified -
the docs can be interpreted correctly, but because a single item GOPATH
is used in the example it is not unambiguous (the addition below would
fix that I think):

This command builds the hello command, producing an executable
binary. It then installs that binary to the [current]
workspace's bin directory as hello (or, under Windows,
hello.exe). In our example, that will be $GOPATH/bin/hello,
which is $HOME/go/bin/hello.

[André Moraes]

In my .bashrc

export GOPATH=~/Source/gopath
export PATH=$GOPATH/bin:$PATH

This is easy and work on almost all cases.

Also, in Go people tend to respect public API for packages, and when a
API change could break code, they either go-fix the thing or keep the
old import path and put the new API under a different path.

--
André Moraes
http://amoraes.info

[Andrew Lytvynov]

The latest version of "How to write go code" (http://tip.golang.org/doc/code.html) seems quite explicit and clear to me.

On Monday, May 6, 2013 5:29:34 AM UTC+3, faqin...@gmail.com wrote:

Fantastic. Thank you. Might it be worth stating this more explicitly in the docs?

[yy]

On 6 May 2013 08:01, <faqin...@gmail.com> wrote:

Actually, I think that part of the confusion comes directly from "How to write go code" (http://tip.golang.org/doc/code.html) .

It refers to "Workspaces" (plural), which implies precisely the opposite of what Andrew Gerrand said above.

Andrew Gerrand said that "#1 is the correct and supported way. The other methods are actively discouraged."

Multiple workspaces are fine, but not a workspace for each project.

As a poor analogy (but I hope a clarifying one) think in the PATH environment variable. It has more than one directory, but adding a directory for each program you want in your path is not good practice.

Back to GOPATH, it can be useful to have for example a /usr/local/go/ directory common to all users and a personal $HOME/go, or $HOME/go and $HOME/go-sandbox so that you have a workspace to experiment with, but having $HOME/goproject1, $HOME/goproject2 and so on is against GOPATH purpose and it is discouraged.

--
- yiyus || JGL .

[Maxim Khitrov]

That's a very unix-centric solution and not one that will work for
many real-world projects. A project is more than just code; it's data,
support files, media, documentation, scripts, etc. None of these
things fit into the GOPATH model without defining a separate entry for
each project.

Discouraged or not, that's how people will use GOPATH, especially on Windows.

[Jan Mercl]

On Mon, May 6, 2013 at 2:40 PM, Maxim Khitrov <m...@mxcrypt.com> wrote:
> That's a very unix-centric solution

Why?

> and not one that will work for
> many real-world projects. A project is more than just code; it's data,
> support files, media, documentation, scripts, etc. None of these
> things fit into the GOPATH model without defining a separate entry for
> each project.

Why?

> Discouraged or not, that's how people will use GOPATH, especially on Windows.

Here I don't ask why 'especially on Windows' and let me rather not
comment it ;-)

-j

PS: Coding Go daily for living for years and never yet needed anything
above 'GOPATH=$HOME'.

[Maxim Khitrov]

On Mon, May 6, 2013 at 8:57 AM, Jan Mercl <0xj...@gmail.com> wrote:
> On Mon, May 6, 2013 at 2:40 PM, Maxim Khitrov <m...@mxcrypt.com> wrote:
>> That's a very unix-centric solution
>
> Why?

Because it's not very common on Windows to have a bunch of unrelated
binaries in a single /bin directory.

>> and not one that will work for
>> many real-world projects. A project is more than just code; it's data,
>> support files, media, documentation, scripts, etc. None of these
>> things fit into the GOPATH model without defining a separate entry for
>> each project.
>
> Why?

Not sure what you mean. How do I fit the categories of items that I've
listed for separate projects into the /bin, /pkg, and /src structure
prescribed by GOPATH?

As a more general answer, however, it seems perfectly reasonable to
say "because that's how I'd rather organize my projects." To me, it's
more convenient to have a separate directory for each project, and to
have all files under that directory under version control.

>> Discouraged or not, that's how people will use GOPATH, especially on Windows.
>
> Here I don't ask why 'especially on Windows' and let me rather not
> comment it ;-)

It may come as a shock to you, but some people do have to work with it
and a few even enjoy it :)

> PS: Coding Go daily for living for years and never yet needed anything
> above 'GOPATH=$HOME'.

That just means that GOPATH fits your workflow, which happens to match
how things are typically organized on unix systems, which is why I
said "unix-centric." Windows users don't rely on environment variables
nearly as much and a per-project directory organization is much more
common.

[Qing Fa]

Thanks. That's a great way to think of it.

Different workspaces are for different contexts (e.g. sandbox), but:

NOT for having different workspaces per project

NOT for separating projects vs. libs.

Within a given context/workspace, all are siblings within src/

Thanks everybody.

 It's different than the /libs and /projects paths concept that I have also seen discussed (option 3 in my original post I think)

[Jan Mercl]

On Mon, May 6, 2013 at 3:21 PM, Maxim Khitrov <m...@mxcrypt.com> wrote:
> Because it's not very common on Windows to have a bunch of unrelated
> binaries in a single /bin directory.

I don' get it. I'm in no way forced to have what you describe, even
when using a single GOPATH. GOBIN is effective for 'go install', but
even then, I practically never install the binaries at my development
machine. Why should I? If I want to run a binary of a project locall,
I always use 'go build' with the resulting binary in the current
directory. Testing the bin in it's supposed environment is done on a
test machine, not on the dev one. IOW, no trouble with 'bin'.

>>> and not one that will work for
>>> many real-world projects. A project is more than just code; it's data,
>>> support files, media, documentation, scripts, etc. None of these
>>> things fit into the GOPATH model without defining a separate entry for
>>> each project.
>>
>> Why?
>
> Not sure what you mean. How do I fit the categories of items that I've
> listed for separate projects into the /bin, /pkg, and /src structure
> prescribed by GOPATH?

Not sure what you mean. 'bin' was discussed above. 'pkg' and 'src'
creates zero problems for multiple projects. Probably not your case,
but let me ask: What problems you have with 'pkg' and 'src'? 'pkg' is
just a root of a tree of installed packages (and it's a globally
unique namespace in my case), the same (modulo the installing thing)
holds for 'src'. Anything local to a project goes to a directory of
that project. That _all_ projects, the tree of all projects, have a
single root doesn't interfere with that in any way.

> As a more general answer, however, it seems perfectly reasonable to
> say "because that's how I'd rather organize my projects." To me, it's
> more convenient to have a separate directory for each project, and to
> have all files under that directory under version control.

Naturally. I'm doing exactly the same with a single GOPATH.

>>> Discouraged or not, that's how people will use GOPATH, especially on Windows.
>>
>> Here I don't ask why 'especially on Windows' and let me rather not
>> comment it ;-)
>
> It may come as a shock to you, but some people do have to work with it
> and a few even enjoy it :)

It's not a shock, even though the word 'enjoying' could base an
interesting (off topic) discussion. Did you mean 'pain' instead? ;-)

>> PS: Coding Go daily for living for years and never yet needed anything
>> above 'GOPATH=$HOME'.
>
> That just means that GOPATH fits your workflow, which happens to match
> how things are typically organized on unix systems,

I see zero connection to Unix. Directories organized as trees were
copied by Windows designers (fortunately). It's the same principle on
Unix as on Windows (only less well implemented).

> which is why I
> said "unix-centric." Windows users don't rely on environment variables
> nearly as much and a per-project directory organization is much more
> common.

Except for some exotic situation (passing extra stuff to gocov, for
example) not a single project of mine depends on environment
variables, IIRC. What _does_ depend on them is the Go build system,
eg. when I'm cross compiling for an embedded ARM system.

-j

[Qing Fa]

It will always be possible to imagine scenarios where somebody wants, or even needs, to structure their code in particular way that doesn't follow the conventions of the community.

The value of the discussion, and Andrew Gerrand's answer, was to first establish exactly what the convention is supposed to be.

The problem right now is that people are reading the docs and looking at projects in the community, and they are arriving at two divergent interpretations. For example, it's probably safe to assume that a fairly significant number of developers use IntelliJ IDEA with the golang plugin, yet the plugin forces an approach very similar to the one-path per project style, and the plugin's developer is adamant that the single $GOPATH style is flat out wrong.

That's why being making the convention perfectly explicit and clear is so important. Then the community can continue advancing the tools we use in a way that facilitates following the standards chosen.

After removing the ambiguity in that regard, then some people can move off in their own directions, but doing so as a conscious choice to "break the rules" to fit their needs.

[Maxim Khitrov]

On Mon, May 6, 2013 at 9:50 AM, Jan Mercl <0xj...@gmail.com> wrote:
>> Not sure what you mean. How do I fit the categories of items that I've
>> listed for separate projects into the /bin, /pkg, and /src structure
>> prescribed by GOPATH?
>
> Not sure what you mean. 'bin' was discussed above. 'pkg' and 'src'
> creates zero problems for multiple projects. Probably not your case,
> but let me ask: What problems you have with 'pkg' and 'src'? 'pkg' is
> just a root of a tree of installed packages (and it's a globally
> unique namespace in my case), the same (modulo the installing thing)
> holds for 'src'. Anything local to a project goes to a directory of
> that project. That _all_ projects, the tree of all projects, have a
> single root doesn't interfere with that in any way.

The single root also determines Go package names. As it happens, I've
been using a single root of D:\projects\<project> on my work computer
since long before learning about Go. But even this existing single
root organization is incompatible with GOPATH without adding an extra
src directory.

Are you suggesting the use of project paths like
D:\projects\src\someproject\src\package1? Seems like a mess,
especially since not everything under D:\projects\src\ is source code.
I don't really understand how you're organizing non-Go files.

My problem (if the goal is to avoid per-project entries in GOPATH)
with bin, pkg, and src is the order of the hierarchy. I want
project\bin, project\pkg, and project\src, in addition to
project\docs, project\media, and project\otherstuff. Incidentally,
look at how Go's own repository is organized. Do you have src/repo or
repo/src?

To be clear, I'm perfectly happy with GOPATH and bin, pkg, src
structure as a way of organizing small programs (mostly Go source code
in a single main package) and reusable libraries. I find it less
suitable for large applications consisting of many non-Go files and
multiple Go packages that are not intended for sharing.

[Dougx]

Just food for thought, but the GOPATH docs certainly confused me at first.

I initially thought you had to do this (wrong, and deeply frustrating to use):

/.git
/src/namespace/blah/ <--- library
/blah1.go <--- Utility one
/blah2.go <--- Utility two (notice how the 'main' packages are individual files, not in a library, so you can go run blah1.go)

or this (major pain to use):

/.git <--- Stupid placehold git repo with multiple submodules
/src/namespace/blah/.git <--- library package
/src/namespace/blah1/.git <--- Utility one in it's own package
/src/namespace/blah2/.git <--- Utility two in it's own package

When in fact you can just do this (afaik, the 'correct' way of doing it):

/src/namespace/blah/.git <--- package you can 'go get'
/src/namespace/blah/ <--- Library
/src/namespace/blah/blah1/ <--- Utility one
/src/namespace/blah/blah2/ <--- Utility two

I know the docs sort of explain it, but it was entirely non-obvious, to me personally, that you could have multiple 'main' packages inside a different namespaces, possibly even as a submodule of the library itself.

Honestly, it's still quite mysterious to me how 'go build name/blah/blah1' builds a binary and places it in the current directory; I think that's weird, because it doesn't tell you WHICH folder you just actually built, it just sort of assumes you only have one GOPATH (which I guess you should).

Just saying, it's not *entirely* obvious. At least, it wasn't to me.

~
Doug.

[Andrew Gerrand]

On 6 May 2013 08:36, Dougx <douglas...@gmail.com> wrote:

Just saying, it's not *entirely* obvious. At least, it wasn't to me.

Did you read the old docs or the new docs?

http://golang.org/doc/code.html <- old docs (until 1.1 is released, that is)

http://tip.golang.org/doc/code.html <- new docs

[Russel Winder]

Andrew,

On Mon, 2013-05-06 at 08:48 -0700, Andrew Gerrand wrote:
[…]

> http://tip.golang.org/doc/code.html <- new docs

Talking of the docs and the hierarchy of a workspace, does it have to be
specifically bin for executables? The pkg directory is structured
according to platform allowing a single workspace to be shared by many
platforms, but there appears to be only a single bin which is platform
specific.

It also seems that many packages install executables directly into
$GOBIN.

--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel...@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: rus...@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder

[Andrew Gerrand]

On 6 May 2013 09:09, Russel Winder <rus...@winder.org.uk> wrote:

Talking of the docs and the hierarchy of a workspace, does it have to be
specifically bin for executables? The pkg directory is structured
according to platform allowing a single workspace to be shared by many
platforms, but there appears to be only a single bin which is platform
specific.

Correct. The pkg directory places packages in directories that reflect import paths and build platform, but there's only a single bin. That's because we typically don't (or cannot) run cross-compiled binaries on our local system.

 

It also seems that many packages install executables directly into
$GOBIN.

How do packages install anything? It's the go tool that installs executables, and it will place them in GOBIN if you have it set. 

Andrew 

[Volker Dobler]

Am Montag, 6. Mai 2013 17:30:47 UTC+2 schrieb Maxim Khitrov:

The single root also determines Go package names. As it happens, I've
been using a single root of D:\projects\<project> on my work computer
since long before learning about Go. But even this existing single
root organization is incompatible with GOPATH without adding an extra
src directory.

Are you suggesting the use of project paths like
D:\projects\src\someproject\src\package1? Seems like a mess,
especially since not everything under D:\projects\src\ is source code.
I don't really understand how you're organizing non-Go files.

This is not about taxonomy or linguistic stuff. All that is used in

building your binary/package is below src, even if not a *.go file. Most

probably it was named src because "all-the-stuff-you-manage-yourself"

or that like is ugly.

My problem (if the goal is to avoid per-project entries in GOPATH)
with bin, pkg, and src is the order of the hierarchy. I want
project\bin, project\pkg, and project\src, in addition to
project\docs, project\media, and project\otherstuff. Incidentally,
look at how Go's own repository is organized. Do you have src/repo or
repo/src?

Here is the problem: "I want". Sorry, Go is the other way around, it is

src/projectX/packageY/subpackageZ. Where your source code management

kicks in - at src, project or package level - does not matter, you choose.

 

To be clear, I'm perfectly happy with GOPATH and bin, pkg, src
structure as a way of organizing small programs (mostly Go source code
in a single main package) and reusable libraries. I find it less
suitable for large applications consisting of many non-Go files and
multiple Go packages that are not intended for sharing.

No need for sharing here. And the build tools just ignore non-Go files, so

why not have src/projectX/icons with the icons for command X?

I have src/project/{mainCommand,packageA,packageB,test,tmp,doc,data}

and only mainCommand and package{A,B} contain code (and tmp :-)

The whole stuff is much easier and flexible than the project/src/{main/test}

structure found in Java world.

V.