Still mystified by go {get,install}

[Greg Ward]

Hi all --

I've been holding back on this, on the assumption that I'm just a
newbie and I don't get it. Well, no more! I've paid my dues, I've
written my first 10,000 lines of Go, and I am still *utterly
mystified* by how go {get,install} want us to organize our packages
(where "us" means "the Go community").

Yes, I have read http://golang.org/doc/code.html. Many times.
Backwards and forwards. I *must* be missing something.

First question: if I want to checkout someone else's code to
contribute changes, *not* to use it in my own project, am I supposed
to "{git,hg} clone" it or "go get" it? I find that very few Go
packages are in a state where I can just clone them and build them.

Examples, picking on some of the packages that my project depends
on...

$ git clone http://github.com/stretchrcom/testify
Cloning into 'testify'...
[...]
$ cd testify
$ go install
go install: no install location for _/tmp/testify
$ go install .
go install: no install location for _/tmp/testify
$ GOPATH=$PWD go install
go install: no install location for _/tmp/testify
$ GOPATH=$PWD go install .
go install: no install location for _/tmp/testify
$ GOPATH=/tmp/go go install
go install: no install location for _/tmp/testify
$ GOPATH=/tmp/go go install .
go install: no install location for _/tmp/testify

Argh, give up in frustration. Guess I won't be contributing to this
project. Try another one:

$ git clone http://github.com/ogier/pflag
Cloning into 'pflag'...
[...]
$ go install
go install: no install location for _/tmp/pflag
$ go install .
go install: no install location for _/tmp/pflag
$ GOPATH=$PWD go install
go install: no install location for _/tmp/pflag
$ GOPATH=$PWD go install .
go install: no install location for _/tmp/pflag
$ GOPATH=/tmp/go go install .
go install: no install location for _/tmp/pflag
$ GOPATH=/tmp/go go install
go install: no install location for _/tmp/pflag

Same result. Maybe it's not these packages, maybe it's me. Maybe I'm
supposed to use "go get" in these cases. Am I? It's really not clear.
If so, it contradicts everything I've known about open source
development since the Apache project put their CVS server on the
Internet about 15 years ago: checkout the project, build it, and start
hacking.

Next question: how exactly are library developers supposed to layout
their source code in source control, and what is the intended mapping
of package to DVCS repository? I haven't found a clear documented
guideline anywhere, so there seem to be a couple of conventions.

1) dump *.go right in the repository root, which is messy and makes it
hard to have sibling packages in the same repo ... but it works
nicely with "go get" and "go install"

2) have a subdirectory for each package:
pkg1/*.go
pkg2/*.go
...
pkgN/*.go
which is tidy, scales nicely, and should allow both sibling and
child packages. But making "go install" understand that is confusing.
I *think* you have to run "go install ./...", but that's unclear and
not mentioned in http://golang.org/doc/code.html.

3) duplicate the "workspace" structure in source control:
src/pkg1/*.go
src/pkg2/*.go
which means I can "git clone" your repo and get right to work,
but it means that apps using your library will have painfully
long import paths, e.g.
import "github.com/sbinet/go-python/pkg/python"

I'm pretty sure this is wrong for libraries, but it makes sense for
applications that are targeted at users rather than other Go
programmers. (At any rate, that's how I've structure my application,
and it seems to work.)

Final question: same as above, but how are *application* developers
supposed to layout their source code?

Thanks!

Greg
--
Greg Ward http://www.gerg.ca
<gr...@gerg.ca> @gergdotca

[Jan Mercl]

jnml@fsc-r630:~$ go get github.com/stretchrcom/testify
jnml@fsc-r630:~$ echo $?
0
jnml@fsc-r630:~$ ls $GOPATH/src/github.com/stretchrcom/testify
assert doc.go http LICENCE.txt mock README.md
jnml@fsc-r630:~$ ls -l $GOPATH/pkg/linux_amd64/github.com/stretchrcom
celkem 4
-rw-rw-r-- 1 jnml jnml 990 úno 6 20:27 testify.a
jnml@fsc-r630:~$

-j

[Patrick Mylund Nielsen]

First path in GOPATH = dependencies, second path = my projects, is how I have it, i.e.

export GOPATH=/home/patrick/gomisc:/home/patrick/goprojects

All my projects are under their respective import paths in /home/patrick/goprojects/src. Any dependency they might need is installed into /home/patrick/gomisc when it's 'go get'-gotten. I think I've only installed one package by cloning the repo and running go install, ever. 'go get' works for all well-behaved packages.

On Wed, Feb 6, 2013 at 8:23 PM, Greg Ward <gr...@gerg.ca> wrote:

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Andy Balholm]

I think you should use go get. At least it works for me:

andys-imac:~ andy$ go get github.com/stretchrcom/testify

andys-imac:~ andy$ go get github.com/ogier/pflag

And now both of them are neatly filed in appropriate places in my GOPATH.

The root package (with no code in it) was all that was compiled for testify, 

but it's easy enough to use go install to install the subpackages that you're interested in

now that you've downloaded.

This simply combines the "checkout the project" and "build it" steps.

 

Next question: how exactly are library developers supposed to layout
their source code in source control, and what is the intended mapping
of package to DVCS repository? I haven't found a clear documented
guideline anywhere, so there seem to be a couple of conventions.

1) dump *.go right in the repository root, which is messy and makes it
   hard to have sibling packages in the same repo ... but it works
   nicely with "go get" and "go install"

If there is only one package in the repo, this is the way to go.

 

2) have a subdirectory for each package:
     pkg1/*.go
     pkg2/*.go
      ...
     pkgN/*.go
   which is tidy, scales nicely, and should allow both sibling and
   child packages. But making "go install" understand that is confusing.
   I *think* you have to run "go install ./...", but that's unclear and
   not mentioned in http://golang.org/doc/code.html.

This is the best approach for a repo that includes several related packages.

If you want to compile all subpackages at once, you can even do:

andys-imac:~ andy$ go get -v github.com/stretchrcom/testify/...

github.com/stretchrcom/testify (download)

github.com/stretchrcom/testify

github.com/stretchrcom/testify/assert

github.com/stretchrcom/testify/http

github.com/stretchrcom/testify/mock

(Here I used the -v option to make go get list the packages it was compiling.)

 

3) duplicate the "workspace" structure in source control:
     src/pkg1/*.go
     src/pkg2/*.go
   which means I can "git clone" your repo and get right to work,
   but it means that apps using your library will have painfully
   long import paths, e.g.
     import "github.com/sbinet/go-python/pkg/python"

   I'm pretty sure this is wrong for libraries, but it makes sense for
   applications that are targeted at users rather than other Go
   programmers. (At any rate, that's how I've structure my application,
   and it seems to work.)

I wouldn't recommend this configuration.

 

Final question: same as above, but how are *application* developers
supposed to layout their source code?

The same as package developers. 

If you have an application, with some packages that are installed

along with it, put the application in the root of the repo and the packages

in subdirectories. 

[Andy Balholm]

One more thought on go get. Maybe you're used to examining projects before you build them, to check for security issues in the makefiles. But go get doesn't use makefiles or shell scripts, so it doesn't execute any code from the project you're downloading; it only compiles it. So it should be safe to do something like go get github.com/l33th4ck0r/myNefariousTrojan/...; just don't run the executables it produces :-).

[Patrick Mylund Nielsen]

Indeed. This is very much not true with e.g. Ruby gems, or nearly anything else with scriptable makefiles.

On Wed, Feb 6, 2013 at 8:49 PM, Andy Balholm <andyb...@gmail.com> wrote:

One more thought on go get. Maybe you're used to examining projects before you build them, to check for security issues in the makefiles. But go get doesn't use makefiles or shell scripts, so it doesn't execute any code from the project you're downloading; it only compiles it. So it should be safe to do something like go get github.com/l33th4ck0r/myNefariousTrojan/...; just don't run the executables it produces :-).

--

[Greg Ward]

On 06 February 2013, Sebastien Binet said:
> > $ git clone http://github.com/ogier/pflag
> > Cloning into 'pflag'...
> > [...]
> > $ go install
> > go install: no install location for _/tmp/pflag
> > $ go install .
> > go install: no install location for _/tmp/pflag
> > $ GOPATH=$PWD go install
> > go install: no install location for _/tmp/pflag
> > $ GOPATH=$PWD go install .
> > go install: no install location for _/tmp/pflag
> > $ GOPATH=/tmp/go go install .
> > go install: no install location for _/tmp/pflag
> > $ GOPATH=/tmp/go go install
> > go install: no install location for _/tmp/pflag
>

> This may be a Ubuntu issue (the Travis ci not for go-python shows the same
> behavior)

Don't *think* so. I use a binary package of go 1.0.3 installed
manually in /usr/local/go. I had Ubuntu's golang-1.0.2 package
installed until 2 minutes ago; just purged it and I still get the same
result.

I'm pretty sure this is one of those cases where the right answer is
so obvious to experienced developers that they don't even realize it
needs documenting. Meanwhile newbies scratch their heads and muddle
along as best they can, wondering if they're doing it right. I'm in
the middle ground now, where I can still see the need for better
documentation, but the right way isn't *quite* blindingly obvious to
me yet.

[Dan Kortschak]

When you set $GOPATH below, you end up with:

$PWD/pflag

And $GOPATH points to the $PWD, correct? You should clone into $PWD/src/pflag at a minimum (where $GOPATH is still pointing to $PWD) to get the build to work and preferabling into $PWD/src/github.com/ogier/pflag so that other packages can import it.

[Sebastien Binet]

Greg,

This may be a Ubuntu issue (the Travis ci not for go-python shows the same behavior)

-s

Sent from my droid.

[Sebastien Binet]

[Greg Ward]

On 06 February 2013, Andy Balholm said:
> > Final question: same as above, but how are *application* developers
> > supposed to layout their source code?
> >
>
> The same as package developers.
> If you have an application, with some packages that are installed
> along with it, put the application in the root of the repo and the packages
> in subdirectories.

OK, but my users are not Go developers. They're just people who want
to play with my application, which happens to be written in Go. I want
them to

1) download the source tarball OR "hg clone" my repo
2) ./build.sh (OR build.bat... someday...)

They shouldn't have to know about "go get" or "go install". And they
definitely shouldn't know about CGO_CFLAGS or
"make -C src/github.com/sbinet/go-python/pkg/python" (to pick an
example of something that "go install" cannot build without assistance).

[Andy Balholm]

If they aren't Go developers, they probably don't have the Go compiler installed.
So it would make the most sense to just ship a binary.

[Greg Ward]

On 06 February 2013, gary b said:
> The two packages install without problems using 'go get'. If you do need
> to use 'git clone' directly, then you should clone the package to the same
> directory that 'go get' will use.
>
> $ echo $PATH
> /home/gary/gopath
> $ mkdir -p $GOPATH/src/github.com/stretchrcom
> $ cd $GOPATH/src/github.com/stretchrcom

> $ git clone http://github.com/stretchrcom/testify

> $ cd testify
> $ go install
>

> If you fork the project project to contribute changes, then you should
> clone your fork to the package's original location:
>
> $ mkdir -p $GOPATH/src/github.com/stretchrcom
> $ cd $GOPATH/src/github.com/stretchrcom
> $ git clone http://github.com/gward/testify

> $ cd testify
> $ go install

My quick history of open-source development:

1990-97? (all languages)
- download the tarball
- figure out how to build the damn thing
- hack on it and email patches

(OK, OK, I've only been doing open source development since 1993 or
'94. The culture appeared pretty firmly established by then, so I'm
assuming it goes back a bit earlier. ;-)

1997-2005 (all languages)
- checkout source (cvs, then later svn)
- figure out how to build+test the damn thing
- hack on it and email patches

2005-present (all languages except Go)
- clone repo
- figure out how to build+test the damn thing
- hack on it and send pull requests (or email patches)

2009-present (Go only?)
- "go get" the repo
- figure out how to build+test the damn thing
(progress! this is usually trivial)
- hack on it and send pull requests (or email patches)

Does this sound right? If so, that means contributing to Go projects
is different from every other language out there. That's not a BAD
thing, but it is a *notable* thing that should be clarified.

[Job van der Zwan]

Since we're on the topic of go get/install, I also have a small issue: on my new laptop it no longer automatically fetches the dependencies of a package - when I tried "go get github.com/skelterjohn/go.wde", I had manually go get the packages imported by go.wde. I thought it used to do that automatically? I'm running tip, in case that matters.

Also: is there a way to set it to use certain flags by default? I'd like to set it to use verbose output all the time.

[Jan Mercl]

On Wed, Feb 6, 2013 at 10:24 PM, Greg Ward <gr...@gerg.ca> wrote:
> OK, but my users are not Go developers. They're just people who want
> to play with my application, which happens to be written in Go. I want
> them to
>
> 1) download the source tarball OR "hg clone" my repo
> 2) ./build.sh (OR build.bat... someday...)
>
> They shouldn't have to know about "go get" or "go install". And they
> definitely shouldn't know about CGO_CFLAGS or
> "make -C src/github.com/sbinet/go-python/pkg/python" (to pick an
> example of something that "go install" cannot build without assistance).

If you would be distributing a Python app, you would have to instruct
your users to ensure they have Python installed(*) before trying to
run your app.

If you would be distributing a Java app, you would have to instruct
your users to ensure they have Java installed(*) before trying to run
your app.

If your intent is to distribute a Go app/package, you have to instruct
your users to ensure they have Go installed(*) before trying to run
your app. Then your users may just do `go get
host.foo/gregs/awesomeapp`. It's IMO simpler then to instruct them how
to

- properly clone the repo to the proper place
- setup the environment somehow (usually).
- build the app by `./build.sh` or `make` whatever
- where to install the produced stuff (think sudo hammer...)

That's why I would recommend to make your project _totally_
go-gettable. That includes e.g. having in the repo already generated
lexer(s) and parser(s) (which go get will not do for you, neither any
other preprocessing, if applicable) - instead of silently installing
some of the required/missing tools and executing them. The later
approach is a security risk too.

(*) using their favorite package manager

-j

[Jan Mercl]

On Wed, Feb 6, 2013 at 10:43 PM, Job van der Zwan
<j.l.van...@gmail.com> wrote:
> Since we're on the topic of go get/install, I also have a small issue: on my
> new laptop it no longer automatically fetches the dependencies of a package
> - when I tried "go get github.com/skelterjohn/go.wde", I had manually go get
> the packages imported by go.wde. I thought it used to do that automatically?
> I'm running tip, in case that matters.

No problem here:

jnml@fsc-r630:~$ go get -v github.com/skelterjohn/go.wde
github.com/skelterjohn/go.wde (download)
github.com/skelterjohn/go.wde
jnml@fsc-r630:~$ go version
go version devel +aee6d7fe395a Sat Jan 26 18:16:43 2013 -0800 linux/amd64
jnml@fsc-r630:~$

> Also: is there a way to set it to use certain flags by default? I'd like to
> set it to use verbose output all the time.

Setup an alias?

-j

[Job van der Zwan]

On Wednesday, 6 February 2013 22:49:28 UTC+1, Jan Mercl wrote:

On Wed, Feb 6, 2013 at 10:43 PM, Job van der Zwan
<j.l.van...@gmail.com> wrote:
> Since we're on the topic of go get/install, I also have a small issue: on my
> new laptop it no longer automatically fetches the dependencies of a package
> - when I tried "go get github.com/skelterjohn/go.wde", I had manually go get
> the packages imported by go.wde. I thought it used to do that automatically?
> I'm running tip, in case that matters.

No problem here:

jnml@fsc-r630:~$ go get -v github.com/skelterjohn/go.wde
github.com/skelterjohn/go.wde (download)
github.com/skelterjohn/go.wde
jnml@fsc-r630:~$ go version
go version devel +aee6d7fe395a Sat Jan 26 18:16:43 2013 -0800 linux/amd64
jnml@fsc-r630:~$

Well, that's my point, if you dig deep enough into the source[1], you'll see:

import (

"fmt"

"github.com/BurntSushi/xgb"

"github.com/BurntSushi/xgb/xproto"

"github.com/BurntSushi/xgbutil"

"github.com/BurntSushi/xgbutil/ewmh"

"github.com/BurntSushi/xgbutil/icccm"

"github.com/BurntSushi/xgbutil/keybind"

"github.com/BurntSushi/xgbutil/xgraphics"

"github.com/BurntSushi/xgbutil/xwindow"

"github.com/skelterjohn/go.wde"

"image"

"sync"

)

I thought go get also fetched imported packages that were missing, so shouldn't BurntSushi's packages also get fetched? Or did I misunderstand?

[1] https://github.com/skelterjohn/go.wde/blob/master/xgb/xgb.go

[Jan Mercl]

Everything works as intended. Because [1] is a different package of course ;-)

jnml@fsc-r630:~$ go get -v github.com/skelterjohn/go.wde/xgb
github.com/skelterjohn/go.wde (download)
github.com/BurntSushi/xgb (download)
github.com/BurntSushi/xgbutil (download)
code.google.com/p/graphics-go (download)
code.google.com/p/jamslam-freetype-go (download)
github.com/BurntSushi/xgb
code.google.com/p/graphics-go/graphics/convolve
code.google.com/p/graphics-go/graphics/interp
code.google.com/p/graphics-go/graphics
github.com/BurntSushi/xgb/xproto
code.google.com/p/jamslam-freetype-go/freetype/raster
code.google.com/p/jamslam-freetype-go/freetype/truetype
code.google.com/p/jamslam-freetype-go/freetype
github.com/BurntSushi/xgbutil/xrect
github.com/skelterjohn/go.wde
github.com/BurntSushi/xgb/xinerama
github.com/BurntSushi/xgb/shape
github.com/BurntSushi/xgbutil
github.com/BurntSushi/xgbutil/xprop
github.com/BurntSushi/xgbutil/xevent
github.com/BurntSushi/xgbutil/ewmh
github.com/BurntSushi/xgbutil/icccm
github.com/BurntSushi/xgbutil/keybind
github.com/BurntSushi/xgbutil/mousebind
github.com/BurntSushi/xgbutil/xwindow
github.com/BurntSushi/xgbutil/xgraphics
github.com/skelterjohn/go.wde/xgb
jnml@fsc-r630:~$

Go get doesn't care about repositories per se(*), but about packages
you name in the command argument.

-j

(*) It downloads the repo where the requested package is located, but
it _doesn't_ install automagically every (other) package which the
repo may contain. Which is a Good Thing (tm).

[Jan Mercl]

On Thu, Feb 7, 2013 at 12:20 AM, Jan Mercl <0xj...@gmail.com> wrote:
> It downloads the repo where the requested package is located, but
> it _doesn't_ install automagically every (other) package which the
> repo may contain.

Should have been noted to be correct: Unless imported from the
_requested_ package, of course. Then they must get installed as well
because otherwise the _requested_ package/command cannot be built.

-j

[Job van der Zwan]

On Thursday, 7 February 2013 00:20:19 UTC+1, Jan Mercl wrote:

Go get doesn't care about repositories per se(*), but about packages
you name in the command argument.

-j

(*) It downloads the repo where the requested package is located, but
it _doesn't_ install automagically every (other) package which the
repo may contain. Which is a Good Thing (tm).

So it installs xgb because that's imported by go.wde, but it doesn't fetch all the dependencies of xgb?

[Jan Mercl]

On Thu, Feb 7, 2013 at 12:29 AM, Job van der Zwan
<j.l.van...@gmail.com> wrote:
> So it installs xgb because that's imported by go.wde, but it doesn't fetch
> all the dependencies of xgb?

Package github.com/skelterjohn/go.wde can be seen here:
https://github.com/skelterjohn/go.wde

There are 3 .go files in the package:

- events.go (https://github.com/skelterjohn/go.wde/blob/master/events.go)
- keys.go (https://github.com/skelterjohn/go.wde/blob/master/keys.go)
- wde.go (https://github.com/skelterjohn/go.wde/blob/master/wde.go)

events.go (https://github.com/skelterjohn/go.wde/blob/master/events.go#L19):

import (
"image"
)

keys.go (https://github.com/skelterjohn/go.wde/blob/master/keys.go#L19):

import (
"sort"
"strings"
)

wde.go (https://github.com/skelterjohn/go.wde/blob/master/wde.go#L19):

import (
"image"
"image/draw"
)

I wonder where do you see that go.wde imports xgb?

-j

PS: Independent source:
http://godoc.org/github.com/skelterjohn/go.wde?view=imports

[Dave Cheney]

Job, here is a simple suggestion to solve the underlying problem

cd $YOURPACKAGE; go get -u -v ./...

Where $YOURPACKAGE is not go.wde or anything else, but your package
that you are writing. The Go tool can resolve / download any
dependencies that your package needs from there, the -u flag (which I
hope is transitive) will update packages on the way, which may expose
additional dependencies.

If that does not solve the problem then you may need to inspect
$GOROOT/src/pkg for 3rd party source that has been left in there.

[Andrew Gerrand]

On 7 February 2013 08:32, Greg Ward <gr...@gerg.ca> wrote:

  2009-present (Go only?)
    - "go get" the repo
    - figure out how to build+test the damn thing
      (progress! this is usually trivial)
    - hack on it and send pull requests (or email patches)

This is the way things are trending, and have been for a long time.

Perl:

  cpan install Foo::bar

Python:

  easy_install foo

Ruby:

  npm install foo

Go:

  go get foo

etc etc.

If people want to hack on your project, they're going to need your dependencies anyway, and so they'll need a GOPATH set up. The fastest way to get your project in a state where it can be hacked on is to use "go get", and anyone who wants to hack on your project is going to need to install Go and understand the build tools.

Embrace the go tool and your life will be better.

Andrew

[Job van der Zwan]

On Thursday, 7 February 2013 00:39:28 UTC+1, Jan Mercl wrote:

I wonder where do you see that go.wde imports xgb?

Here:

https://github.com/skelterjohn/go.wde/blob/master/init/init_linux.go

As you can see, it was a misreading on my part - for some reason I thought it was still part of the go.wde package.

@Dave: Thanks! Will try that out next time dependencies are missing.

[Shivakumar GN]

Since "go get -d" that I use often did not come up in the suggestion => it can be used to download and read code but not build.

My code organization prior Go fits well into GOPATHs structure. After Go:

1. I use "go get" to download non-go projects as well with "-d" (download only) option !

2. If GOPATH is path1:path2, then I use path1 exclusively for sources downloaded from open source projects while path2 for own sources.

3. I have a helper script to set what must be first path in GOPATH (when moving between work environments) - something like virtualenv, but this use-case has not become strong enough and I half the times use direct setting at shell level

[Nick Craig-Wood]

On 06/02/13 20:01, gary b wrote:
> The two packages install without problems using 'go get'. If you do
> need to use 'git clone' directly, then you should clone the package to
> the same directory that 'go get' will use.

There is nothing to stop you using the checkout that `go get` makes.
Just remember to switch branches (which confused me no end!)

$ go get github.com/stretchrcom/testify
$ cd $GOPATH/src/github.com/stretchrcom/testify
$ git branch
* (no branch)
master
$ git checkout master
Switched to branch 'master'
$ git remote -v
origin https://github.com/stretchrcom/testify (fetch)
origin https://github.com/stretchrcom/testify (push)

If you want to contribute via github, then you fork the repository using
the github web interface and change the remotes for your repository.

$ git remote rm origin
$ git remote add origin https://github.com/YOU/testify
$ git remote add upstream https://github.com/strechrcom/testify
$ git remote -v
origin https://github.com/YOU/testify (fetch)
origin https://github.com/YOU/testify (push)
upstream https://github.com/strechrcom/testify (fetch)
upstream https://github.com/strechrcom/testify (push)

You then commit stuff then do a `git push origin master` and submit a
pull request on github.

--
Nick Craig-Wood <ni...@craig-wood.com> -- http://www.craig-wood.com/nick

[Russel Winder]

On Thu, 2013-02-07 at 10:44 +1100, Andrew Gerrand wrote:
[…]

> This is the way things are trending, and have been for a long time.
>
> Perl:
> cpan install Foo::bar
> Python:
> easy_install foo

Probably better to do:

pip install foo

unless of course the correct package is already packaged for your
platform distribution, in which case:

(aptitude|yum|port|brew) install foo

or whatever. I guess someday, someone might create a reasonable
packaging system for Windows ;-)

> Ruby:
> npm install foo
> Go:
> go get foo
> etc etc.

--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel...@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: rus...@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder

[minux]

On Thu, Feb 7, 2013 at 5:50 PM, Graham Pyne <graha...@gmail.com> wrote:

On Thursday, 7 February 2013 06:23:20 UTC+11, Greg Ward wrote:

  $ go install
  go install: no install location for _/tmp/testify

in Go 1.1, this message will be clearer, for example:

go install: no install location for directory /Users/minux/testify outside GOPATH

I had the same problem with "go install", GOBIN needs to be set for "go install" to work correctly.

Check with "go env".

It should be set to your GOPATH + /bin

How to reproduce it? If GOBIN must be set for "go install" to work, it must be a bug.

[Graham Pyne]

Try as I might (short of a fresh install of windows 7) I haven't been able to reproduce it. :(

[Kyle Lemons]

+1

This is how I do it, and (as far as I understand it) how it is usually done with git repositories.  Key points:

0) Make sure you're on a branch (usually master) -- this is not the case after go get

1) Keep the source you're editing in the same place so you don't have to translate import paths

2) Set up remotes for upstream (the original) and origin (your fork) so you can pull updates from upstream and push edits to origin

[Kamil Kisiel]

Exactly what I do as well. It's worthwhile to familiarize yourself with your source control tools, because this workflow is fairly obvious to arrive to if you know how they work. 

The commands below are pretty much verbatim what I use with the exception of

git rename origin upstream

Instead of removing and re-adding.

[Greg Ward]

On 07 February 2013, Andrew Gerrand said:
> This is the way things are trending, and have been for a long time.
>
> Perl:
> cpan install Foo::bar
> Python:
> easy_install foo

In 2006 maybe. All the cool kids use pip these days. ;-)

> Ruby:
> npm install foo
> Go:
> go get foo
> etc etc.

Sure, for *using* dependencies. I'm a read-only consumer, I just want
to add this dependency and get back to hacking on my code: then
cpan/pip/go get/etc are just dandy.

But when I wanted to *contribute* to Foo::Bar in 1997, I downloaded
the tarball and started hacking. Ditto for foo.py. Eventually we
learned to checkout from each other's source repos, and today we clone
each others repos. Maybe I'm out of touch, but I *think* Go is unique
in encouraging use of the download/build tool to hack on upstream
projects as well. It's not *wrong*, it's just *different*, and IMHO
needs better documentation. (And yes, I'll try to come up with a
patch. Complaint without action is rarely helpful.)

Example: today I want to contribute to go-python. If I correctly
understand all the recommendations I'm getting, I'm supposed to do
this:

# create a new workspace
$ cd ~/src && mkdir go-python && cd go-python

(BTW, I'd rather not do this in my application's workspace: upstream
might reject my changes, or suggest a better way, or ignore me. I
could waste a lot of time if I make my app depend on changes not yet
accepted upstream, so I prefer to avoid the risk of that happening.)

# keep everything together in one place, so different
# workspaces don't interfere (just my personal preference)
$ export GOPATH=$PWD

# must use -d because building go-python is slightly tricky
$ go get -d github.com/sbinet/go-python/pkg/python

# let's see what upstream has been up to recently
$ (cd src/github.com/sbinet/go-python && git log)

# build it, run tests (go-python uses a Makefile to
# set CGO_CFLAGS and CGO_LDFLAGS)
$ make -C src/github.com/sbinet/go-python/pkg/python
$ go test -v github.com/sbinet/go-python/pkg/python

OK good, I'm ready to start hacking.

$ emacs src/github.com/sbinet/go-python/pkg/python/*.go

Run the "git remote" trickery that Nick Craig-Wood kindly shared.
Commit and submit:

$ (cd src/github.com/sbinet/go-python && git commit)
$ (cd src/github.com/sbinet/go-python && git push ...)

My reaction to that: wow, that's a lot of repetitive typing! But if
that's the way to embrace the go tool, sure, I'll give it a shot. I
just want to make sure I'm using the tool as intended, which is a bit
hard to figure out from the docs.

Thanks for all the replies!

[Nate Finch]

Changing gopath all the time is not necessary.  If you make gopath into two directories, say $home/code/go/extern:$home/code/go/working then anything that is automatically downloaded will go into the extern directory.  Anything searching for an import that exists there will find it there first.  You can then use dvcs to get a copy of the code you want to modify into the working directory, where it'll be "hidden" from the rest of the code (unless you don't already have a copy of the code in the extern directory).  Then you can work there to your heart's content, without making the rest of your code reference you in-progress stuff.

I don't really know how else workspaces might interfere with one another. All code is downloaded by URL... so if two of your projects reference github.com/jsmith/foo, then you only need to download it once.  Really, in this way, gopath is just a local cache of code that's already out there.

[Eric]

I still can't figure out how to use this go get too, and I would like to add few "use cases" to your list:

I'm using the latest version of the libA in my product P1 that is beeing on a very fast release pace. 

Unfortunately I'm using an older version (commit ?) of that library on my product P2, that is on a slow release pace, just small bug fixes.

Its fine if I'm working alone, but what if I'm on a team ? what version of libA will new developper get when they'll start working on P2 ?

What if instead of new developper I'm thinking "CI" ?

What if I'm starting back on that old version of my product that is 2 years old ? Which damned commit of libA was really working ?

I'm really puzzled by the question! How should I do that ?

I need to handle: team working, Continuous integration, multiple version of a same lib across several projects.

The only solution I've found was to implement gopack, and voilà.

I would really appreciate if someone could expose the way he/she works around those problem: 

• library shared among several project, with different pace
  
• team working (how to share the information about which commit to use for which library (mixing git, ang hg repo)
• continuous integration, and the ability to rebuilt years later the same project again.

Those are the major issues I've faced using go get, and the reason why I built gopack.

[wkharold]

Um, if you want to hack on a project 'go get' it, cd to the place it landed in your GOPATH, 'go install', start hacking. It's already cloned using the appropriate DVCS. That's it.

On Wednesday, February 6, 2013 1:23:20 PM UTC-6, Greg Ward wrote:

Hi all --

I've been holding back on this, on the assumption that I'm just a
newbie and I don't get it. Well, no more! I've paid my dues, I've
written my first 10,000 lines of Go, and I am still *utterly
mystified* by how go {get,install} want us to organize our packages
(where "us" means "the Go community").

Yes, I have read http://golang.org/doc/code.html. Many times.
Backwards and forwards. I *must* be missing something.

First question: if I want to checkout someone else's code to
contribute changes, *not* to use it in my own project, am I supposed
to "{git,hg} clone" it or "go get" it? I find that very few Go
packages are in a state where I can just clone them and build them.

Examples, picking on some of the packages that my project depends
on...

  $ git clone http://github.com/stretchrcom/testify

  Cloning into 'testify'...
  [...]

  $ cd testify
  $ go install

  go install: no install location for _/tmp/testify

  $ go install .

  go install: no install location for _/tmp/testify

  $ GOPATH=$PWD go install

  go install: no install location for _/tmp/testify

  $ GOPATH=$PWD go install .

  go install: no install location for _/tmp/testify

  $ GOPATH=/tmp/go go install  

  go install: no install location for _/tmp/testify

  $ GOPATH=/tmp/go go install .

  go install: no install location for _/tmp/testify

Argh, give up in frustration. Guess I won't be contributing to this
project. Try another one:

  $ git clone http://github.com/ogier/pflag
  Cloning into 'pflag'...
  [...]

  $ go install

[Nate Finch]

So, by default, go get retrieves the latest version of any repo. So you can't really have a package working from an older version. However, you can if you just fork the older version at a specific commit and have your code working against the fork.

So for example, if github.com/foo/libA has the latest code, you import against that for P1, which wants to work against latest. And fork the older commit version to github.com/Eric/libA, and use that to work with P2.

If you need to be absolutely sure you're working against a frozen codebase, you need to have control over the repo you're importing, thus the forking.

In theory, every Go developer should make sure they don't make breaking API changes in the same repo, until the Go Team can figure out how they want versioning to work.... but obviously not everyone is going to do that.

[Eric]

Le samedi 9 février 2013 01:50:01 UTC+1, Nate Finch a écrit :

So, by default, go get retrieves the latest version of any repo. So you can't really have a package working from an older version. However, you can if you just fork the older version at a specific commit and have your code working against the fork.

So for example, if github.com/foo/libA has the latest code, you import against that for P1, which wants to work against latest. And fork the older commit version to github.com/Eric/libA, and use that to work with P2.

To be clear, this implies that I'll have to change in the code every import "github.com/libA" to "github.com/Eric/libA" right?

So, If I understand the consequences, to avoid changing the code for versionning issues: prior to any go get, I need to fork the repo on github ( and launchpad, and googlecode with git mercurial and bazaar) and only use the forked one.

And, in that case, if I want to update the fork ( for instance, I've made the important release, i'm working on a new release) I need to go through every forked, and pull the original changes.

This clearly breaks the "go get" experience..

hum. I'm working on products with hundreds of dependencies, and the projects I'm starting in go are adding dependencies at a fair pace already. 

 

If you need to be absolutely sure you're working against a frozen codebase, you need to have control over the repo you're importing, thus the forking.

In theory, every Go developer should make sure they don't make breaking API changes in the same repo, until the Go Team can figure out how they want versioning to work.... but obviously not everyone is going to do that.

Yes, I've been scanning the http://go-lang.cat-v.org/pure-go-libs list, and thried to go get  a randomly-pick 20-subset, and I've succeeded for almost only 2 or 3. (by the time I also required that the code were licensed (any license was ok, but I needed at least one).

That's why I'm building www.gopack.me so libraries are guaranteed to compile ( a cross platform compilation will be executed and the result will be displayed along with the package list), and all code comes with a license.

[Russ Cox]

I'm not sure why you don't change into the directory you are working in.

export GOPATH=$PWD

go get -d github.com/sbinet/go-python/pkg/python

cd github.com/sbinet/go-python/pkg/python

git log

make

go test -v

emacs *.go

git commit

git push ...

Russ

[Andy Balholm]

On Saturday, February 9, 2013 12:19:55 AM UTC-8, Eric wrote:

Yes, I've been scanning the http://go-lang.cat-v.org/pure-go-libs list, and thried to go get  a randomly-pick 20-subset, and I've succeeded for almost only 2 or 3. (by the time I also required that the code were licensed (any license was ok, but I needed at least one).

That list is rather old. Most of the libraries probably predate Go 1 and the go get tool; some have been updated and some have not. I don't know if anyone is even maintaining it since Uriel passed away.

[minux]

On Sat, Feb 9, 2013 at 4:19 PM, Eric <er...@ericaro.net> wrote:

Yes, I've been scanning the http://go-lang.cat-v.org/pure-go-libs list, and thried to go get  a randomly-pick 20-subset, and I've succeeded for almost only 2 or 3. (by the time I also required that the code were licensed (any license was ok, but I needed at least one).

I suggest you start with http://code.google.com/p/go-wiki/wiki/Projects, which is actively

maintained.