code organization and clean import paths

[Jason McVetta]

When I write a package, I want it to have a nice clean import path like "github.com/jmcvetta/mypackage".  Yet it doesn't seem possible to do this, and still have my package conform to the code organization conventions specified in the docs and assumed by the go tool.  

Suppose the directory tree for my package, checked into the repository at github.com/jmcvetta/mypackage, looks like this:

mypackage/
    README.md
    src/
        mypackage/
            mypackage.go
            mypackage_test.go

Let's suppose I write an application that consumes this package.  It's directory tree is pretty similar:

myapp/
    README.md
    src/
        myapp/
            myapp.go
            myapp_test.go

The application specifies just the path to the github repo in the import statement:

package main

import(
    "github.com/jmcvetta/mypackage"
)

func main() {
    mypackage.DoSomething()
    println("Woot, woot - we did something!")
}

If I set  GOPATH=/path/to/myapp and run go get myapp the tool throws an error:

package github.com/jmcvetta/mypackage
        imports github.com/jmcvetta/mypackage
        imports github.com/jmcvetta/mypackage: no Go source files in /home/jason/work/myapp/src/github.com/jmcvetta/mypackage

I can get rid of this error by changing the import path to "github.com/jmcvetta/mypackage/src/mypackage".. but damn is that ugly.  Is there a better solution?  A quick look over various Go packages says most people are just opting to keep their .go files at the root of the repo rather than under src. 

[Aram Hăvărneanu]

Instead of setting one $GOPATH for each project, set only one global
$GOPATH for all projects. Don't put files from a project inside a src
subdirectory, each package already has a single corresponding
directory where it lives. A project can have many packages.

--
Aram Hăvărneanu

[Jesse McNelis]

On Wed, Apr 25, 2012 at 5:50 PM, Jason McVetta <jason....@gmail.com> wrote:
> When I write a package, I want it to have a nice clean import path like
> "github.com/jmcvetta/mypackage".  Yet it doesn't seem possible to do this,
> and still have my package conform to the code organization conventions
> specified in the docs and assumed by the go tool.

You should go back and read http://golang.org/doc/code.html again.
You're confusing the GOPATH workspace directory layout with the per
package directory layout.

Many people make this mistake and it seems largely due to incorrect
assumptions based on experiences with some other language's package
layouts.





--
=====================
http://jessta.id.au

[Jason McVetta]

On Wed, Apr 25, 2012 at 6:31 PM, Jesse McNelis <jes...@jessta.id.au> wrote:

You're confusing the GOPATH workspace directory layout with the per

package directory layout.

Thanks - I see that now.

 

Many people make this mistake and it seems largely due to incorrect
assumptions based on experiences with some other language's package
layouts.

There was definitely an assumption about workflow that caught me up.  In other languages (e.g. Python & Ruby) I am accustomed to fetch some code from a repository, descend into the checkout, and start work there.  Although in practice I usually check out code into my ~/work/ folder, that's just convention - the workflow goes exactly the same if I checkout into e.g. ~/foo/bar/.  These assumptions may have been helped along by Goclipse, which maps a Go workspace (containing potentially many different repo checkouts) onto an Eclipse "project".  Other Eclipse tools. e.g. Pydev map a single application (from a single repo) onto an Eclipse "project".  

It seems that when one wants to work on some Go code, one needs to checkout the repo into an existing workspace's src folder.  A repo is not a self-contained project.   One thing that made this more confusing to me, is that using a workspace seems necessary only when developing code, to make the go tool happy.  If I just want to compile existing code, I can check out the repo, set GOPATH = repo root,  issue "go get . && go build .", and it works fine.  

Another thing that confused me was the location within a Go workspace where I should check out my various repos.  A real example is a little app I am writing called 'tokenizerd', which depends on a library 'tokenizer'.  Both packages live on github.  The two are developed concurrently, so both should be writable checkouts.  Tokenizerd uses a github path to import tokenizer:

import (

    "github.com/jmcvetta/tokenizer"

)

If I check out both repos into $GOPATH/src, build fails because tokenizer package cannot be found.  Instead a folder tree like this seems best:

$GOPATH/

    src/

        github.com/

            jmcvetta/

                tokenizer/

                tokenizerd/

Although things would work okay if tokenizerd were directly below src, to me it feels better if all my checkouts live under the same folder.  The downside to this layout is that short commands like "go test tokenizerd" don't work.  Instead you need to say "go test github.com/jmcvetta/tokenizerd".  

Apologies if my post is a little long & rambling.  Since many new Go developers get confused on this point, it might be useful to clarify the documentation in http://golang.org/doc/code.html.  I want to make sure my own understanding is solid before attempting that.  

[Kyle Lemons]

On Fri, Apr 27, 2012 at 1:44 PM, Jason McVetta <jason....@gmail.com> wrote:

On Wed, Apr 25, 2012 at 6:31 PM, Jesse McNelis <jes...@jessta.id.au> wrote:

You're confusing the GOPATH workspace directory layout with the per

package directory layout.

Thanks - I see that now.

 

Many people make this mistake and it seems largely due to incorrect
assumptions based on experiences with some other language's package
layouts.

There was definitely an assumption about workflow that caught me up.  In other languages (e.g. Python & Ruby) I am accustomed to fetch some code from a repository, descend into the checkout, and start work there.  Although in practice I usually check out code into my ~/work/ folder, that's just convention - the workflow goes exactly the same if I checkout into e.g. ~/foo/bar/.  These assumptions may have been helped along by Goclipse, which maps a Go workspace (containing potentially many different repo checkouts) onto an Eclipse "project".  Other Eclipse tools. e.g. Pydev map a single application (from a single repo) onto an Eclipse "project".  

It seems that when one wants to work on some Go code, one needs to checkout the repo into an existing workspace's src folder.  A repo is not a self-contained project.   One thing that made this more confusing to me, is that using a workspace seems necessary only when developing code, to make the go tool happy.  If I just want to compile existing code, I can check out the repo, set GOPATH = repo root,  issue "go get . && go build .", and it works fine.  

Another thing that confused me was the location within a Go workspace where I should check out my various repos.  A real example is a little app I am writing called 'tokenizerd', which depends on a library 'tokenizer'.  Both packages live on github.  The two are developed concurrently, so both should be writable checkouts.  Tokenizerd uses a github path to import tokenizer:

import (

    "github.com/jmcvetta/tokenizer"

)

That's correct

 

If I check out both repos into $GOPATH/src, build fails because tokenizer package cannot be found.  Instead a folder tree like this seems best:

$GOPATH/

    src/

        github.com/

            jmcvetta/

                tokenizer/ 

                tokenizerd/

Although things would work okay if tokenizerd were directly below src, to me it feels better if all my checkouts live under the same folder.

That tree looks correct.  There should be a .git directory below each of those "leaf" directories.  What about it doesn't work for you?

 

The downside to this layout is that short commands like "go test tokenizerd" don't work.  Instead you need to say "go test github.com/jmcvetta/tokenizerd".  

go test ...tokenizerd

[Jason McVetta]

On Fri, Apr 27, 2012 at 2:33 PM, Kyle Lemons <kev...@google.com> wrote:

Although things would work okay if tokenizerd were directly below src, to me it feels better if all my checkouts live under the same folder.

That tree looks correct.  There should be a .git directory below each of those "leaf" directories.  What about it doesn't work for you?

This tree layout works fine for me.  However even after reading thru http://golang.org/doc/code.html twice, it wasn't obvious to me that this is the layout I should use.  So I am thinking about how that doc could be improved to make correct workspace layout more obvious.

 

The downside to this layout is that short commands like "go test tokenizerd" don't work.  Instead you need to say "go test github.com/jmcvetta/tokenizerd".  

go test ...tokenizerd

Awesome, that works perfectly - thanks!  While I see the "..." syntax is documented in "go help packages", it would be nice if it were also mentioned in http://golang.org/doc/code.html.

[Kyle Lemons]

If you go get a package, it puts it in the right place.  The rest (and there are lots of other things that could be included) is hard to put in that document because it would be information overload!

[Jason McVetta]

On Fri, Apr 27, 2012 at 2:58 PM, Kyle Lemons <kev...@google.com> wrote:

If you go get a package, it puts it in the right place.  

Go get fetches a read-only checkout of the repo.  That helped confuse me, because I took it as a hint that go get (and the path where it installs code) was just for installing external packages I don't plan to edit.  

It would be helpful if the doc included an example of how to work on a command and its dependency package(s), at the same time.

 

The rest (and there are lots of other things that could be included) is hard to put in that document because it would be information overload!

For sure the doc should avoid bloat and stay short.  But if a lot of newcomers are asking similar questions, that suggests there is room for improvement in  the wording.

[Kyle Lemons]

On Fri, Apr 27, 2012 at 3:21 PM, Jason McVetta <jason....@gmail.com> wrote:

On Fri, Apr 27, 2012 at 2:58 PM, Kyle Lemons <kev...@google.com> wrote:

If you go get a package, it puts it in the right place.  

Go get fetches a read-only checkout of the repo.  That helped confuse me, because I took it as a hint that go get (and the path where it installs code) was just for installing external packages I don't plan to edit.  

It's not read-only per-se.  The default github https checkouts are, but that's not a go-get issue, that's a github and git scm nuance, and I don't think it applies to bitbucket or googlecode hg checkouts.

 

It would be helpful if the doc included an example of how to work on a command and its dependency package(s), at the same time. 

If we put an example of how to do it with git, then mercurial users will want an example.  Again, it's not practical to put that on the "how to write Go code" but it is something that I suspect will make its way into the wiki as time passes.

 

 

The rest (and there are lots of other things that could be included) is hard to put in that document because it would be information overload!

For sure the doc should avoid bloat and stay short.  But if a lot of newcomers are asking similar questions, that suggests there is room for improvement in  the wording.

I have made wiki posts to try to help new Gophers, but I am no longer one and thus am only speculating about what such "common questions" would be.  It would be much more beneficial if new Gophers themselves shared such tips *hint hint* ;-).

[Jason McVetta]

On Fri, Apr 27, 2012 at 6:01 PM, Kyle Lemons <kev...@google.com> wrote:

It's not read-only per-se.  The default github https checkouts are, but that's not a go-get issue, that's a github and git scm nuance, and I don't think it applies to bitbucket or googlecode hg checkouts.

Didn't realize this was git- (or perhaps even github-) specific behavior.  That does make it more appropriate to document on a wiki page than in "How to Write Go Code".  

 

I have made wiki posts to try to help new Gophers, but I am no longer one and thus am only speculating about what such "common questions" would be.  It would be much more beneficial if new Gophers themselves shared such tips *hint hint* ;-).

Sounds reasonable - I just wrote up an example-based guide.  Since I'm not permissioned to create a new page on the Go wiki, it's currently living over here:  http://code.google.com/p/jmcvetta-contrib/wiki/GithubCodeLayout