How to organize a go project that also includes other languages?

[Chris Shroba]

I'm working on a project that has several subprojects.  Some of these are in Go and some of these will be in Python.  I am trying to figure out the best way to structure this project as a whole, and would love any advice.  My current thought is to do something like this:

myproject
├── go-subproject1/
│   ├── bin/
│   ├── pkg/
│   └── src/
│       ├── bar.go
│       ├── foo.go
│       └── main.go
├── go-subproject2/
│   ├── bin/
│   ├── pkg/
│   └── src/
│       ├── bar.go
│       ├── foo.go
│       └── main.go
└── python-subproject/
    └── src/
        └── com/
            ├── __init__.py
            └── myusername/
                ├── __init__.py
                └── api/
                    ├── __init__.py
                    ├── bar.py
                    └── main.py

I could put a one-line shell script in each Go subproject's root directory to export the correct GOPATH for that subproject to run when I am working on that subproject, but I'm just trying to figure out if there is a better way to do this. I'm new to Go, and I'm also unsure of how I would manage this all with GitHub too, making it so that go could install things correctly automatically from GitHub too.  

One other solution I thought of was to simply break out the subprojects into their own repositories (instead of having one repo for everything), and namespacing them as "projectname-api", "projectname-queuer", etc., but I think I would rather contain everything as one repository.

Any advice would be greatly appreciated!! Thanks.

[Lars Seipel]

On Sat, Jan 17, 2015 at 07:23:31PM -0800, Chris Shroba wrote:
> Any advice would be greatly appreciated!! Thanks.

You shouldn't take the name GOPATH too literally. It doesn't mean that
there shall only be Go source code beneath it. It just tells the go tool
where to look for source files. You probably have a directory called
"src" somewhere where you kept your code before starting with Go. Just
set GOPATH to be the parent of that (or anything else you feel
comfortable with) and then leave it at that. Personally, I just set it
to my home directory so that I have my code at ~/src and my binaries at
~/bin (which you probably already have in your PATH, anyway).

Regarding project structure, I'd just use something like that:

GOPATH
└── src
└── path
└── to
└── myproject
├── .git
├── subproject_written_in_fortran
├── subproject_written_in_go
└── subproject_written_in_python

If you keep your code at Github just make "path/to/myproject" match
"github.com/user/repo" and it'll be go-gettable.

[Frank Schröder]

Here is what we have done with our Java/Go project. We have created a single git repo which contains sources, scripts, configs, vagrant boxes, go dependencies, ... to make the project self-contained. Go dependencies are vendored in manually and we note the version in the git commit log (go get xxx, rm -rf xxx/.git, git add xxx, git commit -m 'Vendored in xxx version yyy'). master contains the prod code, development happens on feature branches. Since you always have everything you can easily switch between different prod and test versions.

I've created shell script under bin/ to control the build process. I've also created a Makefile under src/go/src/project which allows us to build the code from the command line. It sets the GOPATH, adds ldflags to include the version (git repo), and cross-compile binaries (i.e. make linux, make test, make installall). 

With this setup we can build all of our go binaries with "GOPATH=<git repo>/src/go go install project/..." I've got another helper which sets the GOPATH to the right directory when I'm in this project so that I can just run go build, go install, ...

├── bin

├── etc

│   ├── nginx

│   └── ssl

├── log

├── src

│   ├── go

│   │   ├── bin

│   │   ├── pkg

│   │   └── src

│   │       ├── project

│   │       │   ├── lib

│   │       │   ├── svc

│   │       │   ├── thrift

│   │       │   └── tools

│   │       ├── code.google.com

│   │       ├── git.apache.org

│   │       ├── github.com

│   │       ├── gopkg.in

│   │       └── labix.org

│   ├── java

│   │   ├── service1

│   │   └── service2

│   └── thrift

└── vagrant

Let me know if you want more detail.

Frank

[Nick Craig-Wood]

On 18/01/15 03:23, Chris Shroba wrote:
> I'm working on a project that has several subprojects. Some of these
> are in Go and some of these will be in Python. I am trying to figure
> out the best way to structure this project as a whole, and would love
> any advice. My current thought is to do something like this:
>
> |myproject
> ├── go-subproject1/
> │ ├── bin/
> │ ├── pkg/
> │ └── src/
> │ ├── bar.go
> │ ├── foo.go
> │ └── main.go

You need another subdirectory under src - you don't want to put your go
files in the root.

> ├── go-subproject2/
> │ ├── bin/
> │ ├── pkg/
> │ └── src/
> │ ├── bar.go
> │ ├── foo.go
> │ └── main.go
> └── python-subproject/
> └── src/
> └── com/
> ├── __init__.py
> └── myusername/
> ├── __init__.py
> └── api/
> ├── __init__.py
> ├── bar.py
> └── main.py|
>
>
> I could put a one-line shell script in each Go subproject's root
> directory to export the correct GOPATH for that subproject to run when I
> am working on that subproject

I've used pretty much that approach. It has one disadvantage though
assuming that the above is all checked into one version control system.

When you do go get, it will make extra directories in
myproject/go-subproject/src and these will annoy you when you do git
status, etc. You can .gitignore them, vendor them, or use a GOPATH with
more than one directory in (which works but has disadvantages).

> One other solution I thought of was to simply break out the subprojects
> into their own repositories (instead of having one repo for everything),
> and namespacing them as "projectname-api", "projectname-queuer", etc.,
> but I think I would rather contain everything as one repository.

You might find that easier

--
Nick Craig-Wood <ni...@craig-wood.com> -- http://www.craig-wood.com/nick