How to manage building project that have "subpackages" in same repo ?

[xan...@gmail.com]

Hi,

Let's say I have my app in github.com/XANi/testrepo and to keep my code nice and tidy I've moved all of app's plugins into separate packages.

I've also did the same with app's APIs and some "toolbox" packages (like parsing certain types of inputs, or outputting data in certain format)

So I have plugin under github.com/XANi/testrepo/pkg1 and following advice of making everything use a "global" import path, I used 

import "github.com/XANi/testrepo/plugin_api"

in "subpackage" that is plugin and 

import "github.com/XANi/testrepo/pkg1"

in "main" package to import that plugin

The question is how I'm supposed to make changes in subpackages without doing:

* commit

* push

* rm of downloaded package

* go get to re-download it

every time ?

So far I've used a dirty trick of symlinking repo to GOPATH ( ln -s $(pwd) $GOPATH/src/github.com/XANi/testrepo ) but surely there is got to be a better way to do it ? 

I'd rather avoid having to use local import path as it often causes problems...

[James Bardin]

On Monday, November 23, 2015 at 9:46:28 AM UTC-5, Mariusz Gronczewski wrote:

Hi,

Let's say I have my app in github.com/XANi/testrepo and to keep my code nice and tidy I've moved all of app's plugins into separate packages.

I've also did the same with app's APIs and some "toolbox" packages (like parsing certain types of inputs, or outputting data in certain format)

So I have plugin under github.com/XANi/testrepo/pkg1 and following advice of making everything use a "global" import path, I used 

import "github.com/XANi/testrepo/plugin_api"

in "subpackage" that is plugin and 

import "github.com/XANi/testrepo/pkg1"

in "main" package to import that plugin

The question is how I'm supposed to make changes in subpackages without doing:

* commit

* push

* rm of downloaded package

* go get to re-download it

every time ?

I'm not sure why you would need to do this. All you code should be in your GOPATH, so any changes you make are immediately used when you `go install`.

So far I've used a dirty trick of symlinking repo to GOPATH ( ln -s $(pwd) $GOPATH/src/github.com/XANi/testrepo ) but surely there is got to be a better way to do it ? 

This might be part of your problem. The go tools don't follow symlinks, so never us any in your GOPATH.

 

[Mariusz Gronczewski]

On Monday, November 23, 2015 at 4:22:40 PM UTC+1, James Bardin wrote:

On Monday, November 23, 2015 at 9:46:28 AM UTC-5, Mariusz Gronczewski wrote:

Hi,

Let's say I have my app in github.com/XANi/testrepo and to keep my code nice and tidy I've moved all of app's plugins into separate packages.

I've also did the same with app's APIs and some "toolbox" packages (like parsing certain types of inputs, or outputting data in certain format)

So I have plugin under github.com/XANi/testrepo/pkg1 and following advice of making everything use a "global" import path, I used 

import "github.com/XANi/testrepo/plugin_api"

in "subpackage" that is plugin and 

import "github.com/XANi/testrepo/pkg1"

in "main" package to import that plugin

The question is how I'm supposed to make changes in subpackages without doing:

* commit

* push

* rm of downloaded package

* go get to re-download it

every time ?

I'm not sure why you would need to do this. All you code should be in your GOPATH, so any changes you make are immediately used when you `go install`.

I develop next version of API in branch "testing". I want other packages (as in "in other repository" ) that use that API always use "master" version of code that is considered stable, not my random experiments. If I just put everything in GOPATH I'd have to switch branch to master every time I wanted other package to use that
 

So far I've used a dirty trick of symlinking repo to GOPATH ( ln -s $(pwd) $GOPATH/src/github.com/XANi/testrepo ) but surely there is got to be a better way to do it ? 

This might be part of your problem. The go tools don't follow symlinks, so never us any in your GOPATH.

They most definitely do, or else that trick wouldn't work

[James Bardin]

On Mon, Nov 23, 2015 at 2:16 PM, Mariusz Gronczewski <xan...@gmail.com> wrote:
>
> I develop next version of API in branch "testing". I want other packages (as
> in "in other repository" ) that use that API always use "master" version of
> code that is considered stable, not my random experiments. If I just put
> everything in GOPATH I'd have to switch branch to master every time I wanted
> other package to use that
>

OK, I think I see where you're getting caught. Version of packages in
the same repo are inherently tied to one another. There's no good way
to checkout part of a repo from one branch, and part from another.
Maybe it would help if you worked with a separate GOPATH for each
project, or class of project? There are dependency tools which may
help with this, some of which vendor all dependencies in a local
GOPATH per project.

>>>
>>> So far I've used a dirty trick of symlinking repo to GOPATH ( ln -s
>>> $(pwd) $GOPATH/src/github.com/XANi/testrepo ) but surely there is got to be
>>> a better way to do it ?
>>>
>>
>> This might be part of your problem. The go tools don't follow symlinks, so
>> never us any in your GOPATH.
>
>
> They most definitely do, or else that trick wouldn't work

Maybe I should say they don't work in all cases, the tools may ignore
them, and it's discouraged.
I forget which situations trigger it, but you'll see `go install: no
install location for directory ... outside GOPATH`.

[Mariusz Gronczewski]

On Monday, November 23, 2015 at 9:39:12 PM UTC+1, James Bardin wrote:

On Mon, Nov 23, 2015 at 2:16 PM, Mariusz Gronczewski <xan...@gmail.com> wrote:
>
> I develop next version of API in branch "testing". I want other packages (as
> in "in other repository" ) that use that API always use "master" version of
> code that is considered stable, not my random experiments. If I just put
> everything in GOPATH I'd have to switch branch to master every time I wanted
> other package to use that
>

OK, I think I see where you're getting caught. Version of packages in
the same repo are inherently tied to one another. There's no good way
to checkout part of a repo from one branch, and part from another.
Maybe it would help if you worked with a separate GOPATH for each
project,  or class of project? There are dependency tools which may
help with this, some of which vendor all dependencies in a local
GOPATH per project.

Tried that a bit with gom but that still doesn't solve "import from same repo's subdir" problem. I guess that is impossible to solve without using relative ( "local" ) import path

>>>
>>> So far I've used a dirty trick of symlinking repo to GOPATH ( ln -s
>>> $(pwd) $GOPATH/src/github.com/XANi/testrepo ) but surely there is got to be
>>> a better way to do it ?
>>>
>>
>> This might be part of your problem. The go tools don't follow symlinks, so
>> never us any in your GOPATH.
>
>
> They most definitely do, or else that trick wouldn't work

Maybe I should say they don't work in all cases, the tools may ignore
them, and it's discouraged.
I forget which situations trigger it, but you'll see `go install: no
install location for directory ... outside GOPATH`.

That sounds like a bug in go tools...

[James Bardin]

On Tue, Nov 24, 2015 at 5:21 PM, Mariusz Gronczewski <xan...@gmail.com> wrote:
>
> Tried that a bit with gom but that still doesn't solve "import from same
> repo's subdir" problem. I guess that is impossible to solve without using
> relative ( "local" ) import path
>>
>>

I would say the relative import is even worse if you're trying to use
the go tools (some project go this route by using makefiles or scripts
to build).

Have you looked at gb (http://getgb.io/)? I haven't used it much, so I
don't know if it can solve this particular problem, but it is a
replacement for the go build tools that doesn't necessarily use
GOPATH.

>> Maybe I should say they don't work in all cases, the tools may ignore
>> them, and it's discouraged.
>> I forget which situations trigger it, but you'll see `go install: no
>> install location for directory ... outside GOPATH`.
>
>
> That sounds like a bug in go tools...

It was a choice, to prevent inadvertently importing the same package
from multiple paths.