What dependency management tool do you use?

[Johann Höchtl]

I use godep. There has been lots of rumour lately to use gb.

What do others use?

If there is a blog post available somewhere comparing the pros and cons, please provide one.
I know https://github.com/golang/go/wiki/PackageManagementTools

I also like the idea of manul to use git submodules https://github.com/kovetskiy/manul

[Henrik Johansson]

I use gb. I like it a lot and I have had no issues.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Peter Mogensen]

gvt

It's KISS and does the job.

/Peter

[Ian Davis]

On Tue, Jul 12, 2016, at 09:23 PM, Henrik Johansson wrote:

I use gb. I like it a lot and I have had no issues.

 

A quick question: how well do tools like go pprof work when your source code is managed by gb, outside of the GOPATH?

 

-- Ian

 

[Henrik Johansson]

I am unsure about pprof but running tests with race for example works great. I would guess that it works.

--

[Sean Russell]

On Tuesday, July 12, 2016 at 4:25:42 PM UTC-4, Peter Mogensen wrote:
> gvt
>
> It's KISS and does the job.

+1

gvt is the best I've used.

* It doesn't force checking in the dependencies
* The manifest is straightforward and easily committed separately
* It's nicely encapsulated entirely in the vendor/ directory
* The commands are straightforward and easy to understand and remember
* The manifest is easily hackable to get around the issues with private repositories that most (all?) vendoring tools have
* It doesn't force a workflow on the users; it is minimally invasive

--- SER

[jonathan...@live.com]

I love the tool github.com/kardianos/govendor.

Works with the standard go tools and does nothing more or less than I need.
I use it for libraries and commands and it rolls it all up.

[Matt Silverlock]

gvt - simple but robust: https://github.com/FiloSottile/gvt

[Andrew Pogrebnoy]

> I love the tool github.com/kardianos/govendor.

> Works with the standard go tools and does nothing more or less than I need.
> I use it for libraries and commands and it rolls it all up.

+1 

govendor is easy to use but has all I need. Also, it cleans vendored packages from test files, images etc.

вторник, 12 июля 2016 г., 23:15:29 UTC+3 пользователь Johann Höchtl написал:

[Nathan Fisher]

Out of curiosity if the upstream deletes their repo does a force push invalidating a sha or in some other way causes a dangling reference how do you protect against that without checking the dependency in? I'm thinking about the recent issue in March with npm (http://m.theregister.co.uk/2016/03/23/npm_left_pad_chaos/). Does it allow for intermediate caches like nexus?

[Sean Russell]

Hi Nathan,

On Wednesday, July 13, 2016 at 11:43:09 AM UTC-4, Nathan Fisher wrote:
> Out of curiosity if the upstream deletes their repo does a force push
> invalidating a sha or in some other way causes a dangling reference how do you
> protect against that without checking the dependency in? I'm thinking about

You can commit external repositories or not, whichever is your preference. gvt doesn't force you to do one or the other. The difference is whether the build process includes a step to pull the dependencies.

--- SER

[roger peppe]

Most of the Canonical Go projects use github.com/rogpeppe/godeps.
It just dependency-locks; it doesn't vendor at all (although it
doesn't preclude it).
It's a no-fuss tool and pretty stable. Good for locking library dependencies.

[Sam Vilain]

I've been using a submodule-based approach (vendetta) for a while now,
and IMHO it beats all the non-submodule alternatives.

With a submodule, you keep a *complete clone* of the imported dependency
and its history. Not just a snapshot which misses the development and
copyright history of it.

For a long time now, switching branches with a submodule has been
completely transparent and fast. It works well with all the git tooling
- log, diff, etc. The subproject is handled as a first class entity.

The bulky imported files are not included directly in your repository,
but you get instead the commit SHA1 of the dependency which precisely
specifies the version, for reproducible builds.

You can update dependencies just by cd'ing into the directory and using
'git pull', or you can use 'git submodule update -r' - standard git
tooling, no extra utility required.

If the remote repository goes away for some reason, you'll have a full
clone of the repository which will probably hold every version you care
about. You can push it up somewhere, update the URL in .gitmodules, and
new clones will work flawlessly, even when you check out historical
versions which have a different URL in that version's .gitmodules.

Overall, go's clean vendoring and general 'import' approach leads to git
submodules being a perfect fit. Everything else is now legacy.

Sam

[Nazri Ramliy]

On Fri, Jul 15, 2016 at 5:50 AM, Sam Vilain <s...@parsable.com> wrote:
> Overall, go's clean vendoring and general 'import' approach leads to git
> submodules being a perfect fit. Everything else is now legacy.

I couldn't agree more to this. Been using git submodule for a few
years, I did try giving other tools a try but it felt like too much
effort given familiarity with using submodules. The only downside is
when the third party repo is not in a git repository (mercurial), but
that can be worked around by git-hg cloning it.

nazri

[mhhcbon]

I use glide, it just works, it s per project vendor
and it just works,
and oh yeah, it uses a regular go project setup,
also i don t have to maintain a global namespace across
several separated projects.

IMHO, the right way to go.

[Mauro Toffanin]

On 12/07/16 22:23, Henrik Johansson wrote:
> I use gb. I like it a lot and I have had no issues.

+1

Gb puts emphasis on reliable and reproducible builds; none of the other
package managers have such feature, and none of them work so well as gb.

[Florin Pățan]

I feel this is a controversial statement. Godep can do this just as well, in fact any vendoring tool can do this, as long as you commit your dependencies. Can you plead elaborate how using gb changes an6 of this? Thank you.

[Mauro Toffanin]

On 16/07/16 01:43, Florin Pățan wrote:
> I feel this is a controversial statement.
>Godep can do this just as well, in fact any vendoring tool can do this,
> as long as you commit your dependencies.

That is not true. Dave Cheney explained it very well at GDG Berlin
Golang DevFest: https://www.youtube.com/watch?v=c3dW80eO88I (see at ~15:00)

Godep, and other package managers too, are at best naive work-arounds
shoehorned on top of 'go get', while gb is a genuine solution to the
problem of reproducible builds in Go.

In particular, I found the concept of 'import rewriting', used by
countless of package managers, quite irksome and in direct opposition to
the very same concept of a "reproducible build". You have no idea how
many times I have been bitten by the disruptive behaviour of the 'import
rewriting' of those package managers, much to my frustration and dismay.

> Can you plead elaborate how using gb changes an6 of this?

There is no need for me to elaborate. You can find all the answers to
your questions here: https://getgb.io and https://github.com/constabulary/gb

[Florin Pățan]

I know that talk very well since I'm the one Dave called a "compiler" at ~5:14.

All the arguments about what's reproducible or not get muted when using vendoring. However, in order for that to be more effective, releases for various packages should be versioned and that controlled via the dependency management of your choice. You can do this with revisions but eh, not very fun to do it.

On Saturday, July 16, 2016 at 10:23:00 AM UTC+1, Mauro Toffanin wrote:

On 16/07/16 01:43, Florin Pățan wrote:
> I feel this is a controversial statement.
>Godep can do this just as well, in fact any vendoring tool can do this,
> as long as you commit your dependencies.

That is not true. Dave Cheney explained it very well at GDG Berlin
Golang DevFest: https://www.youtube.com/watch?v=c3dW80eO88I (see at ~15:00)

Godep, and other package managers too, are at best naive work-arounds
shoehorned on top of 'go get', while gb is a genuine solution to the
problem of reproducible builds in Go.

In particular, I found the concept of 'import rewriting', used by
countless of package managers, quite irksome and in direct opposition to
the very same concept of a "reproducible build". You have no idea how
many times I have been bitten by the disruptive behaviour of the 'import
rewriting' of those package managers, much to my frustration  and dismay.

Import path rewriting is awful but Godep doesn't need that. Now with the "vendor/" folder (which is proposed by Go not GB) this is even better since there's no more GOPATH fiddling. It may have required rewriting in the beginning, I don't know, but since I've been using it (around 3 years) I never rewrote import statements. I cannot speak about other tools but that seems like a problem which can be avoided by reading how the tool works.

 

> Can you plead elaborate how using gb changes an6 of this?

There is no need for me to elaborate. You can find all the answers to
your questions here: https://getgb.io and https://github.com/constabulary/gb

I appreciate you like gb, it's an interesting tool, but you cannot explain why it's different compared to other solutions when it comes to those two particular problems mentioned by you so I'd kindly suggest reading more about it. 

[Henrik Johansson]

I don't really know much about the other tools but afaik the difference is more in the fact that Gb is more similar to other build systems in other languages. It focuses on making the current thing you are working on easy and simple to build rather than being a tool to work around the strangeness that the vendor thing has become. That is what is so good about it. I can use it without needing to bother about what weird hierarchical vendor folder gets imported where. Probably a bit naive but it has worked so far and it has been so nice.

OT, I think the whole recursive vendor folder handling is a mistake and probably one we will regret unless we fix it. Much better would have been to just embrace semver,  stuck to any of the numerous simple existing dep file format and it would have been done,  simple and understandable.

--

[Mauro Toffanin]

On 16/07/16 11:44, Florin Pățan wrote:
> I appreciate you like gb, it's an interesting tool, but you cannot
> explain why it's different compared to other solutions when it comes to
> those two particular problems mentioned by you so I'd kindly suggest
> reading more about it.

You are way too emotional about this topic, and a clear victim of the
Dunning-Kruger effect. Cheney has already explained all your questions.

I'm not interested in a bout, sorry :)

[Florin Pățan]

Thank you for bringing up my condition, I'll make sure I treat it, it sounds like something dangerous (nice insult by the way).

Dave has explained his own reasons about this while unfortunately ignoring some aspects which actually are crucial.

The distinguishing factor between gb and other solutions is that it allows for organizing projects into individual workspaces, which for some is useful. However this means that now I'm forced to use it if I want to compile a Go app, which is in opposition with using the standard Go tools to do it (yes, I'm sure you can probably workaround it and still compile it).

The point is:

- for reproducible builds one needs always the exact dependencies, which is achieved with the vendoring approach (and commiting the source)

- for reliable builds I don't get how gb is any different than go build

- you cannot provide arguments for why gb solves any of this while other tools don't (if you take into account the parts that are left out) with your own words, which combined with the: just look at what Dave says, results in the assumption that you do not have the required understanding of these issues, thus my recommendation to look at what they mean.

Also, I was asking for an explanation on why this is different because I'm too stupid to understand it but I guess I'll remain as such since I still don't have an explanation for it after all this exchange.

Meanwhile, if you or someone else wants too look into this, there are a few resources on this:

- https://docs.google.com/document/d/1xrV9D5u8AKu1ip-A1W9JqhUmmeOhoI6d6zjVwvdn5mc/edit?usp=sharing

- https://github.com/sdboyer/gps/wiki (docs for the tool which resulted from the above document and the discussion based on it, https://github.com/sdboyer/gps)

- a Slack channel dedicated for discussion on this: https://gophers.slack.com/messages/vendor/ (you can get the invite from here: https://invite.slack.golangbridge.org/

I'm not interested in a bout, sorry :)

 I had to look up what bout means and so today I've learned, thank you.

P.S. See why I said this is controversial?

[Henrik Johansson]

For repeatable builds there are many ways and tools that work fine, Gb is not unique in this respect.

I don't get why this should be a controversial topic however. It seems to be an issue only in the go community.
In all the other languages (newer more modern anyway) it is considered a solved problem.
Sure different tools pop up now and then but they all work in the same general way.

I am sure there is a reason but why can't we just settle on the same idea that everyone else is doing?
Applications and libraries would all then use the same principal way of resolving dependencies and different tools could handle conflicts in
whatever way the author likes. Vendoring or not it would work regardless.

--

[Peter Mogensen]

On 2016-07-16 11:22, Mauro Toffanin wrote:
> On 16/07/16 01:43, Florin Pățan wrote:
>> I feel this is a controversial statement.
>> Godep can do this just as well, in fact any vendoring tool can do this,
>> as long as you commit your dependencies.
>
> That is not true. Dave Cheney explained it very well at GDG Berlin
> Golang DevFest: https://www.youtube.com/watch?v=c3dW80eO88I (see at ~15:00)

Interesting...
It's somewhat the same as we use:
* Don't use "go get"
* Have GOTPATH=${PWD} in project root.
* Have a /src directory in the project containing a link to project root
(often ../..) at the package name.
* Use gvt for vendoring everything into /vendor

... and you are free to do as any other non-go project.

I ended up with this since I had a multi-language project which I needed
to build debian pakcages too. And I could fulfill all the laguagees
different "demands" for layout (incl. go) at the same time as having
deb-helper recognize the build systems.
But the above makes it work and doesn't get in the way.


But wrt. to the talk above: Didn't Dave Cheney dismiss the diamond
problem a little too fast? Sure you "cannot" do that, but you can still
end up in a situation where you depend on 2 different libraries, each
which depends on different specific version of the same 3rd package.

> In particular, I found the concept of 'import rewriting', used by
> countless of package managers, quite irksome and in direct opposition to
> the very same concept of a "reproducible build". You have no idea how
> many times I have been bitten by the disruptive behaviour of the 'import
> rewriting' of those package managers, much to my frustration and dismay.

Yeah... the idea of rewriting import paths has always felt like a bad
hack of a workaround.

/Peter

[Mauro Toffanin]

On 16/07/16 13:51, Florin Pățan wrote:
> Thank you for bringing up my condition, I'll make sure I treat it, it
> sounds like something dangerous (nice insult by the way).

This is ridiculous. Now you're even making things up: the DKE is not a
mental/health condition or a disease, but it's just a specific form of
bias; DKE means you're drawing wrong conclusions about something, or
someone, without having a proper knowledge about that 'something', or
'someone', due to personal biases. That phrase of mine only meant that
you're assuming too much about my person and my knowledge on the topic
of reproducible builds. Nothing else.

If you want to learn more about DKE, I suggest the original peer-review
paper from 1999:
http://psycnet.apa.org/index.cfm?fa=buy.optionToBuy&uid=1999-15054-002
It's a fascinating lecture.

Apart from that, seriously, please stop jumping to conclusions and to
stir up controversy for the sake of it.

> Dave has explained his own reasons about this while unfortunately
> ignoring some aspects which actually are crucial.
> The distinguishing factor between gb and other solutions is that it
> allows for organizing projects into individual workspaces, which for
> some is useful. However this means that now I'm forced to use it if I
> want to compile a Go app, which is in opposition with using the standard
> Go tools to do it (yes, I'm sure you can probably workaround it and

> still compile it). [...]

I don't care. If you have problems with Gb, or the way it is
implemented, please go and complain with Cheney. I'm pretty sure he will
be happy to receive constructive criticism from you.

Barking at me won't solve your issues with Gb.

> - you cannot provide arguments for why gb solves any of this while other

> tools don't [...]

Please, stop taunting me with your non-sense. I've already told you that
the explanation to that question is available into Gb's documentation
(and it's written in a simple and clear way). Actually, all the
technical choices made by Cheney are discussed and documented as RFCs;
all you have to do is to read them. I WON'T repeat them here, because
I'm not interested in a technical discussion about Gb's pros and cons,
and I don't have time for these kind of silly skirmishes about topics
that should never be controversial in the first place. I'm not here to
prove a point, or promoting my own PM, oppositely to you.

Package managers, and reproducible builds, have already been discussed
ad nauseam in this ML, and I'm not going to start a new one; the search
function of Google Groups is there to help you dig all those past
discussions and unravel the mystery. But feel free to pick up a new
challenger, if you really must have bouts for championships; I won't
stop you ;)

> Also, I was asking for an explanation on why this is different because
> I'm too stupid to understand it but I guess I'll remain as such since I

> still don't have an explanation for it after all this exchange. [...]

See my previous comment about RFCs wrote by Cheney; there is no need to
repeat myself for a third time, isn't it?

> P.S. See why I said this is controversial?

I only see someone who desperately craves for this topic to be
controversial. I surely don't.

[Mauro Toffanin]

On 16/07/16 15:16, Peter Mogensen wrote:
> * Use gvt for vendoring everything into /vendor

You would be surprised to know that gvt uses gb internally, and both the
projects exchange pieces of code and ideas ;)

> But wrt. to the talk above: Didn't Dave Cheney dismiss the diamond
> problem a little too fast? Sure you "cannot" do that, but you can still
> end up in a situation where you depend on 2 different libraries, each
> which depends on different specific version of the same 3rd package.

I actually agree with Cheney's decision:
http://dave.cheney.net/2015/06/09/gb-a-project-based-build-tool-for-the-go-programming-language

The probability of encountering a diamond dependency conflict is so low
that makes sense to treat it like a corner case and to force the final
user to manually take action; providing some boiler-plate solver will
never guarantee a valid consistent resolution for every single
conflicting case, leading only to an increased complexity¹ of the tool.

Automatically solving the diamond dependency conflict requires tampering
with the Golang source code anyway, which means we are back to the
square one with dirty work-arounds: with source code tampering you lose
the "reliable" and "reproducible" part of your builds.

One could argue that solving the diamond dependency in an automatic way
has benefits per se. I can understand the allure of that reasoning, but
it's not the Go way of doing things; it's like with Generics: some
people like them and want them, some others don't and prefer to stick to
manual implementation.

There isn't an easy answer to such problems.

¹- you have to deal with graph theory, implement Breadth First and Depth
First algorithms, impose barriers and fences all over the PM; I
personally don't see the advantage considering its complexity vs the
little outcome. It's easier, and faster, to manually solve the conflict.

[John Souvestre]

> You would be surprised to know that gvt uses gb internally,
> and both the projects exchange pieces of code and ideas ;)

That's not the impression I get. I might be wrong, but I believe that gvt
does not share most of gb's philosophy: 1) It is based on just gb-vendor, not the rest of gb. 2) It uses Go's vendor directory without the top-level (single project) restriction, and in a manner which is compatible with the go tool (no vendor/src). 3) It uses the go tool, not the gb replacement tool. 4) It uses, rather than ignores, GOPATH. 5) It provides some support for versioning.

John

John Souvestre - New Orleans LA

[John Souvestre]

> Gb puts emphasis on reliable and reproducible builds; none of the other

package managers have such feature, ...

None? Isn't that the main purpose of a package manager. I see many which seem to do this in addition to gb, for example: godep, glide, govendor, and gvt. Perhaps you weren’t aware of them?

John

John Souvestre - New Orleans LA


-----Original Message-----
From: golan...@googlegroups.com [mailto:golan...@googlegroups.com] On Behalf Of Mauro Toffanin
Sent: 2016 July 15, Fri 04:48
To: golan...@googlegroups.com
Subject: Re: [go-nuts] What dependency management tool do you use?

[Ian Lance Taylor]

On Sat, Jul 16, 2016 at 5:29 AM, Henrik Johansson <dahan...@gmail.com> wrote:
> For repeatable builds there are many ways and tools that work fine, Gb is
> not unique in this respect.
>
> I don't get why this should be a controversial topic however. It seems to be
> an issue only in the go community.
> In all the other languages (newer more modern anyway) it is considered a
> solved problem.
> Sure different tools pop up now and then but they all work in the same
> general way.
>
> I am sure there is a reason but why can't we just settle on the same idea
> that everyone else is doing?
> Applications and libraries would all then use the same principal way of
> resolving dependencies and different tools could handle conflicts in
> whatever way the author likes. Vendoring or not it would work regardless.

My understanding is that what other languages do also has problems.
If there is a clear solution here, I certainly think we should adopt
it.

Can you explain what it is that other languages do, and how we should
change to do the same thing?

Ian

[Mauro Toffanin]

On 16/07/16 18:43, John Souvestre wrote:
> [...] I might be wrong, but I believe that gvt

> does not share most of gb's philosophy:

Actually, I didn't write that gvt shares any of Gb's philosophies in the
first place. I wrote that the two PM shares SOME ideas and pieces of
code. Nothing else.

> It is based on just gb-vendor, not the rest of gb

I know. But you took my phrase out of context and than you draw wrong
assumptions. You should have paid attention to the context too, which
was (verbatim quote from Peter Mogensen): "Use gvt for vendoring
everything into /vendor".

[Peter Mogensen]

On 2016-07-16 17:59, Mauro Toffanin wrote:
>> But wrt. to the talk above: Didn't Dave Cheney dismiss the diamond
>> problem a little too fast? Sure you "cannot" do that, but you can still
>> end up in a situation where you depend on 2 different libraries, each
>> which depends on different specific version of the same 3rd package.
>
> I actually agree with Cheney's decision:
> http://dave.cheney.net/2015/06/09/gb-a-project-based-build-tool-for-the-go-programming-language
>
> The probability of encountering a diamond dependency conflict is so low
> that makes sense to treat it like a corner case and to force the final
> user to manually take action;

Yes... I would agree with that too.
... since there's in principle no way to know whether it's at all
possible to resolve that conflict without patching the libraries to
support the same version of the common dependency.

/Peter

[Henrik Johansson]

Well they have issues too of course. The most recent left-pad debacle is most striking.  

In general they do have support for versions all of them. Not all are SemVer, the older seems more fluid on the interpretation of what is a valid version but they have it. There are often many or at least a couple of different tools for each language for example Maven, Gradle, SBT, Ivy etc in the Java landscape. Ruby has a number of such tools as does python. In the static compiled family the two outstanding examples are Haskell and Rust. 

Cabal has had it's problems but the community seems happy with it. The Rust people are all on board with Cargo it seems and it is afaik the solution that would work best for Go to look at. It is in essence a declarative file with dependencies (and some extra stuff) which would be super easy for Go to adopt. Even having the go tool do it seems very easy. The different tools behave differently in how they work to resolve compatible versions. Some have a range notation, some have default selection and some bail. Bailing or allowing the user to choose seems the best possibly with a range notation. 

This you are no doubt aware of already, I just wanted to list where I come from when I say that it is considered solved. 

Not that all possible kinks are hammered out but that it seems to work in a way that a large number of people are OK with. 

Personally I really like the vendor concept but the thing where one of my deps can have the same type as a dep as I do but from my perspective the types of these two are different when they really are the same seems very fishy. 

Flattening makes it better but then something can break because the wrong "version" was chosen. 

Having a common versioning scheme would make this much easier and safer even if we occasionally would run into the diamond problem. At least we would know that there was an issue and can take steps to fix or work around it. 

Having versions would also not hinder tools to resolve and fetch all deps every time if they want to nor does it stop vendoring. In my mind what we have today is hidden versions without any easy way to see or choose what we want.

Having the go tool be version aware may be a too invasive change but it would not be hard.

I don't know, I feel as though I am missing something. It feels so obvious to me that when you guys don't see it the same I start wondering what I missed. 

Hope it makes some sense. 

[William Madison]

FWIW, go build in versions >= 1.6* is basically equivalent to what GB buys us just with a different structure for the vendor directory (i.e /vendor vs /vendor/src in GB). The gb-vendor plugin is a jewel in its simplicity and makes fetching/managing vendored packages very easy, which is what I love about GB as a build tool specifically when the end product is a binary. If I were authoring a library I'd definitely stick to the go tool chain given that GB projects aren't go gettable.

- Will

--

[Henrik Johansson]

Really? Does it flatten dependencies or will I still have potentially more than one context package for example? 

[Mateusz Czapliński]

W dniu wtorek, 12 lipca 2016 22:15:29 UTC+2 użytkownik Johann Höchtl napisał:

What do others use?

Where I work, we've evaluated what was available some year ago, but nothing fit our envisioned usecases. So we took our time, sit down and meticulously analyzed what we need, wrote design docs, argued over them hard, and finally wrote the code. Since then, we've never looked back, it just works for us, we don't even think about it as a problem anymore. We've even open-sourced it; but we didn't advertise, because we weren't at freedom to add the final, yet so important touch: of writing a proper readme... I find it extremely unfortunate and painful from point of view of this project, but internally we had to quickly shift our gears and really focus on much, much more important stuff - which still keeps us really busy.

But enough explaining myself; what I'd really like is to summarize pros & cons of the tool below, plus add some links for further reading for anyone interested:

Features:
- main command "vendo recreate" analyzes all .go files in your repo, across all build tags and platforms; then recursively analyzes all their external dependencies (now assuming only predeclared platforms and default build tags). Whole repos (because LICENSE & consistency) are copied into _vendor/ subdir (because `go test ./...` & `go build ./...`), registered in vendor.json (older version which still supported our approach) with revision ID and precise date (awesome in cases like code.google.com winding down and everyone going hg->git). Git, hg, bzr are supported as source repos (more can be easily added). Code from vendored repos is `git add`-ed automatically, ignoring .git/.hg/.bzr dirs; unused repos are detected and auto-removed.
- private patching is explicitly supported by design ("comment" field in vendor.json should be updated with patch description); though public forks are practically superior and more encouraged.
- `go get -u` functionality is supported via "vendo update". Selected upstream repos are cloned based on info in "vendor.json", then updated as expected.
- permissive BSD license (3-clause).

Limitations:
- does not currently use the "canonical" vendor/ directory structure, thus requiring: `GOPATH=$ROOT/_vendor:$GOPATH`. Reasons: mostly for ease of `go build ./...`, `go test ./...` etc; also vendor/ was not yet certain when we did it. TODO: support vendor/ and add some helper wrapper for ./... ?
- no README yet. Sorry :(
- uses older version of vendor.json specification (many changes were done later, while ignoring backwards compatibility, and we noticed them much too late);
- currently supports only git as the "master" VCS;
- we planned for some helpful `git commit`-time hooks, but didn't implement them yet; lack of them doesn't really hurt us enough that we could with straight face allocate some time for it; at least for the time being.

More details:
- the original use-cases design doc: https://github.com/zpas-lab/vendo/blob/master/use-cases.md
- example results:
  https://github.com/zpas-lab/vendo/blob/master/vendor.json
  https://github.com/zpas-lab/vendo/tree/master/_vendor
- the main repo: https://github.com/zpas-lab/vendo/

NOTE: If anyone has questions, you're very welcome, but *please* make sure my email is in CC. Unfortunately I don't have enough time now to check golang-nuts regularly :/

Best Regards,
/Mateusz Czapliński.

[Nicolas Grilly]

On Saturday, July 16, 2016 at 3:17:11 PM UTC+2, Peter Mogensen wrote: 

It's somewhat the same as we use:
* Don't use "go get"
* Have GOTPATH=${PWD} in project root.
* Have a /src directory in the project containing a link to project root
(often ../..) at the package name.
* Use gvt for vendoring everything into /vendor

I ended up with this since I had a multi-language project which I needed
to build debian pakcages too. And I could fulfill all the laguagees
different "demands" for layout (incl. go) at the same time as having
deb-helper recognize the build systems.

This is an excellent "recipe". I do something similar for a multi-language project (Python + Go).

[mhhcbon]

I will only put some lights on my experience.

Gopath ect
Except at the very beginning of my experience with Go, i don't think anymore about my GOPATH.
I had setup a go work space as defined by the go core team, and declared gopath appropriately.
Since then, every cli project or library, I create will have its folder into the GOPATH
(smthg like ~/gow/src/github.com/mh-cbon/<project>, being a gh user)
Theirs dependencies will never be stored at the root of this path,
they will be stored into the per project vendor/ folder.
This folder is always ignored and is automatically managed
by the dependency manager tool i use, glide.

If i want to modify a dependency, i will fork it.
In this case the fork will exist as a root project into my GOPATH.
The projects depending on it will be modified to import the fork
rather than the origin until somehow the merge to upstream occurs.

Reproducible build

To provide reproducible build, and thus manage my dependencies,
i have decided to use glide because it does understand the semver mechanic
and it naturally uses go engine mechanics.

For each project, a file describing the dependencies exists on the root of the project.
When the project needs to be built against a fresh environment, glide install command
will do the necessary to fetch into the vendor/ folder the right dependencies.
According to the semver rules defined along the dependency graph.

Having reproducible is an important matter, but for flexibility
semver allows to declare ranges over exact version number
(ie: ~1.1.1, >=1.1.1 && <1.2.0).
This feature is extremely important when you depend on projects
you don t maintain yourself, patches will be applied to the final
product without human effort.

It is also true that such behavior induces a weak contract of confidence between
a provider and a consumer which do not closely work together and can lead to
breakage when the provider did not respect the semver rules.
(IE: publish a new version as a patch instead of a minor / major one)
Or, if the consumer did not appropriately declared its dependency rules.

To fix that instability in the dependency graph of the project
 a solution provided by npm was to add a new file
to declare strict dependency rules based not on the latest version available,
but on the version currently installed and consumed by the project
(IE: lookup into the vendor/ folder to discover the current version).

Such feature must exist for production delivery system.

Diamond problem

The situation can occur, and will occur for sure.
There is no ultimate solution to it.
There are some situations where given the dependency tree and its
semver rules, an automatic answer can be provided with no guarantee
of success besides the verbal semver agreements.

In case of breakage after automatic resolution, one should be able
to refine the version at the root project level.
Refine means to enforce a specific version range within the currently acceptable range
of versions in the dependency graph.

If I come to pull in a new dependency which is not compatible
with the current dependency graph of my project,
I will be happy the system forbids me to continue that way.

Other stuff

- So far, i never read anything about the engine compatibility.
I guess, until now, we leave in a wonderful land, every go release are
always backward compatible.
I hope time will come where the go engine will break, for a brighter future.
but, yet anyone can foresee that so far libraries and projects were not tagged
to mark their engine compatibility, which will results into a more chaotic world.

- Until now, go documentation was primarily focused to provide
exact and precise behavioral documentation of the api.
I wish some educational effort will be made to detail, explain
and provide ready-to-go methodologies to setup and publish a project
with the right guidance to embrace social coding
(IE: a global namespace is not a good way to embrace social coding and lazy dependency).

Go team has provided, what i suspect a tremendous effort, to
provide a really simple and easy environment to build programs.
go build and consorts are really awesome.
cross platform build capability is even more awesome.
go fmt is a strong and good decision.

Go, as a language, is really enjoyable,
as an environment to publish and share projects, it lacks some efforts.
Let s hope this road toward simplicity will be continued and reach the last kilometer
to provide a secure and friendly environment for social coding.

[Jeremy Echols]

I hesitate to respond with all the emotions flying high, but I love gb.  Its philosophy seems to match my own very well, pulls dependencies nicely with the built-in vendor plugin, doesn't force vendoring, allows caching semver projects externally by version number, lets my code live in src/ (which I generally prefer to top-level code mixed in with readmes), etc.

I wouldn't claim it's the best, but it's definitely eased some of my biggest frustrations with vanilla `go build`.

[Johann Höchtl]

Instead of thanking all reporters individually I'll do it here. I didn't expect a clear winner but some of the rationales for prefering one dependency mgmt tool over the other became more clear to me.

To sum it up: I'll stick with godeps. It seems it hits somewhere the sweetspot between following go's way to organize source code and provides a thin, meaningful utility layer above.

I aknowledge again gb - an inredible effort and again the reasons drawn are understandable. But I am not adventurous enough and rather stick with the crowd.

I feared that this delecate topic, comparable to browser wars, will start some heated discussions, that was not my intention.

Thanks to the contributors to this discussion!

[roger peppe]

On 24 July 2016 at 13:14, Johann Höchtl <johann....@gmail.com> wrote:
> Instead of thanking all reporters individually I'll do it here. I didn't expect a clear winner but some of the rationales for prefering one dependency mgmt tool over the other became more clear to me.
>
> To sum it up: I'll stick with godeps. It seems it hits somewhere the sweetspot between following go's way to organize source code and provides a thin, meaningful utility layer above.

I'm presuming you mean "godep" here. godeps is a different tool with
an unfortunately
similar name (they were both developed independently at roughly the same time).

[Johann Höchtl]

You are right, godep
https://github.com/tools/godep
Was typing on mobile phone out of memory.
However the submodule approach of godeps is also compeling