[ANN] gonuts.io - centralized repository for versioned Go packages

[Alexey Palazhchenko]

Good news everyone!

I'm happy to announce a preview of http://gonuts.io/ – centralized repository for versioned Go packages.

Why do I think Go ecosystem need this? There are two problem "go get" can't solve.

First, it doesn't support package versioning – it always installs latest version. If someone wants to install previous one, he/she has to use git/hg/svn/bzr manually. Therefore, package authors are forced to maintain backward compatibility since first commit. If they want to remove some API, they should use a different repository.

Second, in practice many developers are moving their code to other places (for example, from Google Code to GitHub), renaming repositories (web.go become web) or just deleting them (at the time of this writing many links in Go Projects Dashboard and GoPkgDoc are dead). Yes, it's a social problem, but we still should deal with it.

So how can we solve those problems? Big companies typically have special repositories for third-party code. Once imported there, code is never deleted. And they have a budget to fix their world of dependencies. So, "go get" probably works okay for "google/..." packages. Smaller companies and individual developers are able to bundle third-party packages with their application and take pain of updating them only when needed. But what should package developers do if they want to use other packages?..

gonuts.io for Go, similar to PyPI for Python, RubyGems.org for Ruby and NPM for Node.js should solve those problems. Published packages (called "nuts") are never deleted, and versioning schema allows to install exact version. There are plans to allow to install a version matching pattern (like 2.*.*), but still be in control (similar to RubyGems' Bundler). And nut client tool was designed to work along with Go conventions: nuts are installed into workspace specified by GOPATH environment variable, and imported as "gonuts.io/test_nut1/0.0.1".

There are few more things I need to do before official launch. First of all, I want to provide a clear path for transition of well-known packages to gonuts.io without names being squatted. So for now gonuts.io works in preview mode, and all nuts will be removed before going into real production.

So – feel free to try Go Nuts, publish your packages, install other and post your comments into this thread or gonu...@googlegroups.com discussion group. You even may contribute. ;)

Thanks.

–-–
Alexey "AlekSi" Palazhchenko

[Jimmy Zelinskie]

There were rumors of Andrew Gerrand doing something with Gary Burd (gopkgdoc author) in order to create a central repo a while back. I know making a central repo has been something on the chopping block, but I don't know if any progress has been made. It's nice to see some initiative towards solving the problem. Even if this doesn't become the de facto repo, I'm sure its innovations will drive future solutions to be more fully featured. I'm looking forward to seeing the community response.

Best Wishes

[Andrew Gallant]

I also like this idea, and I'm glad you've taken the initiative to get
something rolling.

I do have one concern, and I think it's something that has to be
addressed sooner rather than later. That is, the import path naming.
The way I see it, there are two choices:

gonuts.io/username/package-name

Or

gonuts.io/package-name

From your post, it is my understanding that you have chosen the latter
scheme.

The latter is shorter and could be useful to represent "canonical"
packages. But there's a practical problem here: how is a package
determined to be canonical? There are various solutions, but I'm
personally not a big fan of any of them. (What happens when a project
dies? Or if another project becomes more popular?) With the first
scheme, a package can only become canonical through use and
popularity, which IMO, is the "right" way.

Secondly, I think the latter scheme hinders an important feature of
the Go tool: it is stupidly easy to fork an existing project, and add
your own changes. I think the latter scheme makes this more
inconvenient due to having to choose a different name. On the other
hand, the first scheme has the nice property of just having the
username change while the project name remains the same.

- Andrew

[Kevin Gillette]

Agreed. In my experience, the major failings of centralized package systems from other languages is they use a single flat namespace. To be effective, and exist within the social/collaborative development model, you must give each user their own namespace. Besides just the forking problem is the general name collision problem: there are no doubt many goutil packages out there, which more than likely have nothing to do with each other -- it would be detrimental (and drive potential users away) if you provisioned package names on a "first come first serve" basis.

[David DENG]

Good try!

But the name sounds a little bit confusing.

David

[Jesse McNelis]

On Mon, Dec 10, 2012 at 12:50 PM, Andrew Gallant <jam...@gmail.com> wrote:

The latter is shorter and could be useful to represent "canonical"
packages. But there's a practical problem here: how is a package
determined to be canonical?

I agree, this is a problem that exists in all other package systems.

Popularity isn't a good measure of canonical either. 

The fact that a package has the name "email" means that it's much harder for 

any other possibly better email package to get popular.

Bad early packages become popular and stay popular due to their names.

Developers that come later, in 5 years, will find that all the names are taken. 

So they have to make up nonsense names for straight forward packages.

pypi and gems are great examples of this.

I'm not a fan of explicit package versioning either. 

Explicit versioning allows packages to continue to justify depending on older versions of other packages.

The result is multiple packages that depend on many different versions of the same package with a python2/3 like divide being the result.

I'd rather developers acknowledge that dependencies are a hard social problem instead of pretending it's something simple that we can solve in software.

--
=====================
http://jessta.id.au

[Andrew Gerrand]

Not a central repo - just a way to make existing packages more discoverable. That something is GoPkgDoc, which now has a pretty good search engine: http://go.pkgdoc.org/

The community still needs a centralized directory of projects and other resources, rather than just packages. I have some ideas about how that might work, but no code. (I did have a half-written thing on my laptop before it was stolen. :/ Haven't worked up the energy to start it again yet.)

--
 
 

[Jan Mercl]

By accident replied only to Andrew, forwarding to list. Sorry.

---------- Forwarded message ----------
From: Jan Mercl <0xj...@gmail.com>
Date: Mon, Dec 10, 2012 at 8:11 AM
Subject: Re: [go-nuts] Re: [ANN] gonuts.io - centralized repository
for versioned Go packages

To: Andrew Gerrand <a...@golang.org>


Quoting `sgndave` at reddit
(http://www.reddit.com/r/golang/comments/14jrvf/gonutsio_centralized_repository_for_versioned_go/c7dv8ah):

----

> First, [go get] doesn't support package versioning – it always installs latest version.

> If someone wants to install previous one, he/she has to use git/hg/svn/bzr manually.
> Therefore, package authors are forced to maintain backward compatibility since first
> commit. If they want to remove some API, they should use a different repository.

This is only half-accurate. There is a contract between both the
package authors and the package users. If the upstream package API
changes, there needs to be a period of backward-compatibility in which
downstream users are expected to make the transition. It doesn't
necessarily have to work back to the first commit.

If it is completely impossible to resolve this API skew, then yes, it
becomes a different package altogether. Once you remove two legs and
nail a beak, webbed feet, and wings onto a dog, it's not a
dog_v2.01.07a. It is a duck.

In general, this is exactly the transitive dependency problem that go
tries so hard to avoid. If package A depends on B and C, and Balso
depends on C, then A and B had better agree upon which version of C
they will use. If this is impossible, they need to depend on different
targets. Calling it "versioning" simply side-steps the issue that A
and B cannot share a C.

Criticism aside, it's always good to see more options available in the
go community. Packages and APIs do change, so philosophy aside, I hope
this site makes go more useful to a broader audience of developers.

----

-j

[Alexey Palazhchenko]

Thanks for feedback!

About flat namespace first. Issues like this is why gonuts.io works in "preview" mode now.

I forgot to mention one more source of influence: Composer/Packagist for PHP. It uses names in format vendor/project.

Let's see use-cases:
1) Alice fixes a bug in Bob's nut. Bob is on vacation and can't apply fix and release a new version in 2 weeks. Meanwhile, Alice may fork repository, apply patch and use go get.
2) Alice proposes a new feature, but Bob doesn't like it. They discuss it. If Alice thinks that feature is useful only for her, she forks repository and uses go get. If Alice insist this killer-feature is must-have for everyone, she goes ahead and creates new nut.
3) Alice wants to maintain a neglected Bob's nut. She may ask him to add her as author so she can publish new versions, or she may create new nut.

So I think "canonical" packages are useful, and ability to fork code and use go get is useful too. Also I think (and hope) that flat namespace will lead to better packages, not just more packages.

But it's not a hard decision. Adding vendor names isn't hard. I would like to hear more opinions about this issue.

–-–
Alexey "AlekSi" Palazhchenko

[Dave Cheney]

Mate, I'm not going to quote urban dictionary to you, but you have to
find another name for a versioned package. Nut is not acceptable.

Dave

> --
>
>

[Alexey Palazhchenko]

About everything else.

> But the name sounds a little bit confusing.

Oops. Ruby has Gems, Python has Eggs (not quite equivalent, I know). I thought for Go to has Nuts (fruits or hardware, I not decided yet) would be okay, and a nice word play with "go nuts". Is it that bad?

> Mate, I'm not going to quote urban dictionary to you, but you have to find another name for a versioned package. Nut is not acceptable.

I wasn't aware about this, sorry. :( In my defense I may say that this part of human body is called eggs in Russian, and we don't have much jokes about Python eggs.

> The community still needs a centralized directory of projects and other resources, rather than just packages. I have some ideas about how that might work, but no code.

May you share them?

> Explicit versioning allows packages to continue to justify depending on older versions of other packages.

I plan to add support for "optimistic" version constraint like 1.*.* or 1.>=3.* and rewrite imports in code on install. It's not implemented yet.

> I'd rather developers acknowledge that dependencies are a hard social problem instead of pretending it's something simple that we can solve in software.

I 100% agree with this. It's social, and it's not simple. But when why the lucky stiff committed infosuicide and removed his code from everywhere, it was also a problem for me – I wasn't able to deploy easily. RubyGems.org did not existed at this time. If it were, life would be simpler.

–-–
Alexey "AlekSi" Palazhchenko

[Jan Mercl]

.On Mon, Dec 10, 2012 at 9:50 AM, Alexey Palazhchenko

<alexey.pa...@gmail.com> wrote:
>
> I plan to add support for "optimistic" version constraint like 1.*.* or
> 1.>=3.* and rewrite imports in code on install. It's not implemented yet.

A imports C, requires C 1.8.0
B imports C, requires C >= 2.*
main imports A, B

How is the versioning schema supposed to solve this situation?

-j

[Alexey Palazhchenko]

A imports C, requires C 1.8.0
B imports C, requires C >= 2.*
main imports A, B

How is the versioning schema supposed to solve this situation?

It will prevent you from doing this unless you really want to (something like -f flag). And in fact Go compiler will compile it: import paths are different.

–-–
Alexey "AlekSi" Palazhchenko

[Jan Mercl]

Then the versioning schema is completely broken. If C, for example,
happens to export some global state/entity, for instance like package
"log" does, than building a program with two different(ly named)
versions of C linked into the binary is a pretty clear show stopper.
This is never going to work.

-j

[Alexey Palazhchenko]

>> B imports C, requires C >= 2.*
>> main imports A, B
>>
>> How is the versioning schema supposed to solve this situation?
>
> It will prevent you from doing this unless you really want to (something
> like -f flag). And in fact Go compiler will compile it: import paths are
> different.

Then the versioning schema is completely broken. If C, for example,
happens to export some global state/entity, for instance like package
"log" does, than building a program with two different(ly named)
versions of C linked into the binary is a pretty clear show stopper.
This is never going to work.

 
If you do pass -f, it becomes your responsibility to deal with global state and other issues. Just don't.

[Andrew Gerrand]

Would you prefer gonads.io?

--

[José Carlos Nieto]

This is an interesting topic.

I actually like that Go does not have a centralized source code repository: I can fork an open source repo, make some changes and then use the same name to import the package into my application, that's just great and it's one of my favorite features. So I think that's an important concept you should keep in a centralized repo, vendor names.

I am not saying you should do this, but I'd like something like this could be done with the current Go buid and go get:

import "gonuts.io/vendor/package:1.1"

to fetch the "1.1" version/tag of a package or just:

import "gonuts.io/vendor/package"

to fetch the latest bleeding edge revision.

Of course the ":1.1" part should be done within the Go compiler but I just wanted to point it out.

I also think Go is missing is a good centralized directory, I know pkgdoc.org has a search engine and a directory but the problem is not solved yet, the directory does not seem to have a ranking for packages or at least a way to show most relevant stuff first, reading all packages descriptions is too boring, also we could use the help of a designer to fix the directory looks, it misses details but they're important details. These days I prefer reading opinions and the actual code before importing anything instead of the developer's description. I think gonuts.io can learn from how we use pkgdoc.org and how we use third party libraries.

Some centralized code repositories, like Linux distros, have package maintainers, it is not common to let the developers handle their own packages as the quality/popularity must be ensured by a trusted third party (the maintainer), what's your idea about package maintainers? what if some package has evil code in it?

> 3) Alice wants to maintain a neglected Bob's nut.

Well, Poor Bob, I didn't know Alice was that gross.

- Carlos

[tomwilde]

On Monday, December 10, 2012 10:46:02 AM UTC+1, Andrew Gerrand wrote:

Would you prefer gonads.io?

Made my day.

As to the idea; I think it's awesome but regarding versioning I would suggest keeping it simple and making it only part of the import path, not promoting it much with wildcard matching or whatever.

This way nobody gets confused (is it part of the language?) and no author gets too enthusiastic about it. I like the way Gustavo Niemeyer did it. Also, it should be optional.

- Tom

[Kevin Gillette]

On Monday, December 10, 2012 12:13:45 AM UTC-7, Jan Mercl wrote:

This is only half-accurate. There is a contract between both the
package authors and the package users. If the upstream package API
changes, there needs to be a period of backward-compatibility in which
downstream users are expected to make the transition. It doesn't
necessarily have to work back to the first commit.

I agree only to the extent that the package author has a contract with the package user due to a lack of tooling in this area. I do believe that a package author has a contract to keep previously public revisions available in their source tree, except for revisions that sensitive data managed to leak into.

In Go already, go get does not update packages unless you explicitly ask it to, and these are build-time, not run-time concerns. Additionally, many Go programmers are wary (for security reasons and otherwise) of fetching the tip/head of a repo when building in a new environment, and automated build systems for go seem to more frequently use a push-to-target deployment model, rather than a build-on-target model (which makes dependency management a relative non-issue even for massively-parallel deploys).

While I loathe manifest files and such that you see in gem's, egg's, etc., those are designed for a deployment and runtime model nearly opposite to how Go functions; if the go tool were extended with a 'freeze' command that dumped dependency versions (including the installed version of Go and the stdlib) into a .go-deps file within that package's directory, that file could later be used by other tools to conditionally fetch the exact dependency revisions (with some exceptions, like allowing newer stdlib/runtime releases). go get would also benefit from the ability to select a revision/branch via an option switch. If this tooling existed, then package users would not be so troubled when an upstream API is changed in a backwards incompatible way. A tool that dispatches to `go tool api` could even leverage rcs bisect capabilities to find and report on the latest revision supporting the (subset of the) foreign API used by an app or package.

With a complete core toolset, maintaining backwards compatibility should not be a mandate, though that's not to say it wouldn't be a good thing to maintain the old API on a separate branch if there are known to be existing users and bugs which need backported fixes.

[Kyle Lemons]

On Tue, Dec 11, 2012 at 6:41 PM, Kevin Gillette <extempor...@gmail.com> wrote:

On Monday, December 10, 2012 12:13:45 AM UTC-7, Jan Mercl wrote:

This is only half-accurate. There is a contract between both the
package authors and the package users. If the upstream package API
changes, there needs to be a period of backward-compatibility in which
downstream users are expected to make the transition. It doesn't
necessarily have to work back to the first commit.

I agree only to the extent that the package author has a contract with the package user due to a lack of tooling in this area. I do believe that a package author has a contract to keep previously public revisions available in their source tree, except for revisions that sensitive data managed to leak into.

In Go already, go get does not update packages unless you explicitly ask it to, and these are build-time, not run-time concerns. Additionally, many Go programmers are wary (for security reasons and otherwise) of fetching the tip/head of a repo when building in a new environment, and automated build systems for go seem to more frequently use a push-to-target deployment model, rather than a build-on-target model (which makes dependency management a relative non-issue even for massively-parallel deploys).

While I loathe manifest files and such that you see in gem's, egg's, etc., those are designed for a deployment and runtime model nearly opposite to how Go functions; if the go tool were extended with a 'freeze' command that dumped dependency versions (including the installed version of Go and the stdlib) into a .go-deps file within that package's directory, that file could later be used by other tools to conditionally fetch the exact dependency revisions (with some exceptions, like allowing newer stdlib/runtime releases). go get would also benefit from the ability to select a revision/branch via an option switch. If this tooling existed, then package users would not be so troubled when an upstream API is changed in a backwards incompatible way. A tool that dispatches to `go tool api` could even leverage rcs bisect capabilities to find and report on the latest revision supporting the (subset of the) foreign API used by an app or package.

You're describing something similar to what I started, rx : kylelemons.net/go/rx

The problem is what was described above.  If A depends on B v1 and C v2, and B depends on C v1, then no amount of tooling can really help you if you need a feature that's in C v2.

With a complete core toolset, maintaining backwards compatibility should not be a mandate, though that's not to say it wouldn't be a good thing to maintain the old API on a separate branch if there are known to be existing users and bugs which need backported fixes.

--
 
 

[Martin Angers]

A few thoughts about this:

I think gopkgdoc is great for package discovery. This part it handles well, in a decentralized way (don't know how many hosting providers are supported, but clearly most big ones), this is going in the right direction I believe (especially since it supports Go's experimental packages! Now my packages can be found... :).

As for versioning, node's npm has perfected this to an art form. Managing dependencies, like the A that uses B and C with a different B is trivial in node, and versions can be locked down using shrinkwrap, it's really neat. But this is in dynamic javascript-land, not in statically-linked Go-land, where it's a bit harder. The OP announcement is the first attempt that I'm aware of to tackle this problem, this is great news, but I'm not so sure about the centralized repo thing though. Couldn't this work using conventions, such as looking for a vM.m.p tag (Major, minor and patch, as prescribed by http://semver.org/) in source control? No need to publish to a separate site?

Finally, for the "moral contract" between package developers and users, the API compatiblity and all, this is why we have versions. The solution is version-based, not a fragile and implicit "contract"-based that puts a hell of a lot of pressure on the package developer ("yes, your production code will always work!"). This helps not only for breaking changes, but to explicitly announce bug fixes too (hey, here's a new patch-bumping version: bug fix).

[Kyle Lemons]

On Wed, Dec 12, 2012 at 9:25 AM, Martin Angers <martin....@gmail.com> wrote:

A few thoughts about this:

I think gopkgdoc is great for package discovery. This part it handles well, in a decentralized way (don't know how many hosting providers are supported, but clearly most big ones), this is going in the right direction I believe (especially since it supports Go's experimental packages! Now my packages can be found... :).

As for versioning, node's npm has perfected this to an art form. Managing dependencies, like the A that uses B and C with a different B is trivial in node, and versions can be locked down using shrinkwrap, it's really neat. But this is in dynamic javascript-land, not in statically-linked Go-land, where it's a bit harder.

To be clear, it's not "hard" to link Av1 -> {Bv1, Cv2}, Bv1 -> {Cv1} in Go.  It's not possible.  If you use different directories, then it's no longer the same package, so it "works," but you will have two instances of the package in memory whose values cannot be used interchangeably, not to mention the potential of having separate global state if the package employs it.

 

The OP announcement is the first attempt that I'm aware of to tackle this problem,

If you go back in the history of the mailing list, there have been many attempts to tackle this problem through a number of different mechanisms.  I, myself, have made two separate attempts which address one or more of the problems (one as a proxy for packages and one as a tool to help versioning), and neither were the first.

 

this is great news, but I'm not so sure about the centralized repo thing though. Couldn't this work using conventions, such as looking for a vM.m.p tag (Major, minor and patch, as prescribed by http://semver.org/) in source control? No need to publish to a separate site?

I don't think we should enforce versioning semantics on the entire community.  Sure, it's probably a good idea to tag versions in your repository, but (especially because of the linking problem above) I don't think it's as important a requirement as stabilizing your exported API.

 

Finally, for the "moral contract" between package developers and users, the API compatiblity and all, this is why we have versions. The solution is version-based, not a fragile and implicit "contract"-based that puts a hell of a lot of pressure on the package developer ("yes, your production code will always work!"). This helps not only for breaking changes, but to explicitly announce bug fixes too (hey, here's a new patch-bumping version: bug fix).

Again, unless you bump every package in a binary to the bug fix version, it simply won't work.  At that point, you've basically just sync'd to head (on the default branch) and don't really need to be concerned with version numbers.

 

Le dimanche 9 décembre 2012 04:23:45 UTC-5, Alexey Palazhchenko a écrit :

Good news everyone!

I'm happy to announce a preview of http://gonuts.io/ – centralized repository for versioned Go packages.

Why do I think Go ecosystem need this? There are two problem "go get" can't solve.

First, it doesn't support package versioning – it always installs latest version. If someone wants to install previous one, he/she has to use git/hg/svn/bzr manually. Therefore, package authors are forced to maintain backward compatibility since first commit. If they want to remove some API, they should use a different repository.

Second, in practice many developers are moving their code to other places (for example, from Google Code to GitHub), renaming repositories (web.go become web) or just deleting them (at the time of this writing many links in Go Projects Dashboard and GoPkgDoc are dead). Yes, it's a social problem, but we still should deal with it.

So how can we solve those problems? Big companies typically have special repositories for third-party code. Once imported there, code is never deleted. And they have a budget to fix their world of dependencies. So, "go get" probably works okay for "google/..." packages. Smaller companies and individual developers are able to bundle third-party packages with their application and take pain of updating them only when needed. But what should package developers do if they want to use other packages?..

gonuts.io for Go, similar to PyPI for Python, RubyGems.org for Ruby and NPM for Node.js should solve those problems. Published packages (called "nuts") are never deleted, and versioning schema allows to install exact version. There are plans to allow to install a version matching pattern (like 2.*.*), but still be in control (similar to RubyGems' Bundler). And nut client tool was designed to work along with Go conventions: nuts are installed into workspace specified by GOPATH environment variable, and imported as "gonuts.io/test_nut1/0.0.1".

There are few more things I need to do before official launch. First of all, I want to provide a clear path for transition of well-known packages to gonuts.io without names being squatted. So for now gonuts.io works in preview mode, and all nuts will be removed before going into real production.

So – feel free to try Go Nuts, publish your packages, install other and post your comments into this thread or gonu...@googlegroups.com discussion group. You even may contribute. ;)

Thanks.

–-–
Alexey "AlekSi" Palazhchenko

--
 
 

[Martin Angers]

To be clear, it's not "hard" to link Av1 -> {Bv1, Cv2}, Bv1 -> {Cv1} in Go.  It's not possible.  If you use different directories, then it's no longer the same package, so it "works," but you will have two instances of the package in memory whose values cannot be used interchangeably, not to mention the potential of having separate global state if the package employs it.

Of course, you're right. I agree, but I also disagree :) We don't talk from the same perspective. The reasons you mention to say it's not possible, I (implicitly) mention as part of the solution. A different directory is a different package from Go's perspective, of course, otherwise it would not work. But from a user's perspective, if we end up with a good enough tool, this is exactly what he wants. No, they won't share internal state, that's how it is in node too, or in .NET if you load different versions of the same assembly, there's a reason A and B don't use the same version of C, it's because they have separate needs. I believe this would rarely be a problem, but I may be terribly wrong, I have more experience in dynamically-linked languages.

What I see as a real problem (not that I have thought this through, though!) is that in the A-B-C case, assuming B is a general purpose package installed from a 3rd party (and not developed specifically for application A), it is probably not written to import a specific version of C. So the tool would need to rewrite the import (a-la Go Vet?). As I said, a bit harder, but impossible? It depends on what solution we are willing to accept.

I don't think we should enforce versioning semantics on the entire community.  Sure, it's probably a good idea to tag versions in your repository, but (especially because of the linking problem above) I don't think it's as important a requirement as stabilizing your exported API.

No need to enforce it, if there is no version, then fallback on Go1 tag or master. Obviously, these packages would be used on the presumption that they will maintain compatibility. As for stabilizing your exported API, this is only a small subset of the use of versions. If I deployed an app a month ago, and I need to deploy it on another server a month later, what guarantees do I have that your non-versioned, API-compatible package that I use is still stable for my use? With versions, I grab the exact same one. Without versions, leap of faith (or more realistically, try and find the commit hash to fetch this specific commit by hand).

 

[Kevin Gillette]

On Wednesday, December 12, 2012 11:07:58 AM UTC-7, Kyle Lemons wrote:

To be clear, it's not "hard" to link Av1 -> {Bv1, Cv2}, Bv1 -> {Cv1} in Go.  It's not possible.

I'm not convinced this is a major problem with Go. If it's that hard to update B to use Cv2, then either B or C was probably written quite poorly. The real issue comes with whether Cv2 has a disjoint feature-set with Cv1 (and where features in the symmetric difference are needed by A and B respectively. In this case, I'd say the author of C is either is taking a long time on v2 (and didn't have the good sense to delay releasing it until it was at least as capable as v1), or the author of A didn't have the good sense to notice this kind of issue.

In any case, I don't think this is a case that needs tooling. I was suggesting a dependency list, not for the each library, but for the app as a whole (if you can't build it with `go build`, then deps don't matter, and if the problem you describe above happens to occur, you can't `go build` anyway).

[Kyle Lemons]

On Wed, Dec 12, 2012 at 5:31 PM, Kevin Gillette <extempor...@gmail.com> wrote:

On Wednesday, December 12, 2012 11:07:58 AM UTC-7, Kyle Lemons wrote:

To be clear, it's not "hard" to link Av1 -> {Bv1, Cv2}, Bv1 -> {Cv1} in Go.  It's not possible.

I'm not convinced this is a major problem with Go.

Hmm. I didn't mean to imply that I thought it was a problem with Go itself, or even a language problem.  It seems like it exists in lots of languages, and I think it's a social problem that occurs with the changing of external APIs, which often have multiplicative effect on package consumers (or even design of the API without consideration for future updates).

 

If it's that hard to update B to use Cv2, then either B or C was probably written quite poorly.

That is quite true.  In fact, in the cases in which a package author does need to change their API, it would be really swell of them to provide a gofix module for it, after a suitable period of supporting the backward-compatible interface as well.

 

The real issue comes with whether Cv2 has a disjoint feature-set with Cv1 (and where features in the symmetric difference are needed by A and B respectively. In this case, I'd say the author of C is either is taking a long time on v2 (and didn't have the good sense to delay releasing it until it was at least as capable as v1), or the author of A didn't have the good sense to notice this kind of issue.

In any case, I don't think this is a case that needs tooling.

That's one of the reasons I stopped working on rx, actually.  It was conceptualized as a way to identify and track inter-repository dependencies in such a way that it could then pull the updates from a repository and check that nothing depending on it broke, and if it did, play games with (tagged) versions in between to see if one of them still works.  I'll also mention that I planned on having it run tests, as API changes aren't the only thing that could change, and a package author might not even realize that he made a semantic change or that he depended on an undocumented feature/bug in another package.

 

I was suggesting a dependency list, not for the each library, but for the app as a whole (if you can't build it with `go build`, then deps don't matter, and if the problem you describe above happens to occur, you can't `go build` anyway).

--
 
 

[Kevin Gillette]

On Wednesday, December 12, 2012 9:37:01 PM UTC-7, Kyle Lemons wrote:

If it's that hard to update B to use Cv2, then either B or C was probably written quite poorly.

That is quite true.  In fact, in the cases in which a package author does need to change their API, it would be really swell of them to provide a gofix module for it, after a suitable period of supporting the backward-compatible interface as well.

Intriguing. A quick search, however, seems to indicate that there's no "userland" support in gofix yet. I suppose limited changes could be bundled in a file of gofmt invocations that could probably be made to be sh/bat/rc compat all at once.

 

That's one of the reasons I stopped working on rx, actually.  It was conceptualized as a way to identify and track inter-repository dependencies in such a way that it could then pull the updates from a repository and check that nothing depending on it broke, and if it did, play games with (tagged) versions in between to see if one of them still works.  I'll also mention that I planned on having it run tests, as API changes aren't the only thing that could change, and a package author might not even realize that he made a semantic change or that he depended on an undocumented feature/bug in another package.

You've made me realize that I've now danced on both sides of the argument. Certainly with this awareness, I'm now leaning towards the "leverage go's hackability" as an inherent "tool". In something like Java, where you see a lot of copypasta, both library authors and library users may be unwilling to consider or adapt to incompatible changes without automated or batched tooling. In go, those habits may linger, yet I regularly find it's faster to figure out, fix, scrap, and then rewrite large portions of someone else's library than it is to wait for a response to an bug report, for better or worse. Sure, it promotes fragmentation, but only if the fragmenters don't post pull requests or the authors don't consider them.  Projects with merit but without stewardship are asking to get fragmented anyway.

[Dan Kortschak]

Some time ago I moved my project from using gb to the go tool and wrote
a gofix module for that purpose. This involved inserting a module into
the gofix in $GOROOT. I suspect this is the approach Kyle is suggesting,
though a userland plugin approach would be nice.

Dan

[Kyle Lemons]

On Wed, Dec 12, 2012 at 11:55 PM, Kevin Gillette <extempor...@gmail.com> wrote:

On Wednesday, December 12, 2012 9:37:01 PM UTC-7, Kyle Lemons wrote:

If it's that hard to update B to use Cv2, then either B or C was probably written quite poorly.

That is quite true.  In fact, in the cases in which a package author does need to change their API, it would be really swell of them to provide a gofix module for it, after a suitable period of supporting the backward-compatible interface as well.

Intriguing. A quick search, however, seems to indicate that there's no "userland" support in gofix yet. I suppose limited changes could be bundled in a file of gofmt invocations that could probably be made to be sh/bat/rc compat all at once.

No, gofix itself doesn't support userland (yet? maybe this would be worth trying to design), but the few times I've seen it done they basically copied the gofix source and inserted their own modules and made it a go-gettable binary in their repo.

 

That's one of the reasons I stopped working on rx, actually.  It was conceptualized as a way to identify and track inter-repository dependencies in such a way that it could then pull the updates from a repository and check that nothing depending on it broke, and if it did, play games with (tagged) versions in between to see if one of them still works.  I'll also mention that I planned on having it run tests, as API changes aren't the only thing that could change, and a package author might not even realize that he made a semantic change or that he depended on an undocumented feature/bug in another package.

You've made me realize that I've now danced on both sides of the argument. Certainly with this awareness, I'm now leaning towards the "leverage go's hackability" as an inherent "tool". In something like Java, where you see a lot of copypasta, both library authors and library users may be unwilling to consider or adapt to incompatible changes without automated or batched tooling. In go, those habits may linger, yet I regularly find it's faster to figure out, fix, scrap, and then rewrite large portions of someone else's library than it is to wait for a response to an bug report, for better or worse. Sure, it promotes fragmentation, but only if the fragmenters don't post pull requests or the authors don't consider them.  Projects with merit but without stewardship are asking to get fragmented anyway.

--
 
 

[Guillermo Estrada]

Kudos on the announcement!! If this becomes even half of what RubyGems.org is, I will be thrilled, i want to start using it now, any documentation on how to make/package some nuts? Some people might argue that the best approach is the one being used, because it gives you [bla, bla, bla], but really, the central repository approach with a package management tool is just too good to pass out on any language (npm, pypi, luarocks, rubygems, etc..) and I'm glad someone noticed it and made something about it. I would love to help testing and such, to make release come faster. BTW, consider submitting package information like repository, wiki, homepage, author, bug tracker, group/mailing list, etc... on each nut page (kinda like Rubygems.org does), of all the package managers it has the best design as a service.

It's been a while since I was looking for this, way before the go tool existed. There should be a "like" button on Google Groups, just so I can click on it right now.

[Dave Cheney]

I am curious. In this thread there is a common reference to the idea of a singular central repository for packages being highly desirable.

Personally I find the idea unattractive, a SPoF, overly authoritarian and potentially a political football.

Could someone explain what benefits they see in a central singular namespace for packages?

Dave

--
 
 

[Martin Angers]

Especially since I think it could be done in a decentralized way. Get the packages where they are, use a tag convention or some other means to manage versions, forks are easily gettable (they simply have a different path), so namespacing is automatic, etc. *Maybe* register your package in a central repo just so it knows you exist, but the rest is decentralized (pkgdoc could also be used for discovery, as it becomes more and more the de facto standard). So I'm curious too.

[Peter Kleiweg]

Op maandag 10 december 2012 09:18:56 UTC+1 schreef Dave Cheney het volgende:

Mate, I'm not going to quote urban dictionary to you, but you have to
find another name for a versioned package. Nut is not acceptable. 

You do know this is golang-nuts, right?

[bryanturley]

Only because golang-testicles was already taken?

Though I would agree for different reasons.  No matter the definite of nut I can't see it in a metaphorical relation with source code.

That said, I have seen some
( •_•)>⌐■-■

(⌐■_■)
hairy code in some of these packages...