Golang should have a center packages index hosting like npm, rust crates

[zixu mo]

Golang should have a center packages index hosting like npm, rust crates.

For Rust : 

https://crates.io/

For JS:

https://www.npmjs.com/

For PHP

https://packagist.org/

For Ruby

https://rubygems.org/

[zixu mo]

Even Swift

https://swiftmodules.com/

[Ian Lance Taylor]

On Wed, Oct 19, 2016 at 9:05 PM, zixu mo <hufen...@gmail.com> wrote:
> Golang should have a center packages index hosting like npm, rust crates.

Go has https://godoc.org/ .

Ian

[T L]

one reason I like Golang is there is no npm, maven etc for Golang.
:)

[Dave Cheney]

T L, I often hear this comment when a central repo for Go is proposed/suggested/requested. Are you able to give details of the things you do not like about Maven/npm/rubygems central repos. The most specific the better please.

[Dave Cheney]

For some background, I'm trying to flesh out the concerns of the various personas who would interact with a central repository.

https://docs.google.com/document/d/10jbJLkmQgpi1CxHKuO75Av1tKYgtOBSjzX5Z9hnG0vI/edit?usp=sharing

[T L]

On Thursday, October 20, 2016 at 4:43:08 PM UTC+8, Dave Cheney wrote:

T L, I often hear this comment when a central repo for Go is proposed/suggested/requested. Are you able to give details of the things you do not like about Maven/npm/rubygems central repos. The most specific the better please.

central repos are your fingerprint collectors. This is why I also don't use rss readers.
When you update your dependencies or your feeds, you are identified without logging in.

Another reason is central repos weaken, if not break, the connections between the dependency projects and you.
 

[Sri G]

Pros: 

+ Getting stats on popular packages and code. Keeping anonymity should be like apt on debian/ubuntu by requesting permission for anonymous stats reporting.

+ Showing Go's popularity and increasing community and adoption

Cons: 

- Having a central point of failure for pulling repos.. Look at the massive DDoS on Dyn right now taking out quite a few major web platforms (Twitter, Spotify, Reddit, New York Times, WIRED.com, etc)

- Central repos are often terribly slow, creating local mirrors is not easy

    - Not suitable for automated builds / testing without reliable mirrors

Adding anonymous reporting to go get and a website to show results would solve this nicely without creating a bottleneck.

[Ronny Bangsund]

On Thursday, October 20, 2016 at 5:45:03 AM UTC-0:8, zixu mo wrote:

Golang should have a center packages index hosting like npm, rust crates.

It seems some posters didn't notice the distinction you made there ;)

The OP asked about an *index*, not a central repository. It would basically be (a variant of) go get with an additional index of third-party package URLs. Take a look at http://crates.io for the simplest example. Everything there links to the authors' repositories (which these days basically means everybody's fave egg baskets, GitHub or BitBucket).

(Enter the Dreamlands for a bit)

My ideal system would have a few typical package manager features, like versions with individual dependencies for each, categories, priority levels for versions and some way to switch between versions/branches easily. Make mirroring and hosting the index easy. Dependencies could also be figured out via package contents, but sometimes a newer version also relies on specific versions of other packages - this could fail horribly in a downgrade when APIs change.

With all of the above covered, now you'd need to get people to use it, automate as much as possible and avoid name clashes. Package names in the username-package style might work, but the command line tool then needs to be able to suggest package names ("No package foo. Did you mean foo-bar or foo-bar-baz?").

Of course, a pure index solution wouldn't automatically be tracking package popularity in any way. It just serves up URLs for go get, and knows nothing of what people actually download beyond the index (unless it has optional stats tracking sent back to the index site).

The case of packages relying on cgo is also a problem. At that point you'd require some advanced pre-/post-install scripts too. Or be lazy and include a document with compilation instructions for dependencies.

I can see many problems with both an index and a real package repo, but I can see advantages too. It would be relatively simple (for variable values of simple) to get tools and a backend ready if focused on pure Go. It would just take a little effort from package developers to be as smooth as possible.

[Axel Wagner]

On Fri, Oct 21, 2016 at 11:08 PM, Ronny Bangsund <ronny.b...@gmail.com> wrote:

On Thursday, October 20, 2016 at 5:45:03 AM UTC-0:8, zixu mo wrote:

Golang should have a center packages index hosting like npm, rust crates.

It seems some posters didn't notice the distinction you made there ;)

The OP asked about an *index*, not a central repository.

That already exists, in the form of godoc.org. As pointed out by Ian.

My issue with the concept of a central repository is, that the issues it solves are far more elegantly handled by providing mirrors; i.e. doing something similar to godoc.org, but fetching and mirroring full git repos indefinitely (and support for using them transparently in go-get). This mirror infrastructure can be run decentralized and as it mirrors transparently also uses the existing decentralized namespacing, so it alleviates most issues of a centralized repo. At the same time, it provides uptime and protection against unpublishing, which is the greatest benefit of a centralized repository.

[Keith Rarick]

On Thu, Oct 20, 2016 at 1:43 AM Dave Cheney <da...@cheney.net> wrote:

T L, I often hear this comment when a central repo for Go is proposed/suggested/requested. Are you able to give details of the things you do not like about Maven/npm/rubygems central repos. The most specific the better please.

I think this isn't really a fair question when posed in such a general way. If you're looking for specific feedback, it would be really helpful if you can provide a concrete proposal for folks to critique.

Treating the existing Go packaging ecosystem as a specific example, I will mention one thing that I like about it:

When it comes time for me to make a package available to others to use, it's nice to know it'll stay available — online — but high availability is difficult.

Many packages aren't important enough for me (or my company) to spend the effort required to do a more thorough job at this than github, so we can just throw them up in a github repo and call it a day. But when github becomes unavailable due to a DDoS on the DNS, those packages will also be unavailable.

For very important packages, we have the option of hosting them ourselves and assuming full responsibility.

https://www.whoownsmyavailability.com/

We can set up redundant DNS servers, edge caches, and whatever else is warranted by the situation.

Because the 'go' tool doesn't require packages to be hosted in any one particular place,  we can take as much responsibility or as little effort as we need, depending on the package. This tradeoff is available in several important systems on the internet, such as HTTP, and I'm glad Go has it too.

[Nate Finch]

I, for one, really like that there's no central repository for go packages.  I don't like the idea of a central authority telling me how and what I'm allowed to publish.  I like that there's no one between a package I authored and a project using it except git clone (and, evidently, some DNS caching).

I honestly don't see the point of a central package authority in this day and age.  The central authority for Go is the entire internet.  Isn't that awesome?

Godoc.org is pretty good for searching for packages to use.  It's not perfect, of course... it won't find things that it hasn't been told about, but that's on the project author to worry about.

I'd like to see godoc add some quality metrics and maybe some popularity metrics, so it's easier to figure out what's a good package for X and what other people are using.  Obviously, these are very subjective measurements, but having them consolidated in one place could shorten search time.

[Axel Wagner]

On Mon, Oct 24, 2016 at 5:46 PM, Nate Finch <nate....@gmail.com> wrote:

Godoc.org is pretty good for searching for packages to use.  It's not perfect, of course... it won't find things that it hasn't been told about, but that's on the project author to worry about.

Notably, this is no different from a centralized repository. Stuff that is on github and was never uploaded to npm won't be discoverable via npm.

It's rather better with godoc.org, because everyone can tell it about a package, not only the author.

 

I'd like to see godoc add some quality metrics and maybe some popularity metrics, so it's easier to figure out what's a good package for X and what other people are using.

It already offers some of this in the form of "this package is imported by X packages".

But yeah, it could certainly do better.

[Matt Harden]

There is also go-search.org.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Haddock]

Am Samstag, 22. Oktober 2016 01:57:59 UTC+2 schrieb Axel Wagner:

On Fri, Oct 21, 2016 at 11:08 PM, Ronny Bangsund <ronny.b...@gmail.com> wrote:

On Thursday, October 20, 2016 at 5:45:03 AM UTC-0:8, zixu mo wrote:

Golang should have a center packages index hosting like npm, rust crates.

It seems some posters didn't notice the distinction you made there ;)

The OP asked about an *index*, not a central repository.

That already exists, in the form of godoc.org. As pointed out by Ian.

Some mechanism is needed to be able to specify a version number. Downloading code from Github that has changed meanwhile and out of pure luck things all compile and run the same is not viable. Some interface that allows for providing a version number would be useful. Then people out there can work on providing an implementation. Time will tell which one gets most widespread acceptance.

[Axel Wagner]

I disagree, but this question is also completely orthogonal to the one of whether there's a centralized repository or not.

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[Dave Cheney]

I agree strongly and have been advocating for this all year.

[Ain]

kolmapäev, 26. oktoober 2016 12:17.50 UTC+3 kirjutas Dave Cheney:

I agree strongly and have been advocating for this all year.

Dave, there is no context on your post so it is not clear what do you agree with... :)

ain

[Dave Cheney]

I am agreeing with Haddocks post, sorry if the groups interface did not respect the thread of discussion.

[Haddock]

Am Mittwoch, 26. Oktober 2016 11:50:08 UTC+2 schrieb Dave Cheney:

I am agreeing with Haddocks post, sorry if the groups interface did not respect the thread of discussion.

What about a versioning experiment as with the vendor experiment?

[Dave Cheney]

I'd very much like to see that.

https://github.com/golang/go/issues/12302

http://dave.cheney.net/2016/06/24/gophers-please-tag-your-releases

[Edward Muller]

Please start tagging your releases with semver tags.

Start now, there is no reason to wait.

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[mbohu...@gmail.com]

I was just wondering whether a simple list of available versions at the top of a package's godoc.org page wouldn't somehow force package authors to start tagging. There could be some kind of message if there are no available versions.

There would obviously have to be some benefit for those who tag, or some restriction for those who don't, in order to change the current situation.

Michal

Dne středa 26. října 2016 16:05:21 UTC+1 freeformz napsal(a):

[Nate Finch]

Listing versions on godoc is an awesome idea.  Peer pressure goes a long way toward changing individual minds and eventually a community.  Something as simple as sorting projects with tagged versions higher and displaying them prominently on the project's godoc page (with the ability to look at godoc for each version) would go a long way toward encouraging people to tag their releases.  I know it would get me off my butt to tag my projects.

[Dave Cheney]

Indeed. If only there was some standard we could use

https://github.com/golang/proposal/blob/master/design/12302-release-proposal.md#tag-format

[Edward Muller]

Yes, use that proposal's format.

Re: godoc. Great idea. if no one else is interested in doing the work I'll be happy to take a look into it. I've been looking for an excuse to work with gddo

[Axel Wagner]

So, the idea is to penalize people who a) have a different opinion about the usefulness of versioning or b) keep their APIs stable and don't see a need to version something that never changes?

Needless to say, I consider that a bad idea (given, that I have strongly differing opinions about the usefulness of versioning). Why can't the idea not succeed based on it's own merit (or lack thereof)?

But whatever. Set minds won't be swayed by arguments, I guess.

I'll also point out (again) that this thread, originally, was about a central repository. Not about versioning.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[Michal Bohuslávek]

It's not much of a penalisation IMO. It doesn't have to be a large message in red saying: "The author of this package doesn't use semantic versioning. Be careful!". I mean the list of versions can only appear if there are some versions. That way it wouldn't penalise a), although I can't come up with a good reason not to tag versions apart from b).

When it comes to b), there could be some rule for that case. Such authors could for example put their stable API in a branch called "stable" or something like that.

Dne čtvrtek 27. října 2016 1:39:47 UTC+1 Axel Wagner napsal(a):

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[atd...@gmail.com]

On Thursday, October 27, 2016 at 4:41:42 PM UTC+2, Michal Bohuslávek wrote:

It's not much of a penalisation IMO. It doesn't have to be a large message in red saying: "The author of this package doesn't use semantic versioning. Be careful!". I mean the list of versions can only appear if there are some versions. That way it wouldn't penalise a), although I can't come up with a good reason not to tag versions apart from b).

When it comes to b), there could be some rule for that case. Such authors could for example put their stable API in a branch called "stable" or something like that.

That wouldn't work with the way the language is structured (imports).

Just having a version tag is perhaps necessary, but, alas, not sufficient to deal with the entirety of dependency discovery/management.

We have some primitives (canonical imports etc...). Solving availability by redundancy(since part of the issue is about distributed systems) may also tie into import rewritting, or import aliasing.. which could be done a few different ways.

It's not a simple problem.