What lead to the versionning debate?

[Chad]

Just wondering, Because as of now, in Go, the Pkg path is the version and for reproducible builds, your source/version control is probably more suited to handle the task.

Or are there drawbacks to that?

[Dave Cheney]

Hello,

You're pretty late to the discussion. Here's some background reading.

https://getgb.io/rationale/

Dave

[Chad]

Hi Dave,

Thank you for the link.

From the discussions I have seen so far, it seems to me that there is a conflation between the notions of version and revision.

The go tooling AND language remains unaware of revisions indeed, which would be the role of the source control system (often git nowadays but who knows in the future). It introduces decoupling here which I guess is welcomed for forward compatibility with source control systems.

Is that what people find problematic? (no automatic way to retrieve a package at a given revision?)

I understand vendoring as a way to circumvent the reliance on the social contract that fetching HEAD will never be API breaking.

Is that right?

[Dave Cheney]

Yes, exactly.

[Chad]

So if we decided that vendoring always used HEAD, that would ideally force people to have stable APIs (plus, why vendor an unstable/ in-flight API anyway)

Couldn't it enable a change of behaviour for go get when a vendor directory is detected and solve some of the issues related to type equality (by having a single pkg path for a vendored package) ?

On Friday, July 29, 2016 at 12:14:02 AM UTC+2, Dave Cheney wrote:

Yes, exactly.

[Chad]

Actually, nevermind. I was thinking about libraries using vendoring. But that's probably a misuse.

[Dave Cheney]

Who is we ? You and me ?

[Tyler Compton]

It's funny that you mention libraries using vendoring because that's been a big deal surrounding this topic.

https://groups.google.com/d/msg/golang-dev/4FfTBfN2YaI/WCza__MaCgAJ

[Chad]

My reasoning was flawed. (also, I generally mean "one" when I use "we" :)

I think you're right with the project based approach in gb (for people interested in reproducible builds).

I think that the pkg path issue (that leads to type equality issue) may be simply solved by having the vendor directory at the root of the project directory. (so no vendor directory per library)

Any conflict caused by the evolution of a vendored package API would be stemming from the API surface having been augmented. That should trigger an update of the vendored library. 

(If the author did not keep with the backward compatibility requirement, then that would be a breach of versioning as Go defines it.)

Maybe I am overlooking something still.

[Dave Cheney]

Yes, to use the vendor/ feature project authors need to flatten all their dependencies into a single, top level, vendor/ folder. This is currently difficult as there is no common way to look at two copies of the same source code and decode if there are the same, and if not, which should take priority.

[Chad]

Oh I see now. I guess we need something inbetween go get and the different vcs to register and timestamp a package each time it is declared as having been updated. (would still be vcs agnostic though, it's just to timestamp the package files)

Would make releasing a package a bit more of a manual process but it could be a good thing.

The tooling should be able then to decide up on the latest vendored package to use.

That would also decouple the import paths from "github" as is currently often the case.

[Chad]

To be more precise, I am not thinking about a package manager but rather more of a kind of package registration interface. A bit like godoc. But working by submissions of vcs hosts links (thus allowing mirror links).

Backward compatibility requirements are making things simple already: the latest "revision" should have the priority.

[Dave Cheney]

How about a tag? which developers should be doing as part of any mature release process.

[Gregory Golberg]

What is wrong with using git submodules inside the vendor directory, submodules pointing to the tag/revision of your choice?

[Matt Harden]

I like submodules, but they do only work when you're using git and only vendoring projects that also use git.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Gregory Golberg]

But it's already a given that Go is fairly opinionated in favor of github, no?

[Chad]

On Saturday, July 30, 2016 at 1:43:36 AM UTC+2, Dave Cheney wrote:

How about a tag? which developers should be doing as part of any mature release process.

Sure, a tag may be possible. I thought timestamp because it is easily generated by a machine as well as parsable, with a well defined semantic and ever-increasing. Anything that share these features would do I guess.

On Saturday, July 30, 2016 at 1:44:10 AM UTC+2, Gregory Golberg wrote:

What is wrong with using git submodules inside the vendor directory, submodules pointing to the tag/revision of your choice?

I think it's the fact that it locks people into a single version control system. The tooling should ideally remain agnostic for forward-compatibility.

[Matt Harden]

The go tool works with other VCSes, such as mercurial, bitbucket and subversion.

--

[Michael Jones]

Go is strongly opinionated about strategies (reduce errors, simplify structure, rapid builds, memory safety, etc.) but is agnostic about most everything else. Even if the Go team uses CPU X, OS Y, VCS Z, that’s just an example from a multiplicity of present and future choices. The design of Go needs to span the present and the future—whatever comes after intel/arm/power, linux/macOS/windows, sccs/rcs/git, vi/emacs/sublime, and so on in every dimension. It is hard to do at all, much less broadly and elegantly.

 

--

Michael T. Jones

[Gregory Golberg]

That's not the case, though, 

"First, the import path is derived in an known way from the URL of the source code. For Bitbucket, GitHub, Google Code, and Launchpad, the root directory of the repository is identified by the repository's main URL, without the http:// prefix."

https://golang.org/doc/articles/go_command.html

[Dave Cheney]

That documention is incorrect. The language spec assigns no meaning to the import path.

The complier assigns the meaning that a compiled package must exists in a subdirectory with a name that matches the import part, plus .a, in a path provided by the -I flag.

The go tool goes further to interpret the contents of the import path as a URL where it will either find a vcs repo or a piece of metadata pointing it to a repo.

[Florin Pățan]

Go is not already opinionated in favor of Github. GitHub happens to be a popular source code hosting website right now. There's nothing stopping anyone hosting sources anywhere else as long as the repository is understood by the go get tool, in cases that's a requirement, else can be something different as well.

[Sam Vilain]

On 7/29/16 5:51 PM, Matt Harden wrote:
> I like submodules, but they do only work when you're using git and
> only vendoring projects that also use git.

Or vendoring any dependency which can be mirrored to git, which is
essentially all of them.

Sam

[Lucio]

On Saturday, 30 July 2016 02:52:25 UTC+2, Matt Harden wrote:

I like submodules, but they do only work when you're using git and only vendoring projects that also use git.

It seems to me that the solution is staring us in the face. I know nothing about git submodules, but they have been raised in this forum before and the only criticism has been that no other VCS has them. If they accomplish what Go developers need, then Go must model any solution on them.

It surely does not make sense to overlook them because no other VCS has them: in favour of what? No other VCS has anything better or even as good. So, within reasonable limits, (a) Go needs to adopt Git submodules and (b) Go must provide analogous tooling for alternative VCSs.

Inventing something totally orthogonal to serve the same purpose would be unreasonable.

I appreciate that Git submodules are not the final answer, but I do presume that they ought to be the foundations for any discussion.

Lucio.

[atd...@gmail.com]

That's somewhat the intent in enabling the timestamping of packages especially wrt. library-vendored package.

Flattening dependencies will still be needed but it would be simply a matter of switching a package to the latest package release that a package may have vendored.

But again, I would not build a solution by strictly imitating vcs or other package managers from a methodology point of view. They do not handle packages the same so the solution has to be different.

I firmly believe that the current assumptions held by Go tooling can help us find a quite simple(r) solution (than already exists for other languages).

[Dave Cheney]

That's somewhat the intent in enabling the timestamping of packages especially wrt. library-vendored package.

Flattening dependencies will still be needed but it would be simply a matter of switching a package to the latest package release that a package may have vendored.

To do this you need to have some _machine consumable_ notion of what is the "latest release" of a package. This currently doesn't exist for Go projects. 

[atd...@gmail.com]

Indeed, that's where we are coming short. The Go ecosystem does not own its package releasing process. (with a release identification scheme that may different from semver since we do not need to worry about major versions for reasons of different release semantics)

A `go release` command that would prepare a package for release could be the addition required. 

It will most likely  require to have a centralization of these package releases. Declined under a solution that can be internal for companies that need a tighter control over their dependencies OR fully web facing for gophers in the large).

I say package releases, but the repository can be anywhere people want, I don't think Google has to deal with the hosting of repos.

[atd...@gmail.com]

Having an interface between go get and the location of the repositories should really allow people to move packages around at will provided the registration of a package provides a canonical import path. (and a facility for the declaration of mirrors)

Now, to accomodate with vcs and demands for reproducible builds (which require segregation of the dependencies per project), perhaps the go tool could allow switching between multiple $GOPATH. (just to avoid different project builds conflicting at the vcs level).

Also binaries may benefit from these changes if they record the source release point they were compiled from.

Go gets it mostly right because the canonical package path expect backward compatibility. We shouldn't need SAT solvers to resolve dependencies normally. :)