version-spec proposal

[Dave Cheney]

Hello,

For Go 1.6 I intend to propose [1] a specification for assigning
versions to Go projects. Before doing so I'd like to road test the
idea on this group.

* Background

A Go project is defined as some Go source code in a vcs repository.
This could be a library consisting of one or more packages, or an
application consisting of one or more packages that produces one or
more binaries. The definition of project is wider than gb's definition
[2] of a project, please don't be distracted by this.

Currently Go code in one project can depend on code from another
project and we have no way to describe the _temporal_ relationship
between those two projects, save one, the revision identifier
generated by the vcs system that holds the dependant project's source
code. I believe that this vcs identifier (hereafter, a hash) is
insufficient to answer all the questions Go programmers want to ask
about their dependant relationships.

For example, hashes are not sortable, so while you can tell if you
have a different revision of a project, you cannot tell if it is older
or newer than the expected revision.

What is missing from Go projects is a mechanism to turn arbitrary dvcs
hashes into version numbers.

* Proposal

I propose that Go adopt a single specification for describing Go
projects; a version identifier. This version identifier should include

a. The name of the project, probably the name of the topmost package
in the project's source tree.
b. A numbering scheme that lets a human and a computer extract the
information necessary to answer questions like "is there a newer
version of this project?", "Have there been any bug fixes to version
1.4?", etc

As an example (not the final proposal)

webdav-1.0.0

* Where would this version identifier be used ?

Out of scope of this proposal, but clearly in scope for future work. I
imagine the version would be used in cases like

a. Releasing a version of a Go project by tagging it in a vcs
repository; this could be done manually, or automatically with some as
yet to be written tool.
b. Identifying a particular version of a project as a dependency of
another project. This could be in one of the many dependency
management tools in the Go ecosystem at the moment.
c. A central project repositories.

* Projects, not packages

This proposal is not about versioning packages. Go packages do not
have versions in the same way that Go packages do not have dvcs
hashes, however they may inherit a version identifier in the same way
they inherit a dvcs hash by being tracked by that vcs.

* Open questions

a. Should semver be advocated ? Or should a new scheme be invented ?
b. How can alpha, beta, release candidates, etc integrated into this
scheme ? Semver appears mute on this subject.
c. Should additional information be permitted as a suffix, ie most
debian packages carry something like quux-1.3~1ubuntu1 to describe
that this is the first build of a deb package from the quux-1.3
release.

Thanks for your time

Dave

1. https://github.com/golang/proposal
2. http://getgb.io/docs/project/

[Nika Jones]

I'm on board with you about turning version control hashes into something meaningful like a version number. But I don't see where you did that?

How did you get the: webdav-1.0.0 ?

Was it derived from somewhere? Or is this something that you add to the top level package as a comment? If it's not derived are you expected to manually increment the number? If you're manually incrementing the number then how does this differ from Git, Mercurial or other DVCS tags?

Or are you asking that we start thinking at the project level as opposed to the package level for how things are versioned? So we should start thinking of things at a wider scope?

I've read the post a couple of ways and I'm not sure which direction you're trying to take.

Nika

[Dave Cheney]

Hi Nika,

On Fri, Aug 14, 2015 at 2:04 PM, Nika Jones <art...@gmail.com> wrote:
> I'm on board with you about turning version control hashes into something
> meaningful like a version number. But I don't see where you did that?
>
> How did you get the: webdav-1.0.0 ?

webdav is the name of the package at the top of the hypothetical
project, 1.0.0 is just a version number I made up.

> Was it derived from somewhere?

How project owners decide on version numbers is certainly for
discussion, should this proposal recommend semver, or some other
numbering scheme ?

Or is this something that you add to the top
> level package as a comment?

The version number is external to the project's source code, in the
same way that a dvcs revision is external to the projects source code.

> If it's not derived are you expected to manually
> increment the number?

Yes, but this proposal lays the groundwork for tools that automate
this and turn it from tagging a repository, into a fully grown release
process.

> If you're manually incrementing the number then how
> does this differ from Git, Mercurial or other DVCS tags?

This is a proposal to specify the contents of that tag.

> Or are you asking that we start thinking at the project level as opposed to
> the package level for how things are versioned? So we should start thinking
> of things at a wider scope?

I think of this as versioning an import path prefix, because that is
what you go get, ie, right now you can do

go get github.com/foo/bar

and what everyone wants is

go get github.com/foo/bar@[1.0.0,2.0.0)

This is a step towards that (I hope)

> --
> You received this message because you are subscribed to the Google Groups
> "Go Package Management" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-package-manag...@googlegroups.com.
> To post to this group, send email to go-package...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[N. Jones]

Okay, that clears a lot of things up. But I still have some more questions.

So as of now you can do this:

     go get gopkg.in/foo/bar.v1.3

Which gets you pretty far along with versioning, I saw your previous email about not wanting to have

a vanity domain to do your import path magic for you. However gopkg.in is a well maintained domain that

could for all intents compete with github.com as the name you place on your import path, why not

promote a few places that one could rely on for that. (ie. github.com vs bitbucket.com vs gitlab.com)?

And finally for the proposal where would the version number be maintained? Is it at the file level? The VCS level?

protocol level :) ?  I'm not sure where you put the version.

Were you thinking:

     go get github.com/foo/bar@[webdev-1.0.0]

or something like that as the ultimate endgame?

Nika

[Dave Cheney]

On Fri, Aug 14, 2015 at 2:34 PM, N. Jones <art...@gmail.com> wrote:
> Okay, that clears a lot of things up. But I still have some more questions.
>
> So as of now you can do this:
>
> go get gopkg.in/foo/bar.v1.3
>
> Which gets you pretty far along with versioning, I saw your previous email
> about not wanting to have
> a vanity domain to do your import path magic for you. However gopkg.in is a
> well maintained domain that
> could for all intents compete with github.com as the name you place on your
> import path, why not
> promote a few places that one could rely on for that. (ie. github.com vs
> bitbucket.com vs gitlab.com)?

The author of gopkg.in discourages this [1], I agree with him.

> And finally for the proposal where would the version number be maintained?
> Is it at the file level? The VCS level?

If you wanted to use a version identifier on your git repo, it would be a tag.

> protocol level :) ? I'm not sure where you put the version.
>
> Were you thinking:
>
> go get github.com/foo/bar@[webdev-1.0.0]

This is not a proposal to change go get, I'm sorry if I gave that idea
in my previous message. As a replacement consider some dependency.txt
style file that says

github.com/foo/bar [1.0.0,2.0.0)

To be very very clear. This is only about describing the format of an
identifier that _could_ be used in a git tag. It could also be used as
the name of a zip file with the contents of that package, or it could
be used in some dependency.txt file, or it could be used in the
vendor-spec, or it could be used as a query towards some kind of
central package repository.

This is just tiny baby steps towards all of those goals. All I'm
proposing at this point is a single representation that can be applied
to a Go project to say "this copy is version 1.0.0 of the quux
project".

Thanks

Dave

1. http://blog.labix.org/2015/01/14/no-minor-versions-in-go-import-paths

[Nika Jones]

Dave:

Cool. Gotcha. This is for the design of a version number that _could_ sit in one or more of several places, without breaking the go1 compatibility contract. Yeah, I understood that you were just making an example and not a language change proposal.

So hopefully the last question; in the original email introducing the proposal you have the line:

> What is missing from Go projects is a mechanism to turn arbitrary dvcs
> hashes into version numbers.

Could you explain that a bit more? I think I'm overthinking what you mean by that line (as in every previous incarnation of this paragraph has resulted in convoluted mess of wat?? as I tried to explain what I think you meant by it.)

Thanks for taking the time explain everything so far.

Nika

>> > email to go-package-management+unsub...@googlegroups.com.

>> > To post to this group, send email to
>> > go-package...@googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Go Package Management" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to go-package-management+unsub...@googlegroups.com.

[Dave Cheney]

On Fri, Aug 14, 2015 at 3:28 PM, Nika Jones <art...@gmail.com> wrote:
> Dave:
>
> Cool. Gotcha. This is for the design of a version number that _could_ sit in
> one or more of several places, without breaking the go1 compatibility
> contract. Yeah, I understood that you were just making an example and not a
> language change proposal.

Correct. I know this seems like a strange proposal to make, at best
all I'm proposing is a common syntax for tagging releases of Go
projects. But I think this is a crucial first step, and one that if
not present, precludes anything else.

>
> So hopefully the last question; in the original email introducing the
> proposal you have the line:
>
>> What is missing from Go projects is a mechanism to turn arbitrary dvcs
>> hashes into version numbers.
>
> Could you explain that a bit more? I think I'm overthinking what you mean by
> that line (as in every previous incarnation of this paragraph has resulted
> in convoluted mess of wat?? as I tried to explain what I think you meant by
> it.)

I believe that currently Go projects do not have versions; at least
not in the way that other projects and libraries in other languages
_do_ have versions. I believe this is because Go projects (I'm
deliberately not saying Go packages here, but if that helps,
s/project/package) do not have a release process. Right now what
exists is someone does a git push, or clicks the merge button on
github and suddenly what go get got yesterday, is different to what go
get gets today. I believe what Go programmers need is a method to
separate the day to day cut and thrust of cutting code, fixing bugs,
twiddling with test fixtures, noodling with documentation, and all the
other stuff that happens in a repo, from the act of saying "yes, we've
finished this milestone, time to slap a version number on this thing
and send out the release announcement".

On the other side, as consumers of Go projects, people have been
asking for years for a way to say things like

"I want to get the latest stable version of github.com/lib/pq" or
"github.com/revel/revel requires lib/pq version 1.2.x"

and right now these sort of questions are impossible to answer,
because ideas like "latest stable", or "version 1.2.whatever" cannot
be answered if all you have is some git sha1.

>
> Thanks for taking the time explain everything so far.

Thank you for helping me get the ideas out of my head and onto paper.
This is precisely the road testing I wanted before proposing this more
formally.

Thanks

Dave

[Peter Bourgon]

Hello,

I'm following this discussion with interest.

I'd like to voice strong support for [strict] semver as the numbering
scheme. It is the de-facto standard, expressive and well understood.

> --
> You received this message because you are subscribed to the Google Groups "Go Package Management" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to go-package-manag...@googlegroups.com.

[Dave Cheney]

Thanks Peter,

I'm on the fence about semver. I can see it's a sane standard that
makes more sense than inventing our own (only slightly different)
variation. My concern, and this is probably inexperience, is there is
no mechanism to express pre-releases. We have a similar problem at
canonical with deb packages, which have to do gross things like ,
go-1.4.99 rather than go 1.5rc1 as that would sort _after_ 1.5 final.

Thanks

Dave

[Juan Benet]

> My concern, and this is probably inexperience, is there is
> no mechanism to express pre-releases.

I believe you do:  <major>.<minor>.<patch>-<candidate>

> Additional labels for pre-release and build metadata are 

> available as extensions to the MAJOR.MINOR.PATCH format.

https://github.com/coreos/go-semver/blob/master/semver/semver.go#L119

And i believe it tries to do int comparison first, falling back to lexicographic:

https://github.com/coreos/go-semver/blob/master/semver/semver.go#L203-L217

[Jason Buberel]

Dave,

So far, I like what I see. And I completely agree with your problem statement "Go projects do not have versions." Some do, but they are the exception. And the community needs a more semantic solution than hash values. I would also like to voice my support for SemVer. I went so far as to include it here:

https://github.com/matttproud/gochecklist/blob/master/publication/api_stability_and_maturity.md

I'd be supportive of your submitting this via the new change proposal mechanism

-jason

[Chris Hines]

On Friday, August 14, 2015 at 5:31:26 AM UTC-4, Dave Cheney wrote:

Thanks Peter,

I'm on the fence about semver. I can see it's a sane standard that
makes more sense than inventing our own (only slightly different)
variation. My concern, and this is probably inexperience, is there is
no mechanism to express pre-releases. We have a similar problem at
canonical with deb packages, which have to do gross things like ,
go-1.4.99 rather than go 1.5rc1 as that would sort _after_ 1.5 final.

Thanks

Dave

I generally like and try to use SemVer, although I haven't read the spec in detail lately. From my quick research I don't think the concern about lack of pre-release semantics is warranted. From the semver.org page:

Semantic Versioning 2.0.0

 

Given a version number MAJOR.MINOR.PATCH, increment the:

1. MAJOR version when you make incompatible API changes,
2. MINOR version when you add functionality in a backwards-compatible manner, and
3. PATCH version when you make backwards-compatible bug fixes.

Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.

 

[...]

 

9. A pre-release version MAY be denoted by appending a hyphen and a series of dot separated identifiers immediately following the patch version. Identifiers MUST comprise only ASCII alphanumerics and hyphen [0-9A-Za-z-]. Identifiers MUST NOT be empty. Numeric identifiers MUST NOT include leading zeroes. Pre-release versions have a lower precedence than the associated normal version. A pre-release version indicates that the version is unstable and might not satisfy the intended compatibility requirements as denoted by its associated normal version. Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92.

(Emphasis mine)

Chris 

[Daniel Theophanes]

I'm fine with semver.

However rather than just use the top level package in a project, I would suggest the top level import path in a project. Like:

golang.org/x/net/webdav-1.2.3-beta

Second, I would propose exploring the possibility of creating the semver with automatically through a change log + api inspection.

(start at "blah/foo-0.0.0")

//ver:patch 2015-08-01 Fix latent bug.

//ver:security,patch 2015-08-02 Fixed handling of DNS requests

//ver:major 2015-08-10 Adjust API.

//ver:minor,-beta 2015-08-11 Add additional functions

The above would result "blah/foo-1.1.0-beta"

Perhaps a pie in the sky idea. Despite semver being fine, I don't really want to think about what the next semver should be; I just want to document what I'm doing and maybe have an API checker to help keep me honest.

-Daniel

On Thursday, August 13, 2015 at 7:22:35 PM UTC-7, Dave Cheney wrote:

[Dave Cheney]

On 15 Aug 2015 2:56 am, "Daniel Theophanes" <kard...@gmail.com> wrote:
>
> I'm fine with semver.
>
> However rather than just use the top level package in a project, I would suggest the top level import path in a project. Like:
> golang.org/x/net/webdav-1.2.3-beta

Technically this would be

golang.org/x/net-1.2.3-beta

Because the WebDAV package lives in the net project (as defined by the remote import metadata). But with that aside this sounds like a good improvement.

> Second, I would propose exploring the possibility of creating the semver with automatically through a change log + api inspection.

That sounds fine but out of scope for this proposal. I want to remain laser focused on defining a single representation of a go project version, which I would enable tools like the one you propose to interoperate.

[Dave Cheney]

On Sat, Aug 15, 2015 at 10:12 AM, Dave Cheney <da...@cheney.net> wrote:
>
> On 15 Aug 2015 2:56 am, "Daniel Theophanes" <kard...@gmail.com> wrote:
>>
>> I'm fine with semver.
>>
>> However rather than just use the top level package in a project, I would
>> suggest the top level import path in a project. Like:
>> golang.org/x/net/webdav-1.2.3-beta
>
> Technically this would be
>
> golang.org/x/net-1.2.3-beta
>
> Because the WebDAV package lives in the net project (as defined by the
> remote import metadata). But with that aside this sounds like a good
> improvement.

Thanks for this suggestion Daniel. I was concerned that mercurial and
git would not like such complex strings for tags, but I've checked and
they are fine.

I think this is a good improvement as it reinforces the canonical
import rule, ie, https://go.googlecode.com/net is not the canonical
name, golang.org/x/net is the canonical name, so the tag form can
reinforce this.

I also like the symmetry with what I expect will be a dependencies.txt
style (need bunder) requirements style file;

tag: golang.org/x/net-1.5.0 // the tag for the go 1.5 release of the net subrepo
requirements: golang.org/x/net [1.5.0,1.6.0) // anything from 1.5.0 up
to but not including 1.6.0

I will include this change when I make the github proposal issue.

Thanks

Dave

[Dave Cheney]

On the other hand, the other option is to not include the name or
import path of the project at all, just a semver version number. This
also has benefits

a. it's shorter, always a win with Go programmers
b. it doesn't clash with potential filenames, can you imagine trying
to curl a file with a slash in it's name ? In fact I think / is a
restricted url character. Even if you could get it onto your
filesystem, a file with / in it is not permitted on unix filesystems
and probably makes a horrible mess on NTFS.

It is clear that my initial proposal for using the top level package
name was incorrect; logger-1.0.0 doesn't mean anything, but given the
difficulties handling other productions of this version syntax in
filenames, urls, etc, maybe not including the import path at all is a
smarter option. Again the tag and requirements rules look sane

tag: net-1.5.0 // the tag on the repository
requirements: golang.org/x/net [1.5.0,1.6.0) // anything from the
golang.org/x/net vanity url with a tag between 1.5.0 and < 1.6.0

What are your thoughts ?

Thanks

Dave

[Daniel Theophanes]

I think using semver for declaring versions is fine.

You've touched on a number of points including project specification to where version tags would be stored. I agree considering those points is required for a relevant discussion in this.

I agree that for practical reasons versions will almost always be stored in a VCS. However I would want to version a package tree separately from the repo. For instance the following three package trees all live in the same repo, but will have very different versions:

 * golang.org/x/net/context (v1.0.y forever)

 * golang.org/x/net/publicsuffix (v1.x.y API unlikely to change, but data will continue to be updated)

 * golang.org/x/net/websocket (v2.x.y API has already changed already I think, may again in the future)

I know that this is different then how Godep, glock, nut, and gb-vendor assume normal operation. All of these tools (roughly speaking) copy in per repo and record the revision per repo. I agree that is the easiest thing to do. I don't think that is the correct thing to do. Thus kardianos/govendor copy packages and record package revisions per package. You can update "golang.org/x/net/publicsuffix" without updating to the latest API in "golang.org/x/net/websocket".

Similarly when I look at a package tree I would want to see (a) what versions are available and a mapping to fetch them (b) optionally any dependency version requirements. It may be tempting to try to hand curate some list, but such a method quickly falls down for larger projects (k8s.io/kubernetes I'm looking at you). At the end of the day I would like to specify package versions using tools that can handle package trees. To emphasize, versions and revisions should be per package, even when tools allow a higher specification.

I realize I've re-framed the question, but hopefully the perspective is useful.

-Daniel

[erik...@gmail.com]

I would agree that the full name in the tag wouldn't necessarily be a great idea... and semver itself is something I'm fine with (although versioning might relate in some ways to the pkg grouping mechanism so I think progress is needed there, more below).

In your example you show 'tag: net-1.5.0' which still has "part" of the name in the tag name (in this case "net"). I might go so far as to think just "1.5.0" would be better (or v1.5.0 if you need a prefix).

I think the version recommendations might need to also come after the project or manifest "plan" or recommendation is solidified (I believe I saw something Daniel had posted related to this?, I think I need to comment on that if it's still up for review?) as there could be some impact to versioning.

I'm thinking about this from the perspective of a past life and an enterprise system I built managing 10's of thousands of components in various VCS's (pluggable), with thousands of developers, across numerous large and small code bases with sharing of components and such. In this system components (pkgs) could have their own "local" naming based on where a codebase wanted that artifact to appear within that codebases source tree. I know Go has the canonical recommendation of using the "base" repo path type of thing in imports and in the workspace path, but I don't think that should be required with a full dependency/version pinning solution... but not prevented either. Anyhow, if any part of the artifacts "name" is in the tag it would be confusing if your final grouping/dependency management solution allowed this type of flexible artifact naming capability (which I would strongly recommend)... although I would agree that any solutions "easiest" use/setup should resemble Go's currently recommended import naming and such).

Thx,
Erik

[Dave Cheney]

I understand what you are proposing, but I think that is too
complicated. I think it is safer and less complicate to make the unit
of versioning the repository. It's possible to find counter examples
where a more fine grained mechanism is possible, and the net sub repo
is a great example, the exp subrepo could be another, but I don't
think these exceptions warrant a more complicated rule.

[Dave Cheney]

On Sun, Aug 16, 2015 at 4:44 AM, <erik...@gmail.com> wrote:
> I would agree that the full name in the tag wouldn't necessarily be a great idea... and semver itself is something I'm fine with (although versioning might relate in some ways to the pkg grouping mechanism so I think progress is needed there, more below).
>
> In your example you show 'tag: net-1.5.0' which still has "part" of the name in the tag name (in this case "net"). I might go so far as to think just "1.5.0" would be better (or v1.5.0 if you need a prefix).

I can see a consensus forming for no prefix, so a vestigial "v" would seem odd.

> I think the version recommendations might need to also come after the project or manifest "plan" or recommendation is solidified (I believe I saw something Daniel had posted related to this?, I think I need to comment on that if it's still up for review?) as there could be some impact to versioning.

I sorry but I disagree. The vendor spec that Daniel is working on is
affected by this very problem, it can track tags, revisions and
branches, but they are all opaque identifiers which should someone
recreate the local software environment, but there is still no
mechanism to say "given the software I depend on today, are there any
updates I could apply ?"

>
> I'm thinking about this from the perspective of a past life and an enterprise system I built managing 10's of thousands of components in various VCS's (pluggable), with thousands of developers, across numerous large and small code bases with sharing of components and such. In this system components (pkgs) could have their own "local" naming based on where a codebase wanted that artifact to appear within that codebases source tree. I know Go has the canonical recommendation of using the "base" repo path type of thing in imports and in the workspace path, but I don't think that should be required with a full dependency/version pinning solution... but not prevented either. Anyhow, if any part of the artifacts "name" is in the tag it would be confusing if your final grouping/dependency management solution allowed this type of flexible artifact naming capability (which I would strongly recommend)... although I would agree that any solutions "easiest" use/setup should resemble Go's currently recommended import naming and such).

I'm sorry, I didn't really understand what you were saying here. You
touched on a lot of points, could you summarise them ?

[Chris Hines]

On Saturday, August 15, 2015 at 12:01:58 PM UTC-4, Daniel Theophanes wrote:

I think using semver for declaring versions is fine.

You've touched on a number of points including project specification to where version tags would be stored. I agree considering those points is required for a relevant discussion in this.

I agree that for practical reasons versions will almost always be stored in a VCS. However I would want to version a package tree separately from the repo. For instance the following three package trees all live in the same repo, but will have very different versions:

 * golang.org/x/net/context (v1.0.y forever)

 * golang.org/x/net/publicsuffix (v1.x.y API unlikely to change, but data will continue to be updated)

 * golang.org/x/net/websocket (v2.x.y API has already changed already I think, may again in the future)

I know it is unlikely to happen, but I would argue that those three packages should be in separate repositories. I teach Git to a lot of people at my $job, most of them migrating from Subversion, and one thing that always comes up is the proper scope of a Git repository. My stock answer is that everything in the same repo should be released as a unit. The rationale is that Git tags are scoped to the repository.

When you create a Git tag, you are tagging the entire source tree on the tagged commit. Git doesn't let you tag a subdirectory. Other VCS's differ, Subversion lets you create tags at any depth. Then again Subversion doesn't have first class tags, but rather a set of conventional operations that result in an artifact commonly considered a tag. Nonetheless, for the time being, Git seems to have won, so mostly we are bound by its rules.

Chris

[Daniel Theophanes]

Dave and Chris,

Under your proposed limits groups who use mono-repos and groups who use vcs where repo tends to be less granular cannot use such a versioning solution.

Please also note that in vcs that lend themselves to smaller repos and where the project situation allows it I agree that the versioned package tree should be the same as a repo. I disagree that it would be more complicated, just different, in the 80% case. I would prefer a design that can still handle the golang/x/net repo case.

If you haven't explored possible implementation mechanisms for such a package tree based version, I would recommend we do so before rejecting it. Believe it or not, not everyone I care about uses a DVCS.

... I am aware that groups who use mono-repos may not rely on versions in the first place and I'm aware most people in open source Go development use git. My point about handling each package remains. I understand that the other tools made to date are mostly repo based. I'm not totally sure we can come up with a better solution then to use VCS tags/branges, but I'd like to try. ...

-Daniel

--
You received this message because you are subscribed to a topic in the Google Groups "Go Package Management" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/go-package-management/B-MHT98dvEU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to go-package-manag...@googlegroups.com.

[Brian Ketelsen]

I have been wondering for a while now whether a Ruby gems style system (centralized storage and versioning) would work well with Go.  If you take the VCS out of the way and make a centralized (or federated?) source of versioning truth things get a lot easier.

I’m sure there are problems with a gem style system, I most certainly haven’t thought this through.

You received this message because you are subscribed to the Google Groups "Go Package Management" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-package-manag...@googlegroups.com.

[Chris Hines]

Daniel,

I am sympathetic to your line of thinking. I recently made the mistake of believing that gb vendoring worked at the package level until I was corrected by Dave. As Dave is fond of saying, currently [most] Go code isn't released. The golang.org/x/net package is no exception. Were it to be released, I believe the authors would be forced to re-evaluate the repository scope. But with the status quo, that is not a problem they face.

Personally, I prefer to release my packages, or closely related sets of packages. I use Git tags to mark the releases in my repos, and that pushes me to create separate repos for separately released code bases.

I would like to see more people take these issues more seriously, so I would love to see a solution that resolves that tension in an elegant way, I just don't know what that looks like at the moment. So if you have some ideas to make this situation better, I would love to see them.

Chris

[Dave Cheney]

Yup, in essence this is a proposal to "release" a set of packages
rooted at some prefix, by tagging the repository they are in with a
semver compatible identifier.

[erik...@gmail.com]

On Saturday, August 15, 2015 at 4:24:31 PM UTC-7, Dave Cheney wrote:
> On Sun, Aug 16, 2015 at 4:44 AM, <erik...@gmail.com> wrote:
> > I would agree that the full name in the tag wouldn't necessarily be a great idea... and semver itself is something I'm fine with (although versioning might relate in some ways to the pkg grouping mechanism so I think progress is needed there, more below).
> >
> > In your example you show 'tag: net-1.5.0' which still has "part" of the name in the tag name (in this case "net"). I might go so far as to think just "1.5.0" would be better (or v1.5.0 if you need a prefix).
>
> I can see a consensus forming for no prefix, so a vestigial "v" would seem odd.

Odd, yes, I would agree (and not something wanted unless absolutely needed).

Mostly I include it as "optional" and to be ignored if used. Why? The issue I had in the past was a VCS system where tags had to start with an alpha (non-numeric) character. Allowing that was required.

If you check this out (npm packaging system semver details): https://docs.npmjs.com/misc/semver

Note: This touches on an optional 'v' prefix to the version that is ignored, but also goes into great detail about semver matching and ranges and such (which I would recommend be followed, it includes recommended logic on handling extensions like alpha/beta and such as well). Good stuff.

> > I think the version recommendations might need to also come after the project or manifest "plan" or recommendation is solidified (I believe I saw something Daniel had posted related to this?, I think I need to comment on that if it's still up for review?) as there could be some impact to versioning.
>
> I sorry but I disagree. The vendor spec that Daniel is working on is
> affected by this very problem, it can track tags, revisions and
> branches, but they are all opaque identifiers which should someone
> recreate the local software environment, but there is still no
> mechanism to say "given the software I depend on today, are there any
> updates I could apply ?"

Understood now... that you are trying to get at a way to set up "dynamic" version selections (having brushed through the other spec I didn't fully grasp that the first time around). See above link to how NPM does this (match that, perhaps extend it as below). I would go with that.

The above could be extended to also allow something like this:

pkgx@branch/latest (or just "pkgx@branch" to indicate it's riding latest)

I use the term "branch" here loosely in that I might be talking about a multi-pkg development line which has 10 pkg's in it and perhaps those packages are coming from 10 different branches to form that overall "project"... but they are aggregated by a versioned (and fully branch-able) dependency specification "file" (what I might call a "development line" or "lineup" but often referred to as a manifest or dependency list). Just another way to dynamically track work (that for a given org could be preferred to semver rules like NPM's).

As to the versioning... and your disagreement with my thought there, let me try and clarify that end of things. Some terminology might help so let me try this:

project - this would be my deliverable (say 'cobra') or my tool (e.g.: 'camlistore')
-> perhaps 'codebase' or 'deliverable' might be better terms, dunno

Within this "project" I'm delivering I have my code and I likely have dependencies on other projects (i.e.: I'm using other packages for instance, or the std library packages... and they may have transitive dependencies/etc). Internally in my project I might have a few different lines of development... the v1 work on a branch and the v2 work on another branch, etc. Each of these branches might have a development line definition (lineup/manifest/etc) file which identifies what projects it depends on and what versions of each is needed, e.g.:

Example lineup/manifest (of "projects" I depend upon, could be packages w/deps, etc):

pk...@2.4.x
pk...@9.4.1
pkgz@1.x

The version of your "project" that is pulled would determine which dependency file was visible and what dependent projects and versions were needed to build (i.e.: the manifest/lineup file is likely versioned with your tool/pkg "top level" code, and branched with it as needed, etc).

A system like NPM will assume that I can fully identify a version of the project via a single semantic version (9.4.1). This is because this is a requirement for how NPM lineup/manifest files are versioned (to my point). I am fine with this, but I think it needs to be specified as part of how the lineup/manifest file is stored (to some extent)... as not doing so could impact versioning.

Example: Imagine it isn't specified fully that a given semantic version *must* map to exactly *one* version of the overall "project". I've seen this before. We had a "project" that has 5 different lines of development (multiple projects were used/needed by our project), so a v1, v2, v3, v4, v5 set of branches (logically hierarchical) with each branch containing the "lineup" (or manifest) which contains all dependencies of the pulled/cloned project version. If each of these lineups was put in a versioned RCS file, v1.lineup, v2.lineup, v3.lineup, .. then one could, theoretically, tag the v1 lineup/manifest with 1.0.1 but one could also tag the v2 lineup the same exact way (it's a separate RCS file, each one independently versioned along it's "main" branch but not related otherwise). Now if this overall project was called "projx" then referring to pr...@1.0.1 would not map to a single instance of the project. You would need to say 'pr...@v1-1.0.1' or something like that. Anyhow, that's where that statement came from.

> >
> > I'm thinking about this from the perspective of a past life and an enterprise system I built managing 10's of thousands of components in various VCS's (pluggable), with thousands of developers, across numerous large and small code bases with sharing of components and such. In this system components (pkgs) could have their own "local" naming based on where a codebase wanted that artifact to appear within that codebases source tree. I know Go has the canonical recommendation of using the "base" repo path type of thing in imports and in the workspace path, but I don't think that should be required with a full dependency/version pinning solution... but not prevented either. Anyhow, if any part of the artifacts "name" is in the tag it would be confusing if your final grouping/dependency management solution allowed this type of flexible artifact naming capability (which I would strongly recommend)... although I would agree that any solutions "easiest" use/setup should resemble Go's currently recommended import naming and such).
>
> I'm sorry, I didn't really understand what you were saying here. You
> touched on a lot of points, could you summarise them ?

Apologies... not well said. Hard to summarize a fairly powerful system with lots of capabilities "in brief". ;) Let me try and relate it to something else... say 'cargo', the package manager for rust. One can grab any repo and rename it so it appears under a different name in my local workspace. The cargo syntax is something like this (see http://doc.crates.io/pkgid-spec.html):

crates.io/bar#foo:1.2.3

Which says grab the crates.io/bar pkg from their repository system at version 1.2.3 (think github.com/x/bar) and place it into my workspace under the name 'foo' because that's what my codebase likes to see it as. The system I worked on had a capability like this. Anyone could use that artifact (project/pkg/repo) and they could put it anywhere in their workspace tree they wanted (different client "projects" had source tree's with different structural or dir layout needs so it was named differently for different "projects"). In this case if the person with the 'foo' project in their workspace labels it they might not know (easily/trivially) it should be bar-1.2.4 instead of foo-1.2.4 when they tag it (due to various names being possible).

Does that make sense? Ties back to why a name prefix might not be desired.

Beyond that I was trying to say that any Go lineup/manifest/whatever it's called file would need to consider this type of capability. If I want to bring this thing in with some flexibility for my tree's needs then being able to name it locally what I want would be very useful. I probably wouldn't go with the above syntax, maybe for Go I want to "fake things out" so I don't have to re-write import paths:

github.com/myfork/cobra#github.com/spf13/co...@1.2.3

or maybe (or whatever):

github.com/myfork/cobra[ws:github.com/spf13/cobra]@1.2.3

Says grab my version of cobra (my fork, which I can commit to and is stable for my needs) and bring it in under the "original" name so as to avoid import path re-writing. I know when I commit where it's going so all good. For Go perhaps one could also do this:

github.com/myfork/cobra#wslink:github.com/spf13/co...@1.2.3

Which says to bring it in under my name but add a symlink from the "orig" name so I don't have to do import path re-writing. I would always need to use the original name in my imports (otherwise Go would seem them as different modules and not share global vars/etc, at least how it seems to currently work). So the secondary point was that thought should go towards flexible ways to specify projects in the lineup/manifest type file to allow renaming.

Note: I might go beyond what cargo does in terms of giving flexibility here ... allowing something like this but also allowing an *optional* separate file (call it the "project definition file") that could be used for any codebase that needed to do something like this or add a lot of meta-data around the project or it's dependent packages and such. I would need to think about it a bit ... but could toss out some ideas if any interest (the idea being that packages like NPM identify a fair amount of meta-data and can put it in with the version specification file and such... but I think that should be a separate file for some of that data that doesn't change often as it can convolute the basic manifest file).

Not sure if the above fully clarifies... likely too verbose (one of my "challenges") ;)

Cheers,
Erik

[Konstantin Khomoutov]

On Fri, 14 Aug 2015 19:31:25 +1000
Dave Cheney <da...@cheney.net> wrote:

> I'm on the fence about semver. I can see it's a sane standard that
> makes more sense than inventing our own (only slightly different)
> variation. My concern, and this is probably inexperience, is there is
> no mechanism to express pre-releases. We have a similar problem at
> canonical with deb packages, which have to do gross things like ,
> go-1.4.99 rather than go 1.5rc1 as that would sort _after_ 1.5 final.

Tcl has a good definition for package versions including "alpha" and
"beta" releases, codified in its builting "package vcompare" command.
See the "Version Numbers" section in [1].

I honestly don't know much about "semver" -- as I tend to ignore
buzzwords kicked around by kids -- and the Tcl's approach predates it
by a decade at least. So I'm not sure whether it's better or worse but
at least it's known to work (i.e. battle tested).

1. https://www.tcl.tk/man/tcl/TclCmd/package.htm

[Konstantin Khomoutov]

On Sat, 15 Aug 2015 21:09:11 -0400
Brian Ketelsen <bket...@gmail.com> wrote:

> I have been wondering for a while now whether a Ruby gems style
> system (centralized storage and versioning) would work well with Go.
> If you take the VCS out of the way and make a centralized (or
> federated?) source of versioning truth things get a lot easier.
>
> I’m sure there are problems with a gem style system, I most certainly
> haven’t thought this through.

This reminded me of [1].
While centralized things are wonderful, they have severe downsides too.

1. https://www.reddit.com/r/rust/comments/35kf6h/now_that_cratesio_is_down/

[Dave Cheney]

Sure. We all know the dangers of single points of failure. This is not
a proposal for central repository, not even close, just a proposal
that people start tagging their git repos with a meaningful tag.

[Dave Cheney]

Thanks for your answer Erik. Please accept my apologies if I don't
address all your points in this reply.

On Mon, Aug 17, 2015 at 4:54 AM, <erik...@gmail.com> wrote:
> On Saturday, August 15, 2015 at 4:24:31 PM UTC-7, Dave Cheney wrote:
>> On Sun, Aug 16, 2015 at 4:44 AM, <erik...@gmail.com> wrote:
>> > I would agree that the full name in the tag wouldn't necessarily be a great idea... and semver itself is something I'm fine with (although versioning might relate in some ways to the pkg grouping mechanism so I think progress is needed there, more below).
>> >
>> > In your example you show 'tag: net-1.5.0' which still has "part" of the name in the tag name (in this case "net"). I might go so far as to think just "1.5.0" would be better (or v1.5.0 if you need a prefix).
>>
>> I can see a consensus forming for no prefix, so a vestigial "v" would seem odd.
>
> Odd, yes, I would agree (and not something wanted unless absolutely needed).
>
> Mostly I include it as "optional" and to be ignored if used. Why? The issue I had in the past was a VCS system where tags had to start with an alpha (non-numeric) character. Allowing that was required.

You make a good point. As this proposal is driving heavily towards the
format of the tag that will be expected to be applied to "released" Go
packages, it is heavily influenced by what is practical with vcs
systems. You don't know the one that wouldn't accept a number as the
first digit offhand ?

> If you check this out (npm packaging system semver details): https://docs.npmjs.com/misc/semver
>
> Note: This touches on an optional 'v' prefix to the version that is ignored, but also goes into great detail about semver matching and ranges and such (which I would recommend be followed, it includes recommended logic on handling extensions like alpha/beta and such as well). Good stuff.

I'm not a fan of Postal's law in this instance. I've seen the mess
that too many optional forms has made of python's various versioning
schemes. I'm erring on the side of strictness here, and if there are
popular vcs's that need an alpha char in the first position, that
gives further weight to including the import prefix in the tag.

>> > I think the version recommendations might need to also come after the project or manifest "plan" or recommendation is solidified (I believe I saw something Daniel had posted related to this?, I think I need to comment on that if it's still up for review?) as there could be some impact to versioning.
>>
>> I sorry but I disagree. The vendor spec that Daniel is working on is
>> affected by this very problem, it can track tags, revisions and
>> branches, but they are all opaque identifiers which should someone
>> recreate the local software environment, but there is still no
>> mechanism to say "given the software I depend on today, are there any
>> updates I could apply ?"
>
> Understood now... that you are trying to get at a way to set up "dynamic" version selections (having brushed through the other spec I didn't fully grasp that the first time around). See above link to how NPM does this (match that, perhaps extend it as below). I would go with that.

This proposal is only for the format of a unique tag for a specific
released copy of some source code.

> The above could be extended to also allow something like this:
>
> pkgx@branch/latest (or just "pkgx@branch" to indicate it's riding latest)

Let's eat one elephant at a time.

> I use the term "branch" here loosely in that I might be talking about a multi-pkg development line which has 10 pkg's in it and perhaps those packages are coming from 10 different branches to form that overall "project"... but they are aggregated by a versioned (and fully branch-able) dependency specification "file" (what I might call a "development line" or "lineup" but often referred to as a manifest or dependency list). Just another way to dynamically track work (that for a given org could be preferred to semver rules like NPM's).
>
> As to the versioning... and your disagreement with my thought there, let me try and clarify that end of things. Some terminology might help so let me try this:
>
> project - this would be my deliverable (say 'cobra') or my tool (e.g.: 'camlistore')
> -> perhaps 'codebase' or 'deliverable' might be better terms, dunno
>
> Within this "project" I'm delivering I have my code and I likely have dependencies on other projects (i.e.: I'm using other packages for instance, or the std library packages... and they may have transitive dependencies/etc). Internally in my project I might have a few different lines of development... the v1 work on a branch and the v2 work on another branch, etc. Each of these branches might have a development line definition (lineup/manifest/etc) file which identifies what projects it depends on and what versions of each is needed, e.g.:
>
> Example lineup/manifest (of "projects" I depend upon, could be packages w/deps, etc):
>
> pk...@2.4.x
> pk...@9.4.1
> pkgz@1.x
>
> The version of your "project" that is pulled would determine which dependency file was visible and what dependent projects and versions were needed to build (i.e.: the manifest/lineup file is likely versioned with your tool/pkg "top level" code, and branched with it as needed, etc).

I defined a project as the repository the code is stored in. While it
may be possible to release a subpart of a repository separately, I'm
not in favor of this as much more complicated to explain to newcomers
which is always a driving concern for me. I'm sorry for skipping the
next two paragraphs.

> A system like NPM will assume that I can fully identify a version of the project via a single semantic version (9.4.1). This is because this is a requirement for how NPM lineup/manifest files are versioned (to my point). I am fine with this, but I think it needs to be specified as part of how the lineup/manifest file is stored (to some extent)... as not doing so could impact versioning.
>
> Example: Imagine it isn't specified fully that a given semantic version *must* map to exactly *one* version of the overall "project". I've seen this before. We had a "project" that has 5 different lines of development (multiple projects were used/needed by our project), so a v1, v2, v3, v4, v5 set of branches (logically hierarchical) with each branch containing the "lineup" (or manifest) which contains all dependencies of the pulled/cloned project version. If each of these lineups was put in a versioned RCS file, v1.lineup, v2.lineup, v3.lineup, .. then one could, theoretically, tag the v1 lineup/manifest with 1.0.1 but one could also tag the v2 lineup the same exact way (it's a separate RCS file, each one independently versioned along it's "main" branch but not related otherwise). Now if this overall project was called "projx" then referring to pr...@1.0.1 would not map to a single instance of the project. You would need to say 'pr...@v1-1.0.1' or something like that. Anyhow, that's where that statement came from.
>
>> >
>> > I'm thinking about this from the perspective of a past life and an enterprise system I built managing 10's of thousands of components in various VCS's (pluggable), with thousands of developers, across numerous large and small code bases with sharing of components and such. In this system components (pkgs) could have their own "local" naming based on where a codebase wanted that artifact to appear within that codebases source tree. I know Go has the canonical recommendation of using the "base" repo path type of thing in imports and in the workspace path, but I don't think that should be required with a full dependency/version pinning solution... but not prevented either. Anyhow, if any part of the artifacts "name" is in the tag it would be confusing if your final grouping/dependency management solution allowed this type of flexible artifact naming capability (which I would strongly recommend)... although I would agree that any solutions "easiest" use/setup should resemble Go's currently recommended import naming and such).
>>
>> I'm sorry, I didn't really understand what you were saying here. You
>> touched on a lot of points, could you summarise them ?
>
> Apologies... not well said. Hard to summarize a fairly powerful system with lots of capabilities "in brief". ;) Let me try and relate it to something else... say 'cargo', the package manager for rust. One can grab any repo and rename it so it appears under a different name in my local workspace. The cargo syntax is something like this (see http://doc.crates.io/pkgid-spec.html):
>
> crates.io/bar#foo:1.2.3

This is a specification for describing a dependency on another
package. This is different to the proposal I am making, and I'm trying
to keep the scope of my proposal as small as possible as I'm already
swimming upstream.

>
> Which says grab the crates.io/bar pkg from their repository system at version 1.2.3 (think github.com/x/bar) and place it into my workspace under the name 'foo' because that's what my codebase likes to see it as. The system I worked on had a capability like this. Anyone could use that artifact (project/pkg/repo) and they could put it anywhere in their workspace tree they wanted (different client "projects" had source tree's with different structural or dir layout needs so it was named differently for different "projects"). In this case if the person with the 'foo' project in their workspace labels it they might not know (easily/trivially) it should be bar-1.2.4 instead of foo-1.2.4 when they tag it (due to various names being possible).
>
> Does that make sense? Ties back to why a name prefix might not be desired.

This sounds like a proposal for renaming imports, possibly rewriting
imports, on the way in. This is outside the scope of my proposal, and
I am strongly in the camp of not rewriting imports, ever.

> Beyond that I was trying to say that any Go lineup/manifest/whatever it's called file would need to consider this type of capability. If I want to bring this thing in with some flexibility for my tree's needs then being able to name it locally what I want would be very useful. I probably wouldn't go with the above syntax, maybe for Go I want to "fake things out" so I don't have to re-write import paths:
>
> github.com/myfork/cobra#github.com/spf13/co...@1.2.3
>
> or maybe (or whatever):
>
> github.com/myfork/cobra[ws:github.com/spf13/cobra]@1.2.3
>
> Says grab my version of cobra (my fork, which I can commit to and is stable for my needs) and bring it in under the "original" name so as to avoid import path re-writing. I know when I commit where it's going so all good. For Go perhaps one could also do this:
>
> github.com/myfork/cobra#wslink:github.com/spf13/co...@1.2.3

This all looks horribly complicated. I think this is further evidence
for not allowing sub portions of a project to have their own release
number. You'll have to release a project as a whole.

>
> Which says to bring it in under my name but add a symlink from the "orig" name so I don't have to do import path re-writing. I would always need to use the original name in my imports (otherwise Go would seem them as different modules and not share global vars/etc, at least how it seems to currently work). So the secondary point was that thought should go towards flexible ways to specify projects in the lineup/manifest type file to allow renaming.

The go tool (and gb) disallow symlinks as they allow the source of one
package to appear twice in the import graph.

[Daniel Theophanes]

Hi Chris,

Fair enough. Let me explore one possibility of several I've thought of. Let's take the "golang.org/x/net" example again and let's continue to use repo tags/branches to associate version numbers with revisions.

Quick specification:

 1. Versions package trees cannot be nested.

 2. If repo != package tree root, specify releases by "<relative-path-to-root>-semver"

 3. If repo == package tree root, specify releases by "semver"

Quick Implementation:

 1. Update repo (not files, "git fetch") in GOPATH.

 2. Checkout specific tree version into $PROJECT/vendor/... folder. Only copy used packages.

Example:

    "golang.org/x/net/context" the tag would be "context-1.0.0"

For tool implementations that already work with packages and not repos, it doesn't add any additional complexity. It would add one additional step for tools tools that currently only work with repos.

I realize this isn't what other tool makers are used to implementing nor is it what they hold as an ideal. But to me it would appear to be a net gain.

-Daniel

--

[erik...@gmail.com]

On Monday, August 17, 2015 at 4:25:53 AM UTC-7, Dave Cheney wrote:
> Thanks for your answer Erik. Please accept my apologies if I don't
> address all your points in this reply.

No worries... appreciate the time. No response required here.

> On Mon, Aug 17, 2015 at 4:54 AM, <erik...@gmail.com> wrote:
> > On Saturday, August 15, 2015 at 4:24:31 PM UTC-7, Dave Cheney wrote:
> >> On Sun, Aug 16, 2015 at 4:44 AM, <erik...@gmail.com> wrote:
> >> > I would agree that the full name in the tag wouldn't necessarily be a great idea... and semver itself is something I'm fine with (although versioning might relate in some ways to the pkg grouping mechanism so I think progress is needed there, more below).
> >> >
> >> > In your example you show 'tag: net-1.5.0' which still has "part" of the name in the tag name (in this case "net"). I might go so far as to think just "1.5.0" would be better (or v1.5.0 if you need a prefix).
> >>
> >> I can see a consensus forming for no prefix, so a vestigial "v" would seem odd.
> >
> > Odd, yes, I would agree (and not something wanted unless absolutely needed).
> >
> > Mostly I include it as "optional" and to be ignored if used. Why? The issue I had in the past was a VCS system where tags had to start with an alpha (non-numeric) character. Allowing that was required.
>
> You make a good point. As this proposal is driving heavily towards the
> format of the tag that will be expected to be applied to "released" Go
> packages, it is heavily influenced by what is practical with vcs
> systems. You don't know the one that wouldn't accept a number as the
> first digit offhand ?

That was an internal system at a previous job I had. The naming restriction (in that system) made it trivial to decide if what was given was a tag, a branch or another VCS specifier of sorts (it was actually an upper case alpha char to start tags). Think of it as a "governance" decision even if the base VCS did not require it (it reduced confusion amongst the clientele and sped up some operations and avoided tag: prefixes and such). Basically VCS's may be used in interesting ways by different entities (enterprise infra/etc), and some flexibility in version "naming" would be beneficial to those environments.

I do understand the somewhat opposing desire to keep a spec as simple/tight as possible to avoid bloat... as I know such items add up over time. However, like the NPM semantic version doc/url below, having a couple (extras) to allow for greater or easier adoption could be worth it.

> > If you check this out (npm packaging system semver details): https://docs.npmjs.com/misc/semver
> >
> > Note: This touches on an optional 'v' prefix to the version that is ignored, but also goes into great detail about semver matching and ranges and such (which I would recommend be followed, it includes recommended logic on handling extensions like alpha/beta and such as well). Good stuff.
>
> I'm not a fan of Postal's law in this instance. I've seen the mess
> that too many optional forms has made of python's various versioning
> schemes. I'm erring on the side of strictness here, and if there are
> popular vcs's that need an alpha char in the first position, that
> gives further weight to including the import prefix in the tag.

I might disagree about the perceived weight it gives to the import name related prefixing as I still don't think that is a good idea and is, I believe, not directly related to allowing a single alpha char prefix to handle some governance or limitation that might be in play with some VCS's in enterprise/etc.

As to the NPM semver spec reference... yes, it refers to the single character prefix but I was more focusing on this aspect of what you stated (with reference to any manifest format that might be used and bringing in dynamic versions... at least I assumed that is what you were referring to):

"golang.org/x/net [1.5.0,1.6.0) // anything from 1.5.0 up to but not including 1.6.0"

Hence that put me into the though of... "oh, we're talking about manifest version selection and such so I'll spout of some silly thoughts there". ;)

If we go there then I would say I prefer the NPM mechanism for specifying this, which could be done a few ways:

"golang.org/x/net 1.5.x" // any valid 1.5.# would work (except alpha/betas)
"golang.org/x/net ~1.5.2" // 1.5.2 and greater but not 1.6.0 or above (no alpha/beta)
"golang.org/x/net 1.5.2-1.6 // >= 1.5.2 and <1.6.0
"golang.org/x/net 1.5.2-1.6.3" // >=1.5.2 and <1.6.3

As covered there one can specify ranges, major, minor, maintenance, etc bumps in various ways that give a fair amount of flexibility. I may have misunderstood your point though with the above bracketed range and dynamic version selection (perhaps for Daniel's manifest discussion is where that discussion belongs or a new thread... is there a preferred thread?).

Aside: dynamic versions within a manifest have their own problems ... fine to use for small teams and such but not something that scales well for large teams/code bases (and can potentially lead to version skew if you have no cross-pkg dependency system in place, semver will help). In such environments fully static manifests that are published to work best IMO (atomically publishing 1..N project/versions at the same time when needed).

> >> > I think the version recommendations might need to also come after the project or manifest "plan" or recommendation is solidified (I believe I saw something Daniel had posted related to this?, I think I need to comment on that if it's still up for review?) as there could be some impact to versioning.
> >>
> >> I sorry but I disagree. The vendor spec that Daniel is working on is
> >> affected by this very problem, it can track tags, revisions and
> >> branches, but they are all opaque identifiers which should someone
> >> recreate the local software environment, but there is still no
> >> mechanism to say "given the software I depend on today, are there any
> >> updates I could apply ?"
> >
> > Understood now... that you are trying to get at a way to set up "dynamic" version selections (having brushed through the other spec I didn't fully grasp that the first time around). See above link to how NPM does this (match that, perhaps extend it as below). I would go with that.
>
> This proposal is only for the format of a unique tag for a specific
> released copy of some source code.

Thanks for clarifying your key goal.

I guess I was confused with discussions like the above "range" selection that might go into a manifest and such... perhaps I will table manifest/lineup and workspace mgmt discussions for another thread with that focus. I have no shortage of opinions there. ;)

As to "basic" semver's placed on a released package and it's VCS... I'm good with it with my stated preference of no name preceding it (related to the repo naming) and I would personally prefer it if an optional single alpha char prefix (perhaps restrict to 'v' and 'V') would be allowed and ignored (but I would survive without the latter I suppose, I just recommend it).

> > The above could be extended to also allow something like this:
> >
> > pkgx@branch/latest (or just "pkgx@branch" to indicate it's riding latest)
>
> Let's eat one elephant at a time.

Fine with that "to some extent"... but if/when we get back to manifest discussions there would be a concern that what was chosen there could become canonical and if you leave things out in early drafts it would be that much harder to add them into later drafts (as use of the new syntax would break all tools relying upon the early draft of how it worked... unless one knew all tools one was using that drove off that draft... and that they now supported version <x/new> of the draft... but it gets much more challenging doing this later... or so I think). Anyhow, manifest discussions for another day.

> > I use the term "branch" here loosely in that I might be talking about a multi-pkg development line which has 10 pkg's in it and perhaps those packages are coming from 10 different branches to form that overall "project"... but they are aggregated by a versioned (and fully branch-able) dependency specification "file" (what I might call a "development line" or "lineup" but often referred to as a manifest or dependency list). Just another way to dynamically track work (that for a given org could be preferred to semver rules like NPM's).
> >
> > As to the versioning... and your disagreement with my thought there, let me try and clarify that end of things. Some terminology might help so let me try this:
> >
> > project - this would be my deliverable (say 'cobra') or my tool (e.g.: 'camlistore')
> > -> perhaps 'codebase' or 'deliverable' might be better terms, dunno
> >
> > Within this "project" I'm delivering I have my code and I likely have dependencies on other projects (i.e.: I'm using other packages for instance, or the std library packages... and they may have transitive dependencies/etc). Internally in my project I might have a few different lines of development... the v1 work on a branch and the v2 work on another branch, etc. Each of these branches might have a development line definition (lineup/manifest/etc) file which identifies what projects it depends on and what versions of each is needed, e.g.:
> >
> > Example lineup/manifest (of "projects" I depend upon, could be packages w/deps, etc):
> >
> > pk...@2.4.x
> > pk...@9.4.1
> > pkgz@1.x
> >
> > The version of your "project" that is pulled would determine which dependency file was visible and what dependent projects and versions were needed to build (i.e.: the manifest/lineup file is likely versioned with your tool/pkg "top level" code, and branched with it as needed, etc).
>
> I defined a project as the repository the code is stored in. While it
> may be possible to release a subpart of a repository separately, I'm
> not in favor of this as much more complicated to explain to newcomers
> which is always a driving concern for me. I'm sorry for skipping the
> next two paragraphs.

OK. I wasn't concerning myself with current VCS's containing N (many) packages (yet)... I was instead discussing more that the spec needs to indicate that there is a 1:1 mapping from semver to just one "project" (codebase) version, always.

If I had to comment offhand on items like 'net' that have many sub-packages and such... that would tie into the manifest discussion and how to manage more complex tree's, nested VCS's in the workspace and such. The std lib is a more "complex" source base and would probably use that mechanism to form a workspace and such if you wanted more modular use of VCS's (eventually).

I know your watchword is "KISS"... and that's in line with the Go ecosystem and the opinionated Go tools and workspace structure and such... and agree that should always remain an overall goal (until it falls down under certain needs, then extensions to handle those needs, i.e.: a solution that still aims towards simplicity as much as possible but provides reproducible builds). As indicated the current "simple" workflow shouldn't go away for simple use cases. For more complicated use cases though I think a more powerful story is needed.

> > A system like NPM will assume that I can fully identify a version of the project via a single semantic version (9.4.1). This is because this is a requirement for how NPM lineup/manifest files are versioned (to my point). I am fine with this, but I think it needs to be specified as part of how the lineup/manifest file is stored (to some extent)... as not doing so could impact versioning.
> >
> > Example: Imagine it isn't specified fully that a given semantic version *must* map to exactly *one* version of the overall "project". I've seen this before. We had a "project" that has 5 different lines of development (multiple projects were used/needed by our project), so a v1, v2, v3, v4, v5 set of branches (logically hierarchical) with each branch containing the "lineup" (or manifest) which contains all dependencies of the pulled/cloned project version. If each of these lineups was put in a versioned RCS file, v1.lineup, v2.lineup, v3.lineup, .. then one could, theoretically, tag the v1 lineup/manifest with 1.0.1 but one could also tag the v2 lineup the same exact way (it's a separate RCS file, each one independently versioned along it's "main" branch but not related otherwise). Now if this overall project was called "projx" then referring to pr...@1.0.1 would not map to a single instance of the project. You would need to say 'pr...@v1-1.0.1' or something like that. Anyhow, that's where that statement came from.
> >
> >> >
> >> > I'm thinking about this from the perspective of a past life and an enterprise system I built managing 10's of thousands of components in various VCS's (pluggable), with thousands of developers, across numerous large and small code bases with sharing of components and such. In this system components (pkgs) could have their own "local" naming based on where a codebase wanted that artifact to appear within that codebases source tree. I know Go has the canonical recommendation of using the "base" repo path type of thing in imports and in the workspace path, but I don't think that should be required with a full dependency/version pinning solution... but not prevented either. Anyhow, if any part of the artifacts "name" is in the tag it would be confusing if your final grouping/dependency management solution allowed this type of flexible artifact naming capability (which I would strongly recommend)... although I would agree that any solutions "easiest" use/setup should resemble Go's currently recommended import naming and such).
> >>
> >> I'm sorry, I didn't really understand what you were saying here. You
> >> touched on a lot of points, could you summarise them ?
> >
> > Apologies... not well said. Hard to summarize a fairly powerful system with lots of capabilities "in brief". ;) Let me try and relate it to something else... say 'cargo', the package manager for rust. One can grab any repo and rename it so it appears under a different name in my local workspace. The cargo syntax is something like this (see http://doc.crates.io/pkgid-spec.html):
> >
> > crates.io/bar#foo:1.2.3
>
> This is a specification for describing a dependency on another
> package. This is different to the proposal I am making, and I'm trying
> to keep the scope of my proposal as small as possible as I'm already
> swimming upstream.

I'm a little slow but I think it's sinking in... thanks for your patience. :)

> >
> > Which says grab the crates.io/bar pkg from their repository system at version 1.2.3 (think github.com/x/bar) and place it into my workspace under the name 'foo' because that's what my codebase likes to see it as. The system I worked on had a capability like this. Anyone could use that artifact (project/pkg/repo) and they could put it anywhere in their workspace tree they wanted (different client "projects" had source tree's with different structural or dir layout needs so it was named differently for different "projects"). In this case if the person with the 'foo' project in their workspace labels it they might not know (easily/trivially) it should be bar-1.2.4 instead of foo-1.2.4 when they tag it (due to various names being possible).
> >
> > Does that make sense? Ties back to why a name prefix might not be desired.
>
> This sounds like a proposal for renaming imports, possibly rewriting
> imports, on the way in. This is outside the scope of my proposal, and
> I am strongly in the camp of not rewriting imports, ever.

I too am strongly opposed to rewriting imports... so you misunderstand.
It's not a proposal for that. This is more towards "naming thoughts"...
which could potentially work towards a proposal which could "augment"
the imports mechanism so one would NOT have to re-write them.

> > Beyond that I was trying to say that any Go lineup/manifest/whatever it's called file would need to consider this type of capability. If I want to bring this thing in with some flexibility for my tree's needs then being able to name it locally what I want would be very useful. I probably wouldn't go with the above syntax, maybe for Go I want to "fake things out" so I don't have to re-write import paths:
> >
> > github.com/myfork/cobra#github.com/spf13/co...@1.2.3
> >
> > or maybe (or whatever):
> >
> > github.com/myfork/cobra[ws:github.com/spf13/cobra]@1.2.3
> >
> > Says grab my version of cobra (my fork, which I can commit to and is stable for my needs) and bring it in under the "original" name so as to avoid import path re-writing. I know when I commit where it's going so all good. For Go perhaps one could also do this:
> >
> > github.com/myfork/cobra#wslink:github.com/spf13/co...@1.2.3
>
> This all looks horribly complicated. I think this is further evidence
> for not allowing sub portions of a project to have their own release
> number. You'll have to release a project as a whole.

I don't think we can create a powerful manifest mechanism without having
some flexibility/capabilities. Ideally that complexity will be hidden from most
use cases but having some flexibility will allow Go to be used in more situations, more
powerfully. I'll agree that simplicy of use is the #1 key goal (tied into robustness)
but I think some power will need to be "optionally" available.

> > Which says to bring it in under my name but add a symlink from the "orig" name so I don't have to do import path re-writing. I would always need to use the original name in my imports (otherwise Go would seem them as different modules and not share global vars/etc, at least how it seems to currently work). So the secondary point was that thought should go towards flexible ways to specify projects in the lineup/manifest type file to allow renaming.
>
> The go tool (and gb) disallow symlinks as they allow the source of one
> package to appear twice in the import graph.

I've been using symlinks with some success... as long as I always import from the "canonical" path and all my pkg's import with that (even if it is a symlink) then Go seems to do the right thing today (I know it's not guaranteed by the Go authors but it does work... so far). Why do I do this?:

a) I've forked a number of repo's on github so my path to them is different (github.com/me/...)
- some changes won't be desired by the authors so forks will likely remain

b) I despise import path re-writing... so I use symlinks to get around that and I now import from the "orig" pkg owners path (which is a symlink to my copy)... I never import my copies path.
- if I import my path it's treated as a separate pkg by Go (similar to the 1.5 vendor exp I think if the same pkg is imported many times), so I don't do that
- aside: would be nice if the Go tool "recognized" symlinks and treated as one pkg with two "names"... preferred to this though would be optional support for an "alias" mechanism (below)

c) I dislike the idea of taking a snapshot of various packages and dropping them into my codebase (vendoring in this way sucks IMO), I want to leverage git and merging and history and such and having a full clone is far more powerful

This does work, although I believe far better workflows are possible if the workspace and pkg (project) management is improved (via manifest and related definition files). I don't want to work how I do now but I haven't seen what I would call a great alternative (yet).

My preference for a "real" solution (not my current workflow) would be to have Go work as it does today if you're not using the workspace mgmt/manifest mgmt mechanism that eventually comes about from discussions on this list/elsewhere. Ride latest, use canonical paths, optionally support a way to avoid import path re-writing (that being the the only "simple workflow" extension I might recommend). Lets call it the "alias" extension... and a file that lists aliases is optional but if used it would result in 'import' statements being routed through those aliases before resolving to a name (and the Go tool recognizing these and allowing aliases to be used when resolving to the pkg to use). Not symlinks. Basically the DAG would supported multiple aliases that would map to a single pkg's node in the tree. The alias mechanism would cross over with the fuller solution below (obviously the config would be as simple as possible just for this one capability, but teams could grow this into the fuller solution as needed). Note that it's not just aliases, it's the users preferred local workspace naming with aliases.

The more complex solution: manifest with version pinning (and dynamic semver selection) and workspace mgmt solution would, once configured, still "feel" the same (given time the Go tool could recognize whatever solution "wins out" here and, after that, "detect" a client was in a workspace backed by this and 'go get' and 'go build' and such could leverage it's data and simply do the right thing... eventually). As to details on this solution, that should be a fun discussion.

I believe it's possible without getting overly complicated (at least for the user). I believe for Go to blossom it needs a reproducible build story that is far better. That's not to say the implementation of said tooling and infra might not have complexities but that the end user use cases for day to day use should remain fairly trivial (with setup of codebases with more "complex" needs like a manifest vs the current Go tools capabilities requiring a bit more work of course since manifests aren't free).

Anyhow, no need to respond if you don't wish as I think I digressed. ;) Thanks for the time.

Cheers,
Erik

[Eric Johnson]

What concerns me about this whole thread (and I might have missed a part of it by skimming), is if we adopt semantic versioning, then what do those versions mean?

What makes for a major version change in Go?

What makes for a minor version change?

It seems obvious to say a major version is any backwards incompatible change, but what does that mean, exactly?

- Has anyone built tooling to detect backwards incompatible changes, and report on them?

- Is there a spec for what it means?

- This (http://blog.merovius.de/2015/07/29/backwards-compatibility-in-go.html) post indicates that surprisingly little is backwards compatible. To me, it sounds a little strict - if only because implementations should be able to add new fields to structures. There are interesting use-cases, such as when a structure is initialized via a "New" method - which hinder composition, but also means that adding a public field should be safe.

The reddit discussion (https://www.reddit.com/r/golang/comments/3f1zi4/in_go_almost_no_api_changes_are_backwards/) has a response suggestion which is kind of interesting "define a backwards-compatible API-change roughly as 'no package that is referenced on godoc.org will stop compiling after this change'" - a very utilitarian solution. Although I don't think it is strict enough - perhaps all tests should also pass.

The golang compatibility guarantee (https://golang.org/doc/go1compat) suggests that the Go authors can make breaking changes in certain circumstances. In a world of semantic versioning, we can welcome those changes as a new major version - so the baseline compatibility guarantee doesn't seem to apply to packages outside the core language, once we start applying semantic versions.

Eric.

[Chris Hines]

You raise excellent points. Here are some additional references:

Gustavo Niemeyer does a good job documenting reasons for major version changes in the docs for gopkg.in.

The Go team uses a tool to write all of the exported APIs in the standard library to a file in a canonical format that is then used to check for breaking changes relative to previous releases. The code for that tool is here: https://github.com/golang/go/tree/master/src/cmd/api. The output for each of the Go releases are stored here: https://github.com/golang/go/tree/master/api.

I haven't studied the code much, but I imagine it would be a good starting point for tooling to help make sure versions are following the rules, at least as far as can be determined from the code. There is an issue related to using the api tool on external packages: https://github.com/golang/go/issues/4993

A related go-nuts thread: https://groups.google.com/d/topic/golang-nuts/5RbkvIlg5Tc/discussion

I'm sure there is more out there, but that's what I am aware of.

Chris

[Dave Cheney]

On Tue, Aug 18, 2015 at 8:55 AM, Eric Johnson <er...@tibco.com> wrote:
> What concerns me about this whole thread (and I might have missed a part of
> it by skimming), is if we adopt semantic versioning, then what do those
> versions mean?

You make a fair point, but I think pragmatism has to rule the day. If
we did not recommend semver, and instead recommended something else
that in practice would look 80% like semver (we may have slightly
different recommendations on what is considered a forward compatible
change for instance, but 1.x to 2.x will always be considered a major
upgrade to most observers), what would be the benefit ?

This proposal is basically about getting people to start releasing
their software by tagging their git repos, that's all. Other things
can be built upon it, like a syntax for version ranges and central
repositories and all that good stuff, but none of it can happen unless
people start to release their software.

To the specifics of people not following semver, or misinterpreting
its neuances, or whatever, I'm not that concerned, The market will
deal with projects that cannot do reliable releases, and I don't think
this takes away from the value of having a release process.

Thanks

Dave

[N. Jones]

Chris:

I'm really interested in the idea of standardizing a tool for Go around this idea. However I think this is out of scope for Dave's original proposal. Would you be interested in starting a new one around this, with the output being the version number and format that comes out of this proposal?

Nika

--

[Chris Hines]

On Saturday, August 15, 2015 at 7:24:31 PM UTC-4, Dave Cheney wrote:

On Sun, Aug 16, 2015 at 4:44 AM,  <erik...@gmail.com> wrote:
> I would agree that the full name in the tag wouldn't necessarily be a great idea... and semver itself is something I'm fine with (although versioning might relate in some ways to the pkg grouping mechanism so I think progress is needed there, more below).
>
> In your example you show 'tag: net-1.5.0' which still has "part" of the name in the tag name (in this case "net").  I might go so far as to think just "1.5.0" would be better (or v1.5.0 if you need a prefix).

I can see a consensus forming for no prefix, so a vestigial "v" would seem odd.

An argument for allowing a "v" prefix is that existing repos that have followed the guidelines for gopkg.in will likely have tags starting with "v". Since the gopkg.in versioning rules are highly compatible with the discussion here, it would seem a shame not to support existing tags in such repositories.

Chris

[Edward Muller]

My 2c.

Semver tags (with or without a "v" prefix) at the repo level.

If you mix packages in a repo then they are released as a unit with the tag.

Anything else becomes complex and violates a lot of assumptions. I believe I understand the counter arguments though.

FWIW: semver and the like is a tool to guide people wrt assumptions that the author has made about compatibility, nothing more. I've seen plenty of breakage (even caused it myself) over the last 20+ years.

--

You received this message because you are subscribed to the Google Groups "Go Package Management" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-package-manag...@googlegroups.com.
To post to this group, send email to go-package...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Edward Muller
@freeformz

[Dave Cheney]

Thanks Edward,

Do you have any feedback on the idea of putting the import prefix (the
path to the topmost package in the repo) in the tag name ? ie
"golang.org/x/crypto-1.5.0" ?

Thanks

Dave

[Tom Lanyon]

On 19 August 2015 at 09:42, Dave Cheney <da...@cheney.net> wrote:

Do you have any feedback on the idea of putting the import prefix (the
path to the topmost package in the repo) in the tag name ? ie
"golang.org/x/crypto-1.5.0" ?

Doing so suggests to me that you're versioning the individual package(s) that live within a repo, which conflicts with the intention to version the whole repo as one entity.

Tom

[Dave Cheney]

golang.org/x/crypto is the authorative import prefix [1] for this repository

% curl https://golang.org/x/crypto/ssh?go-get=1 2>/dev/null | grep go-import
<meta name="go-import" content="golang.org/x/crypto git
https://go.googlesource.com/crypto">

The first element in the content attribute is the import prefix,
that's what we're using as the identifier in the tag. The source might
live elsewhere, in this case https://go.googlesource.com/crypto.

Thanks

Dave

1. http://golang.org/cmd/go/#hdr-Remote_import_paths

[Daniel Theophanes]

Moving repos to a new location would be a bigger pain if you include the whole path. If the associate between revisions and versions was in a text file it might be less of an issue, but if someone uses branches or tags, that might be a larger issue to "change" history to reflect where it currently lives.

Thus in my example to use package trees, I only included the path to the root of the package tree, from the root of the repo.

-Daniel

--
You received this message because you are subscribed to a topic in the Google Groups "Go Package Management" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/go-package-management/B-MHT98dvEU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to go-package-manag...@googlegroups.com.

[Dave Cheney]

I look at it like this

a. if you move the repository your project is in, then it's name has
changed, bitbucket.org/foo/bar -> github.com/foo/bar. This is a new
project, and there is no way that consumers of your project can deal
with this automatically. There is also no harm it putting
"bitbucket.org/foo/bar-1.0.0" in the release tag - you can either
choose to restart the sequence when you migrate to github or continue
the same sequence, "github.com/foo/bar-2.0.0" , it doesn't make any
difference.

-or-

b, if you are using a vanity import url, then that gives extra weight
to including the import prefix in your release tag, the sequence is
uninterrupted because the name of the import prefix has not changed,
which we saw with the move off google code via the golang.org/x/
redirects.

The summary from this thread that I'm seeing is

1. a bare number 1.0.2 is less desirable, even to the point of
optional prefixes to make it look less like a number.
2. if there is an optional prefix, let's commit and make it a mandatory prefix
3. if the prefix is mandatory, then I say let's put something useful
in it and use the import prefix for that project.

Thanks

Dave

[Chris Hines]

I don't understand the value of adding a portion of the project name to tags inside the project repo. In my mind tags are within the scope of the repo. Why repeat a portion of the scope name inside that scope? Version identifiers must always be used in tandem with a project identifier, and if they some pieces in common the result will have the stutter Go tried hard to eliminate.

When I think about tooling that will look at a list of tags within a repo and interpret them as version identifiers, I would rather have a simple prefix that is the same in all repos in part because it makes the pattern matching simpler, both for machines and people. I think we want the tags to match a pattern because I don't think we want to dictate that project owners cannot create other tags to identify revisions for purposes other than release versioning.

If we do require a prefix, my instinct suggests either the basic and terse "v" that has been suggested, or something a bit more descriptive such as "release-", which seems much more tailored to the semantics of the tag.

Chris

>> go-package-management+unsub...@googlegroups.com.

[erik...@gmail.com]

On Tuesday, August 18, 2015 at 5:49:56 PM UTC-7, Dave Cheney wrote:
> I look at it like this
>
> a. if you move the repository your project is in, then it's name has
> changed, bitbucket.org/foo/bar -> github.com/foo/bar. This is a new
> project, and there is no way that consumers of your project can deal
> with this automatically. There is also no harm it putting
> "bitbucket.org/foo/bar-1.0.0" in the release tag - you can either
> choose to restart the sequence when you migrate to github or continue
> the same sequence, "github.com/foo/bar-2.0.0" , it doesn't make any
> difference.
>
> -or-
>
> b, if you are using a vanity import url, then that gives extra weight
> to including the import prefix in your release tag, the sequence is
> uninterrupted because the name of the import prefix has not changed,
> which we saw with the move off google code via the golang.org/x/
> redirects.
>
> The summary from this thread that I'm seeing is
>
> 1. a bare number 1.0.2 is less desirable, even to the point of
> optional prefixes to make it look less like a number.
> 2. if there is an optional prefix, let's commit and make it a mandatory prefix
> 3. if the prefix is mandatory, then I say let's put something useful
> in it and use the import prefix for that project.
>
> Thanks
> Dave

I might not be on the rest of the thread (?)... but that direction isn't clear
to me as the preferred direction. I know it is what you want and I respect
that as you have a great deal of experience (aside: gb is very cool!)... but
I'm not sold on this.

If I had to state the main alternative based on the thread it sounds like:

1. a bare semver is fine with an optional 'v' or 'V' prefix allowed so as to
handle various VCS's (or governance or existing patterns such as gopkg.in)

If we have a bare number (with optional prefix) then any future manifest (if one comes) would not be redundant (in *most* cases, and in cases where not redundant it could be confusing like the golang.org example you mentioned), I would be concerned about verbosity like this:

github.com/joe/what...@github.com/joe/whatever-2.2.0
github.com/dave/pr...@github.com/dave/projx-2.2.0

This vs:

github.com/joe/what...@2.2.0
github.com/dave/pr...@2.2.0

I prefer the latter. If I rename it I prefer continuing to use the basic semver and moving forward which that would help with consistency for the project (before and after the rename)... which I view as a plus as that is important.

Another plus (perhaps not according to all) would be that if that is the canonical repo... the "first" or master then anyone forking it and trying to take over the project without permission (bad idea/etc) would be more "hard pressed" to keep using the same numbering (i.e.: if they continue to "borrow" from the main repo they would start to have tag conflicts and such... this would instead force them to truly "split" and start from 0 again and try to "sell" their fork on their own, perhaps dropping history/etc and pushing them to stop borrowing/etc). Maybe I'm wrong there (more on this thought below).

Anyhow, if the repo is the canonical one and is renamed I think using an older version in my product going forward might start to look confusing as well, i.e.: I have to use the new repo location and the old tags:

github.com/joe/what...@bitbucket.org/joe/whatever-1.0.2

vs just having:

github.com/joe/what...@1.0.2

Here it's just the rev I want, period. It conveys all semver can convey in this packages lifecycle, etc. I would say that if you're moving the package that would be preferred... that consistency and continuous versioning going forward, no?

Anyhow, regardless of what is chosen it would be good for the recommendation to cover "best practices" when it comes to renaming the package/repo and how that should impact the tags (as well as what happens when forking it and releasing from the fork).

Aside: for the forks you could have recommendations under "Go" semver extension that
maybe requires a prefix or postfix to help indicate that this is NOT the canonical
repo (e.g.: if the tag is 2.0.2-joe then it's clear we're not the canonical repo)...
yeah, might be a bad example but prefix/postfix could perhaps help to identify
the difference between the core/canonical repo (?) Just an offhand thought

Anyhow... food for thought. Cheers folks...

Erik

[Dave Cheney]

On Wed, Aug 19, 2015 at 11:24 AM, Chris Hines <chris....@gmail.com> wrote:
> I don't understand the value of adding a portion of the project name to tags
> inside the project repo. In my mind tags are within the scope of the repo.
> Why repeat a portion of the scope name inside that scope? Version
> identifiers must always be used in tandem with a project identifier, and if
> they some pieces in common the result will have the stutter Go tried hard to
> eliminate.

> When I think about tooling that will look at a list of tags within a repo
> and interpret them as version identifiers, I would rather have a simple
> prefix that is the same in all repos in part because it makes the pattern
> matching simpler, both for machines and people. I think we want the tags to
> match a pattern because I don't think we want to dictate that project owners
> cannot create other tags to identify revisions for purposes other than
> release versioning.

This is a good point, apart from the verbosity, a format which stole
the tag namespace from other purposes wouldn't get much support.

> If we do require a prefix, my instinct suggests either the basic and terse
> "v" that has been suggested, or something a bit more descriptive such as
> "release-", which seems much more tailored to the semantics of the tag.

That's fair. It sounds like everyone is _not_ ok with a bare number,
so some kind of prefix is in order, but similarly people are not ok
with the full import prefix.

Why not "v<semver>" ? For the same reason that Go programmers don't
name their interface types ISomething, or ReaderInterface. It's
redundant.

Why not "release-<semver>", because it's redundant, this is release
tag, and it might interferer with someone elses tagging scheme
(probably a lot probability), especially as 99% of Go projects don't
tag anything.

Why not "go-release-<semver>", lower probability of collision, but
still feels redundant, any system consuming these tags would just
strip it off as a needless prefix. I'd like to put as much useful
information in there as possible.

What about the following

<toplevel package>-<semver>

Where toplevel package is the final component of the import prefix, so

https://go.googlecode.com/crypto is tagged crypto-1.5.0

and

https://github.com/juju/loggo is tagged loggo-1.24.2

Thanks

Dave

>> >> go-package-manag...@googlegroups.com.

>> >> To post to this group, send email to
>> >> go-package...@googlegroups.com.
>> >> For more options, visit https://groups.google.com/d/optout.
>

> --
> You received this message because you are subscribed to the Google Groups
> "Go Package Management" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-package-manag...@googlegroups.com.
>

[Tom Lanyon]

On 19 August 2015 at 10:03, Dave Cheney <da...@cheney.net> wrote:

golang.org/x/crypto is the authorative import prefix [1] for this repository

 % curl https://golang.org/x/crypto/ssh?go-get=1 2>/dev/null | grep go-import
<meta name="go-import" content="golang.org/x/crypto git
https://go.googlesource.com/crypto">

The first element in the content attribute is the import prefix,
that's what we're using as the identifier in the tag. The source might
live elsewhere, in this case https://go.googlesource.com/crypto.

Right, sorry - I read your "path to the topmost package in the repo" and missed the bit about import prefix.

Regardless, I echo most of Erik's points although disagree about any prefix being optional.  If it's allowed, it should be mandatory.

I'd prefer to see "v<semver>" to keep it short and easily programatically parseable across differently named Go projects.

Tom

[Dave Cheney]

No, you are right to call me on this. I asked for this group's
feedback and I am not respecting that.

> If I had to state the main alternative based on the thread it sounds like:
>
> 1. a bare semver is fine with an optional 'v' or 'V' prefix allowed so as to
> handle various VCS's (or governance or existing patterns such as gopkg.in)

I feel very adverse to anything optional. I has seen the basket case
that is the python egg/wheel/pip numbering scheme come undone because
early implementations were too permissive. The v1 in gopkg.in/check.v1
comes directly from the branch name, v1, it's not optional.

> If we have a bare number (with optional prefix) then any future manifest (if one comes) would not be redundant (in *most* cases, and in cases where not redundant it could be confusing like the golang.org example you mentioned), I would be concerned about verbosity like this:
>
> github.com/joe/what...@github.com/joe/whatever-2.2.0
> github.com/dave/pr...@github.com/dave/projx-2.2.0
>
> This vs:
>
> github.com/joe/what...@2.2.0
> github.com/dave/pr...@2.2.0

That's a compelling argument. But you were also arguing for an "v"
prefix, that would have to be stripped out anyway, so I don't think
this argument is a strong one.

The format of the tag that I proposed above was
<importprefix>-<semver> only the <semver> part would be of interest to
an as yet to be defined dependencies.txt style file. But this is
putting the cart before the horse, if there is no method of releasing
Go projects, then there can be no dependencies.txt file to declare the
released versions that your project depends on.

> I prefer the latter. If I rename it I prefer continuing to use the basic semver and moving forward which that would help with consistency for the project (before and after the rename)... which I view as a plus as that is important.
>
> Another plus (perhaps not according to all) would be that if that is the canonical repo... the "first" or master then anyone forking it and trying to take over the project without permission (bad idea/etc) would be more "hard pressed" to keep using the same numbering (i.e.: if they continue to "borrow" from the main repo they would start to have tag conflicts and such... this would instead force them to truly "split" and start from 0 again and try to "sell" their fork on their own, perhaps dropping history/etc and pushing them to stop borrowing/etc). Maybe I'm wrong there (more on this thought below).

This is a very interesting point, but I think it goes both ways. If
someone forks github.com/pkg/sftp to say, github.com/badactor/sftp
what precautions does a tagging scheme offer to prevent confusion?

In your scheme, the "1.2.2" tag may already exist in the original
repo, so could not be reused without rewriting the repo history.
In my scheme, the "github.com/pkg/sftp-1.2.2" (or any number) tag
would not match the import prefix of github.com/badactor/sftp so the
tag would be ignored.

> Anyhow, if the repo is the canonical one and is renamed I think using an older version in my product going forward might start to look confusing as well, i.e.: I have to use the new repo location and the old tags:
>
> github.com/joe/what...@bitbucket.org/joe/whatever-1.0.2
>
> vs just having:
>
> github.com/joe/what...@1.0.2
>
> Here it's just the rev I want, period. It conveys all semver can convey in this packages lifecycle, etc. I would say that if you're moving the package that would be preferred... that consistency and continuous versioning going forward, no?

In the example you provided, the project's name is

bitbucket.org/joe/whatever

if they rename the repo, then it's a new project as it has a new name.
There may be a discontinuity between the numbering,
bitbucket.org/joe/whatever only goes up to version 1.0.2, and
github.com/joe/whatever starts at version 1.1. I don't see that as a
problem.

> Anyhow, regardless of what is chosen it would be good for the recommendation to cover "best practices" when it comes to renaming the package/repo and how that should impact the tags (as well as what happens when forking it and releasing from the fork).

I think this is a crucial point. Renaming a repo would leave, in my
scheme, oddly named tags in the repo, which by the rules of git cannot
be removed. New tags could be added to match the new repo's import
prefix, but I'm sure that would tic off people who care a lot about
their repo.

It should be noted that renaming is very bad because it would break
people trying to pull old versions of project.

>
> Aside: for the forks you could have recommendations under "Go" semver extension that
> maybe requires a prefix or postfix to help indicate that this is NOT the canonical
> repo (e.g.: if the tag is 2.0.2-joe then it's clear we're not the canonical repo)...
> yeah, might be a bad example but prefix/postfix could perhaps help to identify
> the difference between the core/canonical repo (?) Just an offhand thought

I think selling semver is going to be hard enough. Selling an
extension is too much of a stretch. Sorry

Thanks again for your continued feedback.

Dave

> You received this message because you are subscribed to the Google Groups "Go Package Management" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to go-package-manag...@googlegroups.com.

[Dave Cheney]

On Wed, Aug 19, 2015 at 12:05 PM, Tom Lanyon <t...@oneshoeco.com> wrote:
> On 19 August 2015 at 10:03, Dave Cheney <da...@cheney.net> wrote:
>>
>> golang.org/x/crypto is the authorative import prefix [1] for this
>> repository
>>
>> % curl https://golang.org/x/crypto/ssh?go-get=1 2>/dev/null | grep
>> go-import
>> <meta name="go-import" content="golang.org/x/crypto git
>> https://go.googlesource.com/crypto">
>>
>> The first element in the content attribute is the import prefix,
>> that's what we're using as the identifier in the tag. The source might
>> live elsewhere, in this case https://go.googlesource.com/crypto.
>
>
> Right, sorry - I read your "path to the topmost package in the repo" and
> missed the bit about import prefix.

This was probably my fault. At one point earlier in the discussion I
gave an example of "webdav-x.y.z" for golang.org/x/net/webdav. I
should have called that "net-x.y.z". This was confusing, sorry.

> Regardless, I echo most of Erik's points although disagree about any prefix
> being optional. If it's allowed, it should be mandatory.
>
> I'd prefer to see "v<semver>" to keep it short and easily programatically
> parseable across differently named Go projects.

I agree that any prefix should be manditory. My objection to v is it
is like like the I on .net interface types, superfluous.

If there is to be a prefix (and that is the message I see here) then I
want it to serve some purpose and transfer some information to justify
having to look at it all the time.

[Tom Lanyon]

On 19 August 2015 at 11:45, Dave Cheney <da...@cheney.net> wrote:

I agree that any prefix should be manditory. My objection to v is it
is like like the I on .net interface types, superfluous.

If there is to be a prefix (and that is the message I see here) then I
want it to serve some purpose and transfer some information to justify
having to look at it all the time.

A "v" prefix is only superfluous in the sense that it indicates "version" and that we already know we're working with version identifiers.

Ignoring the "version" meaning, it's not a superfluous prefix in the sense that it also provides a small, standard alphabetical prefix to avoid any aforementioned issues with bare numerics and a simple way to pattern match version tags in your VCS versus other tags.  It's a short prefix and is not repetitive, avoiding the issues Erik rightly pointed out about their use in future dependency definition files (e.g. "github.com/joe/what...@github.com/joe/whatever-2.2.0").

I don't see the import prefix in a version tag providing any real benefit and, conversely, can see reasons it would be confusing or annoying (e.g. renaming a project/repo and wanting version numbers to match).

If there's opposition to something small and generic like a "v" or "rel(ease)" prefix, I'd much rather see bare semver without a prefix than something which matches the current project/package/import prefix.

Tom

[Dave Cheney]

Thanks for your feedback Tom.

What are the benefits of the "v" prefix other than being short ? I can
understand the pushback to including the full import prefix in the
tag, and consider that idea dead and buried, but if "v" is a
placeholder to filter out release tags for Go projects from other
tags, why not make it longer and more explicit, ie

go-version-<semver>

I would not imagine that "v" or "go-version-" would be part of any
dependencies.txt file as neither of those prefixes make any sense when
you want to describe a range of versions.

Thanks

Dave

[Tom Lanyon]

On 19 August 2015 at 15:04, Dave Cheney <da...@cheney.net> wrote:

What are the benefits of the "v" prefix other than being short ? I can
understand the pushback to including the full import prefix in the
tag, and consider that idea dead and buried, but if "v" is a
placeholder to filter out release tags for Go projects from other
tags, why not make it longer and more explicit, ie

    go-version-<semver>

I guess the feeling I had from reading this thread was:

  (a) it can't be a bare version string, so we need a prefix; and

  (b) it doesn't *really* matter what the prefix is.

In which case, why not make it the shortest possible, whilst still being relevant.

 

I would not imagine that "v" or "go-version-" would be part of any
dependencies.txt file as neither of those prefixes make any sense when
you want to describe a range of versions.

Surely that depends on how you implement those representations? If the prefix is 'go-version-' (or something similarly long), I would not want to continually type that into my dependencies.txt so would be looking for help from the tooling to do some magic.  However, if the prefix is just 'v' (or similar), it's much less of an inconvenience to repeatedly type and so potentially needs less complicated and less magic tools.  It would be perfectly reasonable to just have the full 'v<semver>' tags in the manifest, e.g.

    github.com/foo/bar v1.5.x

    golang.org/x/crypto v1.0.0-v2.1.2 

Finally, I don't like the idea of making any release tags go-specific.  A project doesn't always contain only Go code.

Tom

[Dave Cheney]

Thanks for your comments. Can I ask you to not get distracted by the
format of any future dependencies.txt file, they can interpret the tag
in whatever way they want, and I hope would strip of any prefix and
concentrate only on the part of the tag that represented the <semver>
element.

Your point about repositories containing code that is not just Go code
is well made. But having said that, given the strict layout of Go
projects, in my experience I have not found many who have persisted in
trying to create a pologlot project. I'd like to hear the experiences
of the group on this.

Thanks

Dave

[erik...@gmail.com]

I appreciate the open discussion Dave, wherever it goes. All good.

> > If I had to state the main alternative based on the thread it sounds like:
> >
> > 1. a bare semver is fine with an optional 'v' or 'V' prefix allowed so as to
> > handle various VCS's (or governance or existing patterns such as gopkg.in)
>
> I feel very adverse to anything optional. I has seen the basket case
> that is the python egg/wheel/pip numbering scheme come undone because
> early implementations were too permissive. The v1 in gopkg.in/check.v1
> comes directly from the branch name, v1, it's not optional.

I understand your opinion and hear you... it sounds like you are not alone
there as well (although some others don't seem to have a problem with it).

My preference, to be clear, is frankly to have no 'v' or 'V' or any other prefix
*unless* it is needed for one or more of the following reasons:
a) conveys useful information
b) needed ("enough") for "real world" use

I agree with you Dave that it does not convey (a), the question is if it does (b)
in terms of perhaps allowing wider use across VCS's with perhaps more
governance rules in play and such. Maybe we start with *no* prefix in rev 1
of the recommendation. Later, if folks come with pitchforks (or more real
world examples than have been mustered here come up) then rev 2 of the
recommendation could then revisit said prefix. Hopefully there will be
no need for rev 2.

> > If we have a bare number (with optional prefix) then any future manifest (if one comes) would not be redundant (in *most* cases, and in cases where not redundant it could be confusing like the golang.org example you mentioned), I would be concerned about verbosity like this:
> >
> > github.com/joe/what...@github.com/joe/whatever-2.2.0
> > github.com/dave/pr...@github.com/dave/projx-2.2.0
> >
> > This vs:
> >
> > github.com/joe/what...@2.2.0
> > github.com/dave/pr...@2.2.0
>
> That's a compelling argument. But you were also arguing for an "v"
> prefix, that would have to be stripped out anyway, so I don't think
> this argument is a strong one.

As above, I wasn't desiring the 'v' prefix, only indicate it might be needed
to handle rarer cases where there was no choice (hence the original "optional"
suggestion).

However, as above, I can live without it in rev 1 (assuming it's agreed upon
and others don't come up with perhaps more pressing examples than I and
others have mentioned).

> The format of the tag that I proposed above was
> <importprefix>-<semver> only the <semver> part would be of interest to
> an as yet to be defined dependencies.txt style file. But this is
> putting the cart before the horse, if there is no method of releasing
> Go projects, then there can be no dependencies.txt file to declare the
> released versions that your project depends on.

Thanks for the clarification. I think though if the dependencies.txt or
whatever it's called just has 1.2.3 and we need to find the "true" tag name
that the process for doing that might add unneeded complexity (especially
if the repo needed to be moved from one service to another and had
various tag names across that move)... for little or no benefit that I
can see (this is assuming your <importprefix>-<semver> tag name).

For "fixed" prefixes that is not the case as we can do a direct mapping
always. Same with no prefixes, no problem.

> > I prefer the latter. If I rename it I prefer continuing to use the basic semver and moving forward which that would help with consistency for the project (before and after the rename)... which I view as a plus as that is important.
> >
> > Another plus (perhaps not according to all) would be that if that is the canonical repo... the "first" or master then anyone forking it and trying to take over the project without permission (bad idea/etc) would be more "hard pressed" to keep using the same numbering (i.e.: if they continue to "borrow" from the main repo they would start to have tag conflicts and such... this would instead force them to truly "split" and start from 0 again and try to "sell" their fork on their own, perhaps dropping history/etc and pushing them to stop borrowing/etc). Maybe I'm wrong there (more on this thought below).
>
> This is a very interesting point, but I think it goes both ways.

Agreed on the "both ways"... it was a half-baked (or less) thought.

> If someone forks github.com/pkg/sftp to say, github.com/badactor/sftp
> what precautions does a tagging scheme offer to prevent confusion?

This was only briefly thought out point on my part ... but the basic thought
I had was that if I'm the good guy and I have 1.2.2 and badactor forks
me at that point. Now I keep working as the good guy and in my good
repo I add in 1.2.3 and 1.2.4 and 1.3.0, 1.3.1, 2.0.0, 2.0.1, etc... if the
badactor repo is still taking changes from my repo (I don't want his)
then he'll be getting those newer tags. If he's already used his own
tags after the last fork point (1.2.2) and he has his own 1.2.3 and 1.2.4
and such then he'll have some "dual tags" and such to have to consider
or just ignore and leave around (might be messier in some VCS's vs.
others, git would be fine with origin/<goodguytag> and the local clone
tags in the local clones namespace and such I believe). Regardless,
those remote tags and having them point at different commits than the
local tags would flag the repo as a fork perhaps (good or bad fork would
be up in the air though). Other impacts/issues would depend upon VCS/etc
but, upon reflection, no big deal I'd guess.

Now, if he's good actor that forked it and add some cool functionality that the
main repo owner just doesn't want (although he's fine with the fork existing
in case others want it... the core maintain is fine with them using goodactor's
fork). In this case the good actor will still want updates from the canonical repo
and he'll want to release his own versions... perhaps in such a way as to indicate
he's not the canonical version but that he's got his own versions to release.
Same boat as the bad actor above... but with git, as above, no big deal,
He can use raw semvers on his local clone with git, with SVN he's not
likely to be merging but diff/patching so no version # conflicts to worry about
so still "ok" to use semvers... and with hg, not sure how it would work.

In both cases any of our proposed semver tag formats wouldn't really help
in either direction (although I could be missing something of course).

Anyhow, this does jingle a little bell back about prefixes used differently perhaps
(maybe being of some "useful information" use, maybe not). Mostly just thinking
out loud.

This could (we're getting into less thought out territory again, need to ponder
a bit) be a reason for a prefix... one that conveys information about if a repo is
the canonical/main repo or not. Eg: maybe using v1.2.3 is only recommended
for use by the true owner of a repo. Maybe forks or derivations would use
something else for their versions for clarity (f1.2.3). With that I know just by
the version # tags if I'm using "the original stuff" or not. Nothing prevents the
bad actor of course from trying to fake things out but the community could
call him out and black list the repo or something if he won't rename tags.
Dreaming? Ok, ok... probably... but just food for thought that perhaps
could be "run with" in a different way by others. Yeah, I'm waving my
hands so fast I might take off (wave hands here). Don't consider this one
at this point... just tossing out a few bits of pondering.

I think it'll come back to recommendations on tag names for forks (be
they good or bad) or "local enterprise copies" as the case may be.

> In your scheme, the "1.2.2" tag may already exist in the original
> repo, so could not be reused without rewriting the repo history.
> In my scheme, the "github.com/pkg/sftp-1.2.2" (or any number) tag
> would not match the import prefix of github.com/badactor/sftp so the
> tag would be ignored.

With git it wouldn't matter too much as it's just a ref from that upstream
remote clone (whatever I named it) and mine would still be fine (I think)...
so really all good In the case of a longer tag name with the import
name included in the tag... I would know they are different but I wouldn't
really know which was the "good" one... so no real information benefit
I can see.

These thoughts also brings me back to what you are looking to solve near-term?
Lets say we get semver on VCS's. You're calling them "projects" but I'd probably
just call them VCS's as I don't think we can move to a different term until there
is a way to identify semver's all the way through my "project/codebase" for all
dependencies (both direct and those indirectly brought in by pkg's I use). I
think the "project"/"product"/"codebase" terminology can only come into play
when we have full reproducibility across any artifact(s) being identified (in that
"codebase"/"product"/"project"). If they are in various git repo's that will
require a manifest "somewhere". Exception: if each "project"/"product" is
in a single repo (with all dependencies vendored within that repo, maybe
using the Go 1.5 vendor experiment stuff or something, perhaps we can
avoid it but that gets ugly if I want to override one dependent pkg and share
my modified version across all my dependencies, direct and indirect, that
use that package).

For separate repo's/clones and no code copying/vendoring everything
then, again, it seems that a dependency list file will be needed at some
point (or some other mechanism to cover all those transitive dependencies).

I guess I agree that having a semver all the way down, for all VCS's, will at
least be able to clue folks in to when a pkg they may depend upon may be
more likely to break things (if they are monitoring/polling for major # updates
for example)... and some tooling could be generated around things like that.

I guess if you're looking to "sell" this as a recommended standard we would
need a list of near term benefits for semver versions on VCS's and then longer
term vision of where that might lead down the road as well. Thoughts on how
the proposal might be worded with respect to this?

> > Anyhow, if the repo is the canonical one and is renamed I think using an older version in my product going forward might start to look confusing as well, i.e.: I have to use the new repo location and the old tags:
> >
> > github.com/joe/what...@bitbucket.org/joe/whatever-1.0.2
> >
> > vs just having:
> >
> > github.com/joe/what...@1.0.2
> >
> > Here it's just the rev I want, period. It conveys all semver can convey in this packages lifecycle, etc. I would say that if you're moving the package that would be preferred... that consistency and continuous versioning going forward, no?
>
> In the example you provided, the project's name is
>
> bitbucket.org/joe/whatever

It was, yes, Then that provider went away (perhaps) and it was moved to github.
It can't be found on bitbucket.org now. The clone that was moved has all the
old tags of course (from the old name)... and newer tags (if following your full
name mechanism) would have the newer name. My product, in this example,
is using an older tag (with the old name) in the now moved repo (that is the
basic scenario I was looking at with the bitbucket example I gave).

> if they rename the repo, then it's a new project as it has a new name.

I don't know that I buy that. It is the same clone content, the same deliverables...
the same code. It has all the old versions. I've just shifted hosting services.
I want to keep using my version numbers and just move on (well, at least
I would want to in that situation).

> There may be a discontinuity between the numbering,
> bitbucket.org/joe/whatever only goes up to version 1.0.2, and
> github.com/joe/whatever starts at version 1.1. I don't see that as a
> problem.

I guess I do see it as an issue as then we might see the example in a
dependency manifest somewhere:

github.com/joe/what...@bitbucket.org/joe/whatever-1.0.2

Which kind of scares me. Either that or the person needs to re-tag all old
versions after the move so it can become this:

github.com/joe/what...@github.com/joe/whatever-1.0.2

Which I think is still too verbose (and folks won't like retagging either I don't think).

> > Anyhow, regardless of what is chosen it would be good for the recommendation to cover "best practices" when it comes to renaming the package/repo and how that should impact the tags (as well as what happens when forking it and releasing from the fork).
>
> I think this is a crucial point. Renaming a repo would leave, in my
> scheme, oddly named tags in the repo, which by the rules of git cannot
> be removed. New tags could be added to match the new repo's import
> prefix, but I'm sure that would tic off people who care a lot about
> their repo.

Exactly.

> It should be noted that renaming is very bad because it would break
> people trying to pull old versions of project.

Undesired, I would agree, but hosting services terms change or they go
away or whatever. It will happen and is normal I would say.

> > Aside: for the forks you could have recommendations under "Go" semver extension that
> > maybe requires a prefix or postfix to help indicate that this is NOT the canonical
> > repo (e.g.: if the tag is 2.0.2-joe then it's clear we're not the canonical repo)...
> > yeah, might be a bad example but prefix/postfix could perhaps help to identify
> > the difference between the core/canonical repo (?) Just an offhand thought
>
> I think selling semver is going to be hard enough. Selling an
> extension is too much of a stretch. Sorry

That was a totally offhand thought so no worries at all... didn't mean to convey
it as anything else. Just sort of beginning to "think out loud" about tags and
what other information might be conveyed by tagging convention perhaps
beyond semver (and if that might be wanted). I agree at this stage though
that a pure semver with no prefix is "the best we have". So far (except for
maybe rev 2 recommendations that might need a prefix if any pitchforks
come out as per the above).

I still believe a story around dependency lists (or other like mechanisms) will
need to come as we move beyond this.

> Thanks again for your continued feedback.

Sure... a pleasure. Appreciate your thoughts and time,

Erik

[Edward Muller]

Dave,

-1 on prefixing like that.

AFAICT, It makes them non portable later on (or maybe I'm implying too much on the implementation?). For instance, would need to re-tag everything if the project name / location changes? What if there are conflicting tags (2 different sources)?

FWIW: I re-read your original post and I'm not sure I really care about versions **that** much. I care about revisions and dvcs hashes. From the dvcs we can determine the relative position using dvcs meta data and differences between the copy of the package that I have and what's new / different. That's all that really matters. 

In the end these versioning schemes apply some abstract semantic meaning that obfuscates the underlying changes, leading you into a false sense of "yeah, that's safe to update" and "Oh! that's a major version change. I need to be careful / take more time / etc."

Understandably this is **exactly** why people want to move towards them.

So I'm still +1 if we want to say you should use semver based tags for versions and there's a tool that uses that data to tell you stuff. But I (eventually) hope to add some stuff to godep to answer some (maybe all?) of those questions at the repo level.

--

Edward Muller
@freeformz

[Edward Muller]

Also ...

"v" means version. Tags can, and are, used for things other than versions, although versions are the primary use I see.

--

Edward Muller
@freeformz

[Dave Cheney]

Thanks for your reply Edward,

So, to summarise, everyone appears to be ok with semver as a numbering
scheme; modulo questions of if people will abide by it's principals or
just randomly tag stuff - i'm happy to table those concerns, the
market will sort out those who cannot reliably version their software
from those who can.

To also summarise, while semver has broad support, ideas around any
kind of prefix before the semver component have been widely
contentious.

Last night in trying to gather some data to support the various
positions I did a survey of trending repos on github and found the
following

Of those that do have tags, the tags were in many forms

x.YYYYY vim
x.y.x swift projects
vx,y,z netflix/falcor (.js), facebook/relay, docker/docker,
kubernetes/kubernetes, mitchellh/vagrant, rails/rails (although their
pre release suffix was not compatible with semver, this may be a
bundler artifact), also many scala projects
x.y (sometimes with a suffix like -preview) microsoft/winObjc
vx.y spf13/hugo, other swift projects
projectname-x.y.z - apache maven, and projects that are published by maven

The tag formats were clearly delineated by language, assembly projects
did not use tags at all, C projects tended towards x.y and so forth.
Indeed amongst javascript and ruby projects the format is always
v<semver>, which I attribute to the influence of npm, even to the
extend that projects like robotjs, which is written in C, uses the npm
inspired vx.y.z format.

Now, keeping in mind that I did this survey, and on top of it being
completely unscientific, I also have a bias. But with that said, there
is a clear preference for tagging versions in the format

v<semver>

So this will be the format I recommend in my proposal.

Thanks

Dave

[Andrew Gerrand]

On 20 August 2015 at 03:31, Dave Cheney <da...@cheney.net> wrote:

So, to summarise, everyone appears to be ok with semver as a numbering
scheme; modulo questions of if people will abide by it's principals or
just randomly tag stuff - i'm happy to table those concerns, the
market will sort out those who cannot reliably version their software
from those who can.

If people don't abide by the principles of semver then what is the point of using it?

I've observed this in the node/npm community. I'm not convinced that semver solves the actual problem. Can we do better?

Andrew

[Dave Cheney]

On Thu, Aug 20, 2015 at 7:11 PM, Andrew Gerrand <a...@google.com> wrote:
>
> On 20 August 2015 at 03:31, Dave Cheney <da...@cheney.net> wrote:
>>
>> So, to summarise, everyone appears to be ok with semver as a numbering
>> scheme; modulo questions of if people will abide by it's principals or
>> just randomly tag stuff - i'm happy to table those concerns, the
>> market will sort out those who cannot reliably version their software
>> from those who can.
>
>
> If people don't abide by the principles of semver then what is the point of
> using it?

If authors don't use semver reliably, the market will choose other
competing package. I think that's a strong enough forcing function. It
worked for go fmt, it'll work here.

>
> I've observed this in the node/npm community. I'm not convinced that semver
> solves the actual problem. Can we do better?

I don't think so, and I feel there is more value in following the
general lead of others languages who have adopted semver.

>
> Andrew

[Andrew Gerrand]

OK, I look forward to discussing how this should fit into the workflow.

[Philippe Lafoucrière]

On Sunday, August 16, 2015 at 4:31:18 PM UTC-4, erik...@gmail.com wrote:

If you check this out (npm packaging system semver details): https://docs.npmjs.com/misc/semver

 

As someone working fulltime on versions management (gemnasium), I strongly recommend to follow the semver 2.0 convention.

The npm semver lib above is clearly the most complete, and neat in the dependencies eco-system.

PHP and Python follow their own rules, and believe me, it's really, really nasty sometimes :

https://github.com/composer/composer/blob/master/src/Composer/Package/Version/VersionParser.php

and the corresponding tests:

https://github.com/composer/composer/blob/master/tests/Composer/Test/Package/Version/VersionParserTest.php

ie: 

'parses branches' => array('1.x-dev', '1.9999999.9999999.9999999-dev'),

Semver 2.0 is the cleaner way to handle versions, believe me. We fighting daily to standardize its usage, so please _really_ consider it before starting anything.

Even rails doesn't follow the convention, the last version being 4.2.4.rc1:

https://rubygems.org/gems/rails/versions/4.2.4.rc1

(it should be 4.2.4-rc1 ...)

thanks

Philippe

[Dave Cheney]

Thank you to everyone who participated in this discussion. This
proposal is now submitted, you can find it here.

https://github.com/golang/go/issues/12302

Thanks

Dave

[Dave Cheney]

Not really I'm afraid, the compile has a fixed mechanism for resolving .a files, and changes to the compiler are out of scope.

Thanks

Dave

On Wed, 30 Sep 2015 23:17  <giacomo...@gmail.com> wrote:

Hi,
I saw this thread only now I hope I am not too late in the discussion.

I like this approach very much and I was thinking that a tool could compile and store the packages tagged with the version in the GOPATH:
 $GOPATH/golang.org/x/net/context.a // built with go build
 $GOPATH/golang.org/x/net/context-1.0.0.a // built with coolTool

In this way a tool can produce and store versioned packages/artifacts under the project build  directory.
This would solve the vendoring as well?

My two cents, I hope it makes sense.





On Monday, 17 August 2015 17:53:09 UTC+2, Daniel Theophanes  wrote:
> Hi Chris,
>
>
> Fair enough. Let me explore one possibility of several I've thought of. Let's take the "golang.org/x/net" example again and let's continue to use repo tags/branches to associate version numbers with revisions.
>
>
> Quick specification:
>  1. Versions package trees cannot be nested.
>  2. If repo != package tree root, specify releases by "<relative-path-to-root>-semver"
>  3. If repo == package tree root, specify releases by "semver"
>
>
> Quick Implementation:
>  1. Update repo (not files, "git fetch") in GOPATH.
>  2. Checkout specific tree version into $PROJECT/vendor/... folder. Only copy used packages.
>
>
> Example:
>     "golang.org/x/net/context" the tag would be "context-1.0.0"
>
>
> For tool implementations that already work with packages and not repos, it doesn't add any additional complexity. It would add one additional step for tools tools that currently only work with repos.
>
>
> I realize this isn't what other tool makers are used to implementing nor is it what they hold as an ideal. But to me it would appear to be a net gain.
>
>
> -Daniel
>
>
>
>
>
> On Sat, Aug 15, 2015 at 6:23 PM Chris Hines <chris....@gmail.com> wrote:
>
> Daniel,
>
>
> I am sympathetic to your line of thinking. I recently made the mistake of believing that gb vendoring worked at the package level until I was corrected by Dave. As Dave is fond of saying, currently [most] Go code isn't released. The golang.org/x/net package is no exception. Were it to be released, I believe the authors would be forced to re-evaluate the repository scope. But with the status quo, that is not a problem they face.
>
>
> Personally, I prefer to release my packages, or closely related sets of packages. I use Git tags to mark the releases in my repos, and that pushes me to create separate repos for separately released code bases.
>
>
> I would like to see more people take these issues more seriously, so I would love to see a solution that resolves that tension in an elegant way, I just don't know what that looks like at the moment. So if you have some ideas to make this situation better, I would love to see them.
>
>
>
> Chris
>
>
> On Saturday, August 15, 2015 at 9:02:46 PM UTC-4, Daniel Theophanes wrote:
>
>
> Dave and Chris,
>
>
> Under your proposed limits groups who use mono-repos and groups who use vcs where repo tends to be less granular cannot use such a versioning solution.
>
>
> Please also note that in vcs that lend themselves to smaller repos and where the project situation allows it I agree that the versioned package tree should be the same as a repo. I disagree that it would be more complicated, just different, in the 80% case. I would prefer a design that can still handle the golang/x/net repo case.
>
>
> If you haven't explored possible implementation mechanisms for such a package tree based version, I would recommend we do so before rejecting it. Believe it or not, not everyone I care about uses a DVCS.
>
>
> ... I am aware that groups who use mono-repos may not rely on versions in the first place and I'm aware most people in open source Go development use git. My point about handling each package remains. I understand that the other tools made to date are mostly repo based. I'm not totally sure we can come up with a better solution then to use VCS tags/branges, but I'd like to try. ...
>
>
> -Daniel
>
>
>
>
>
>
> On Sat, Aug 15, 2015 at 5:28 PM Chris Hines <chris....@gmail.com> wrote:
> On Saturday, August 15, 2015 at 12:01:58 PM UTC-4, Daniel Theophanes wrote:
> I think using semver for declaring versions is fine.
>
>
> You've touched on a number of points including project specification to where version tags would be stored. I agree considering those points is required for a relevant discussion in this.
>
>
> I agree that for practical reasons versions will almost always be stored in a VCS. However I would want to version a package tree separately from the repo. For instance the following three package trees all live in the same repo, but will have very different versions:
>  * golang.org/x/net/context (v1.0.y forever)
>  * golang.org/x/net/publicsuffix (v1.x.y API unlikely to change, but data will continue to be updated)
>  * golang.org/x/net/websocket (v2.x.y API has already changed already I think, may again in the future)
>
>
> I know it is unlikely to happen, but I would argue that those three packages should be in separate repositories. I teach Git to a lot of people at my $job, most of them migrating from Subversion, and one thing that always comes up is the proper scope of a Git repository. My stock answer is that everything in the same repo should be released as a unit. The rationale is that Git tags are scoped to the repository.
>
>
> When you create a Git tag, you are tagging the entire source tree on the tagged commit. Git doesn't let you tag a subdirectory. Other VCS's differ, Subversion lets you create tags at any depth. Then again Subversion doesn't have first class tags, but rather a set of conventional operations that result in an artifact commonly considered a tag. Nonetheless, for the time being, Git seems to have won, so mostly we are bound by its rules.
>
>
> Chris
>
>
>
>
>
>
> --
>
> You received this message because you are subscribed to a topic in the Google Groups "Go Package Management" group.
>
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/go-package-management/B-MHT98dvEU/unsubscribe.
>
> To unsubscribe from this group and all its topics, send an email to go-package-manag...@googlegroups.com.

[giacomo...@gmail.com]

[manlio....@gmail.com]

Il giorno lunedì 24 agosto 2015 22:52:51 UTC+2, Dave Cheney ha scritto:
> Thank you to everyone who participated in this discussion. This
> proposal is now submitted, you can find it here.
>
> https://github.com/golang/go/issues/12302
>

The link https://ftp-master.debian.org/new/golang-github-odeke-em-command_0.0~git20150727.0.cf17ee2-1.html is broken.


Regards Manlio

[Matt Farina]

For anyone interested in SemVer, I wrote a package that works with them and does most of the same comparisons as you can get with NPM. You can see it at https://github.com/Masterminds/semver and I used it to add SemVer to Glide. It was intentionally separated from Glide for others to be able to easily use.

[Dave Cheney]

Thanks. I've updated the link once before. The irony on not being able
to link to a stable version of a Go package in the debian archive is
not lost on me.

>
>
> Regards Manlio