Changes in gopkg.in

[Gustavo Niemeyer]

After further conversations in the go-pm mailing list, the version
selector in gopkg.in URLs was constrained further to be the major
version only. With that said, repositories in GitHub can still use
richer versioning schemes in their tags and branches, and
gopkg.in will continue to automatically select the best match for
the provided selector.

In principle this should not affect anyone that was observed
adopting gopkg.in officially, according to the logs, as these are
already using the major version convention.

The relevant section from the documentation (http://gopkg.in) that
details the convention and provides reasoning for this approach
is copied below:


== Version number ==

The number used in the gopkg.in URL looks like "v1" or "v42", and
represents the major version for the Go package. No incompatible
changes should be done to the package without also changing that
version, so that packages and applications that import the package
can continue to work over time without being affected by broken
dependencies.

When using branches or tags to version the GitHub repository, gopkg.in
understands that a selector in the URL such as "v1" may be satisfied
by a tag or branch "v1.2" or "v1.2.1" (vMAJOR[.MINOR[.PATCH]]) in the
repository, and will select an exact match, if available, or otherwise
the highest version satisfying the requested selector.

Even when using richer versioning schemes, the version selector used
in the gopkg.in URL is restricted to the major version number to
ensure that all packages in a dependency tree that depend on a given
versioned URL will pick the same package version whenever possible,
rather than importing slightly different versions of that package
(v1.0.1 and v1.0.2, for example). The only situation when such
duplicated imports may occur is when the package versions are
incompatible (v1 and v3, for example), in which case allowing the
independent imports may provide a chance for the software to still
work.

For clarity, assuming a repository containing the following tags or
branches:

v1.0
v2.0.3
v2.1.2
v3
v3.0.1

The following selectors would be resolved as indicated:

v1 => v1.0
v2 => v2.1.2
v3 => v3 (exact match)



gustavo @ http://niemeyer.net

[matthe...@gmail.com]

+1 on this. It's simple and I think it will be quite effective.

[Nick Snyder]

What is the rationale for having a special case for exact matches?

The 3.0.1 branch is essentially inaccessible through gopkg.in simply because of the existance of the v3 branch?

From a branch naming perspective, aren't v3, v3.0, and v3.0.0 semantically equivalent?

As a library consumer, I would expect a v3 selector to always give me the latest branch out of v3, v3.x, v3.x.y

[Andrew Gerrand]

On 25 March 2014 17:42, Nick Snyder <nickd...@gmail.com> wrote:

As a library consumer, I would expect a v3 selector to always give me the latest  branch out of v3, v3.x, v3.x.y

Unless I'm misunderstanding, that's exactly what it does.

[Nate Finch]

I wish there was an option to *not* allow this auto-upgrade behavior.  It's actually not what I would expect by default.  I would expect that if I say v2, that I get v2 and no other.  Now, for some projects, getting "the most recent v2.x" is fine... but I think for a lot of projects, they'd prefer to be able to test new changes *before* they get automatically included in new builds.

My ideal behavior would be: v2 (without minor version specified) means "use the latest 2.x", and v2.3 (explicitly choosing a minor version) means "use this exact version and no other".  Yes the latter requires manually updating when you want to move to 2.4, but it lets me gate upstream changes on my own terms.  It's the more paranoid choice for people who really need their stuff not to break.

[Taru Karttunen]

On 25.03 03:18, Nate Finch wrote:
> I wish there was an option to *not* allow this auto-upgrade behavior. It's
> actually not what I would expect by default. I would expect that if I say
> v2, that I get v2 and no other. Now, for some projects, getting "the most
> recent v2.x" is fine... but I think for a lot of projects, they'd prefer to
> be able to test new changes *before* they get automatically included in new
> builds.
>
> My ideal behavior would be: v2 (without minor version specified) means "use
> the latest 2.x", and v2.3 (explicitly choosing a minor version) means "use
> this exact version and no other". Yes the latter requires manually
> updating when you want to move to 2.4, but it lets me gate upstream changes
> on my own terms. It's the more paranoid choice for people who really need
> their stuff not to break.

Nothing guarantees that the repo won't break with new commits and the
same version. If one needs guarantees then vendoring is the right
approach.

- Taru Karttunen

[Tamás Michelberger]

Hi,

Preferring the exact match (v3 over v3.0.1) mean that as long as the v3 branch/tag exists there is no way reaching the upgrades on the that major version. Isn't that strange? I would expect it to always match the best possible version.

Tamás

--
You received this message because you are subscribed to the Google Groups "Go Package Management" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-package-manag...@googlegroups.com.
To post to this group, send email to go-package...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[kpo...@gmail.com]

On Tuesday, 25 March 2014 02:42:34 UTC-4, Nick Snyder wrote:

What is the rationale for having a special case for exact matches?

The 3.0.1 branch is essentially inaccessible through gopkg.in simply because of the existance of the v3 branch?

Agreed.  The "exact match" case seems to create an unexpected and inconsistent behaviour.  I see no reason for it.

[Joshua Poehls]

At first glance, yes, it is strange. But I think it would be even more strange to have a branch that you can't reach via gopkg.in.

Put another way, I think the intention is that if you (as package author) plan on using minor version numbers, you should simply not create a "v3" branch. Create "v3.0.0" instead. So long as you are consistent in naming your branches with the same version number format (major.minor.build, major.minor, or major), everything will work as expected.

--

Joshua Poehls

[Gustavo Niemeyer]

On Tue, Mar 25, 2014 at 3:42 AM, Nick Snyder <nickd...@gmail.com> wrote:
> What is the rationale for having a special case for exact matches?
>
> The 3.0.1 branch is essentially inaccessible through gopkg.in simply because of the existance of the v3 branch?

This behavior supports the use of branches. It's trivial to avoid it
by just not naming the the tag as v3. It may be v3.0 or v3-dev or
something else depending on your intended semantics. Without that, it
would be impossible to maintain a single branch for v3 as soon as
something was tagged under it.

With that said, it's indeed inconsistent, and perhaps surprising, and
the same argument can be made in the other way: to avoid that behavior
one might name the individual version tags differently.

I'll change it, thanks for the feedback.


gustavo @ http://niemeyer.net

[Gustavo Niemeyer]

There was a catch: if you name a branch as v3, it would just stick to
the exact match. Multiple people seem surprised by that, so let's take
it away.


gustavo @ http://niemeyer.net

[kpo...@gmail.com]

On Tuesday, 25 March 2014 06:18:30 UTC-4, Nate Finch wrote:

I wish there was an option to *not* allow this auto-upgrade behavior.

Having thought about this a bit more, I have to agree with Nate.

The approach in gopkg.in seems based on the concept of semantic versioning (http://semver.org/).  This is all well and good, but there are far from any guarantees that all package developers will adopt this versioning methodology.

If gopkg.in was created to address the limitation of go-get always retrieving a repo's HEAD, why limit gopkg.in to the Major of a branch/tag with forced auto-upgrading?  

[Gustavo Niemeyer]

On Tue, Mar 25, 2014 at 10:28 AM, Joshua Poehls <jos...@poehls.me> wrote:
> At first glance, yes, it is strange. But I think it would be even more
> strange to have a branch that you can't reach via gopkg.in.
>
> Put another way, I think the intention is that if you (as package author)
> plan on using minor version numbers, you should simply not create a "v3"
> branch. Create "v3.0.0" instead. So long as you are consistent in naming
> your branches with the same version number format (major.minor.build,
> major.minor, or major), everything will work as expected.

This is true, but what made me change my mind is that the same
argument can be made to the other side: to have gopkg.in sticking to a
branch v3, one can create tags v3.0.0-final, for example. Another
valid argument is that it's way less compelling to have such tags at
all if developers are just following whatever is in the branch.


gustavo @ http://niemeyer.net

[Gustavo Niemeyer]

The exception has been dropped.

Thanks for the feedback.

On Tue, Mar 25, 2014 at 3:42 AM, Nick Snyder <nickd...@gmail.com> wrote:

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.



--

gustavo @ http://niemeyer.net

[Gustavo Niemeyer]

On Tue, Mar 25, 2014 at 11:21 AM, Pavan Kumar Sunkara
<pavan....@gmail.com> wrote:
>
> Is there anything which is holding back gopkg.in from supporting the following?
>
> gopkg.in/v3.2/pkg => github.com/go-pkg/pkg (branch/tag
> "v3.2", "v3.2.M")
> gopkg.in/user/v3.2.1/pkg => github.com/user/pkg (branch/tag "v3.2.1")

This was supported for a while, but the main change announced in this
thread is that support for it was dropped to increase the chances of
sanity.

To understand why, consider this situation:

Package A depends on B and C
Package B depends on user/pkg v3.2
Package C depends on user/pkg v3.3

Using your suggestion we'd end up with two user/pkg instances in the
binary, while v3.3 alone would do.

Dropping support for that, we enforce that both package B and C will
be using user/pkg v3, which means go get -u on A will bring v3.3,
making both of them happy at once and without having two instances of
user/pkg in the final binary for A.


gustavo @ http://niemeyer.net

[Gustavo Niemeyer]

> What if v3.3 contains a minor breaking change that Package B doesn't want?

That's ignoring what the major version v3 means to people. No such
minor breaking changes should be introduced without also changing the
major version to v4.

From the first paragraph of the Version number section in the
documentation at https://gopkg.in:

The number used in the gopkg.in URL looks like "v1" or "v42", and
represents the major version for the Go package. No incompatible
changes should be done to the package without also changing that
version, so that packages and applications that import the package
can continue to work over time without being affected by broken
dependencies.

> Users having more choice is never a problem.

I wish things were that simple, but they tend not to be.


gustavo @ http://niemeyer.net

[Nate Finch]

On Tuesday, March 25, 2014 11:09:15 AM UTC-4, Gustavo Niemeyer wrote:

On Tue, Mar 25, 2014 at 11:54 AM, Pavan Kumar Sunkara
<pavan....@gmail.com> wrote:
> What if v3.3 contains a minor breaking change that Package B doesn't want?

That's ignoring what the major version v3 means to people. No such
minor breaking changes should be introduced without also changing the
major version to v4.

(emphasis mine)  ... see, there's the problem.  But I guess you have to trust the package authors at some point, afterall, even if you're pointed at v3.1.5, there's nothing to actually stop the authors from introducing breaking changes on that specific patch version either.

[Gustavo Niemeyer]

On Tue, Mar 25, 2014 at 2:22 PM, Nate Finch <nate....@gmail.com> wrote:
> (...) But I guess you have to trust the package authors at some point, afterall,

> even if you're pointed at v3.1.5, there's nothing to actually stop the authors
> from introducing breaking changes on that specific patch version either.

Exactly. Tooling, documentation, conversation, a community behind it..
all of that helps, but by the end of the day we need to care about
doing a good job, or no system will work.


gustavo @ http://niemeyer.net

[Pavan Kumar Sunkara]

>
> Package A depends on B and C
> Package B depends on user/pkg v3.2
> Package C depends on user/pkg v3.3
>
> Using your suggestion we'd end up with two user/pkg instances in the
> binary, while v3.3 alone would do.
>
> Dropping support for that, we enforce that both package B and C will
> be using user/pkg v3, which means go get -u on A will bring v3.3,
> making both of them happy at once and without having two instances of
> user/pkg in the final binary for A.

What if v3.3 contains a minor breaking change that Package B doesn't want?

In my opinion, it's better letting users have both v3.2 and v3.3 in
the binary (which generally would be rare) than enforcing the
users/developers to not make use of minor/patch versions.

Users having more choice is never a problem.

--
- Pavan Kumar Sunkara

[Pavan Kumar Sunkara]

Hey,

Is there anything which is holding back gopkg.in from supporting the following?

gopkg.in/v3.2/pkg => github.com/go-pkg/pkg (branch/tag
"v3.2", "v3.2.M")
gopkg.in/user/v3.2.1/pkg => github.com/user/pkg (branch/tag "v3.2.1")

Thanks

> --
> You received this message because you are subscribed to the Google Groups "Go Package Management" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to go-package-manag...@googlegroups.com.
> To post to this group, send email to go-package...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--

- Pavan Kumar Sunkara

[Gustavo Niemeyer]

On Tue, Mar 25, 2014 at 10:51 AM, <kpo...@gmail.com> wrote:
> The approach in gopkg.in seems based on the concept of semantic versioning
> (http://semver.org/). This is all well and good, but there are far from any
> guarantees that all package developers will adopt this versioning
> methodology.

Semantic versioning is a nice idea, but gopkg.in doesn't really
require it in all its extent. All it needs to work well is that people
respect the concept of major version, in that no revision in a major
can break what was there before.

> If gopkg.in was created to address the limitation of go-get always
> retrieving a repo's HEAD, why limit gopkg.in to the Major of a branch/tag
> with forced auto-upgrading?

Let's go over a very common scenario: If Alice and Bob maintain
packages A and B and have absolute control over which versions
of package P gets used, then Cath cannot pick *her* preferred version
of P when importing A and B. Not only that, but Alice and Bob might
well pick slightly different versions of P, which is also different from
what Cath would like to use. As a result, the application built by Cath
may end up with versions 3.0.1, 3.0.2 and 3.0.3 of P, because
everybody got "absolute control" of the dependencies.

That doesn't look good, right?

The approach taken by gopkg.in, which uses the major version only,
means that every package in a dependency tree that depends on a
compatible package API will use the same instance of that package.


gustavo @ http://niemeyer.net

[Eric Johnson]

I’m starting to feel like a broken record on this:

Give me a way to quickly test against multiple versions of a package. That’s the biggest problem with vendoring - I’m “locked in” to a particular version of a library. What I really want is a way to test my library against the earliest version of a library that I’m compatible with, and then everything later.

Preferably automated.

Semantic version is very useful insofar as major version at least gives me a target for the higher-end of future versions that I should test automatically.

And then, ideally, define a way for library to include metadata that declares what versions of its dependencies it has been validated against. Possibly generated/updated automatically. That way I can signal to downstream users maximum compatibility, instead of just supporting “the latest” as of the last time I did a release.

With this support in place, the “versioning” / “vendoring” / “central package management” questions become far less important, because you can safely take any of the approaches outlined in the various email threads.

Eric.

[Gustavo Niemeyer]

Yes.. and some wine, please!

;-)

> --
> You received this message because you are subscribed to the Google Groups
> "Go Package Management" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-package-manag...@googlegroups.com.
> To post to this group, send email to go-package...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--

gustavo @ http://niemeyer.net

[Kyle Lemons]

Probably the biggest thing preventing me from using gopkg.in for my own random projects is that having gopkg.in/kylelemons/v2/gousb as the latest when I'm working on gopkg.in/kylelemons/v10/rx seems strange.  If I were browsing it as a filesystem, I'd click on v10 and see that only one package exists, and assume that I  gave up on the rest.  I would much prefer gopkg.in/kylelemons/gousb/v2, despite the slight awkwardness that v2 isn't the imported name.

[Jesse McNelis]

On Tue, Mar 25, 2014 at 9:18 PM, Nate Finch <nate....@gmail.com> wrote:

I wish there was an option to *not* allow this auto-upgrade behavior.  It's actually not what I would expect by default.  I would expect that if I say v2, that I get v2 and no other.  Now, for some projects, getting "the most recent v2.x" is fine... but I think for a lot of projects, they'd prefer to be able to test new changes *before* they get automatically included in new builds.

My ideal behavior would be: v2 (without minor version specified) means "use the latest 2.x", and v2.3 (explicitly choosing a minor version) means "use this exact version and no other".  Yes the latter requires manually updating when you want to move to 2.4, but it lets me gate upstream changes on my own terms.  It's the more paranoid choice for people who really need their stuff not to break.\

This is precisely the problem I have with gopkg.in and specifying the versions in the import path. You have to fork a package to change it's dependencies.

It's especially bad as your dependency chain gets deeper as you end up having to fork every package in that chain to change the version dependency of one of those packages

If you allow packages to specify minor versions then the chance of two packages using two difference minor versions of a package is much greater and requires great effort to fix.

I much prefer no versions to versions I can't change.

Having only major versions reduces this risk greatly while still allowing package developers to specify breaking changes.

Note that every package manager(bundler, npm, pypi etc.) separates the name of the package being imported from the version being imported and allows the end developer to intentionally pick which version to use for their end program and checks that against some range of versions that their dependancies expect to support.

Weirdly, npm has the terrifying attribute of just importing multiple different versions of a package in to the same program, 

They just seem to ignore the problem of these packages sharing global resources.

 

[Gustavo Niemeyer]

On Tue, Mar 25, 2014 at 10:17 PM, Jesse McNelis <jes...@jessta.id.au> wrote:
> This is precisely the problem I have with gopkg.in and specifying the
> versions in the import path. You have to fork a package to change it's
> dependencies.

With the current conventions of gopkg.in you only have to fork the
package to change the import path if you either are changing to an
incompatible major version, in which case the code may require manual
fiddling either way, or if you're changing to a completely different
package, in which case it's no different than any other convention
adopted around Go. So I don't really see what is the problem you have
with gopkg.in in those terms.

> Note that every package manager(bundler, npm, pypi etc.) separates the name
> of the package being imported from the version being imported and allows the
> end developer to intentionally pick which version to use for their end
> program and checks that against some range of versions that their
> dependancies expect to support.

Again, gopkg.in is no different in that regard, in that it won't get
in your way if you want to pick different versions. What it does do is
encode in the import path what is the API version it expects from the
package, to prevent tools from picking dependencies that would not
satisfy what the package expects from them. I'm hoping this sounds
like a good thing.


gustavo @ http://niemeyer.net

[Joshua Poehls]

This definitely my biggest annoyance as well. I'm using gopkg.in anyway, but I'd MUCH prefer to have the version at the end of the URL.

--

Joshua Poehls

[Joshua Poehls]

Whenever I read these threads I feel like most people just don't understand that Go import paths already encode the equivalent of a major version number (in semantic versioning terms). That is, the import path already implies a "no breaking changes" stable API. Good package authors respect this already and create new repositories for "V2" APIs with breaking changes. (See the latest Go SSH package announcement on go-nuts.)

All of the discussion about being concerned about breaking changes "sneaking in" without a major version/import path change are irrelevant. This can happen today, this can happen on any other package manager in any other language that I'e seen. The only solution to protect against that is vendoring (IMO) and that is just as easy to do with gopkg.in as it is without it.

From my point of view, all that gopkg.in does is allow you to keep your major versions in the same repository which is a huge convenience for package authors and pretty much makes no difference to consumers of packages.

--

Joshua Poehls

--

[t0]

Hey gus, put me down for

gopkg.in/pkg/v3 => github.com/go-pkg/pkg-v3

I agree with the version numbering.

Do you sleep? ;) keep up the good work! 

[John Eikenberry]

Kyle Lemons wrote:

> Probably the biggest thing preventing me from using gopkg.in for my own
> random projects is that having gopkg.in/kylelemons/v2/gousb as the latest
> when I'm working on gopkg.in/kylelemons/v10/rx seems strange. If I were
> browsing it as a filesystem, I'd click on v10 and see that only one package
> exists, and assume that I gave up on the rest. I would much prefer
> gopkg.in/kylelemons/gousb/v2, despite the slight awkwardness that v2 isn't
> the imported name.

Why not use a tag or branch?

--

John Eikenberry
[ j...@zhar.net - http://zhar.net ]
[ PGP public key @ http://zhar.net/jae_at_zhar_net.gpg ]
________________________________________________________________________
Sic gorgiamus allos subjectatos nunc