Go += Package Versioning

[Russ Cox]

Hi everyone,

I have a new blog post you might be interested in.

https://research.swtch.com/vgo.

I'll try to watch this thread to answer any questions.

Best,

Russ

[Zellyn]

A couple of initial questions:

• The "pretend you're importing github.com/foo/bar but actually pull it from localgit.host/foo/bar" functionality of dep was going to really simplify things for us. I can't quite tell whether vgo includes that functionality or not. We would want to do it at the monorepo level: ie. *everything* should use our forked version of the `glog` and `protobuf` packages.
• For our (go monorepo) usecase: do you envision a single go.mod file being placed at the root, or would we sprinkle them as needed for each sub-project in the monorepo? Does vgo search upward for go.mod files?
• More details on the zipfile module representation will be welcome.
• Getting rid of GOPATH is nice. Where will `vgo install` drop binaries?
  

Thanks for your work on this. It looks very “Go-ish”. Interested to learn more.

Zellyn

[David Anderson]

I love this. I want it now.

I've struggled with `glide` and `dep` dependency hell a lot in my Go projects (especially those that use the Kubernetes client library, which abuses "I, the library author, can define byzantine constraints on my users" to the extreme). The way I've described it informally is that traditional "arbitrary version selection" algorithms create the wrong incentive structure for library authors, in the sense that they have all the power to lock versions arbitrarily, but bear none of the pain associated with their decisions - instead, the binary author at the apex of the dependency tree gets that pain. "Authority without responsibility" is the death of many incentive structures.

The proposed rules around semver and minimal version selection address every pain point I've had so far, by aligning available authority with the responsibility of the library authors: tell me what versions definitely will not work, and I'll take it from there.

Questions that arose during reading:

• In the "defining Go modules" section, you say: "Although semantic versions are strongly preferred, referring to specific commits will be supported as well." Then, you specify that such commits order before v0.0.0. To me, this suggests that this feature will be useless, because any non-trivial example will have some semver constraint somewhere in the dependency tree, such that in practice commit-hash versions will always be overridden by minimal version selection. I don't have a solution to this (other than removing support for commit-hash versions), but it seems like it's something to think about
• One alternative I considered, but discarded, was to order all tags and commits by their date, and run minimal version selection on that. However, this doesn't work because semver tag dates don't grow monotonically - you might have 0.1.2 released after 0.2.0, which leads to confusing behavior to the user - why did 0.2.0 get selected when 0.1.2 was a "better" minimal version?
• Modules as zip archives: how do I discover which versions are available? Minimal version selection seems to rely on being able to list the version continuum for a module, so that constraints can be applied. What's the expected way to do that?
• Writing module files sounds like a job for a machine, in cases where I don't care which version gets used. Can `goimports` (or a new sibling tool) be taught to update module definitions based on my code?
• No GOPATH required: YES! Thank you. I share Zellyn's question on what happens to `go install` in this world. Does it go away? Does it install to some new $GOBINARYINSTALL or somesuch?

Haven't read the vgo tour yet, it may answer some of these. I may be back with more :)

- Dave

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Russ Cox]

On Tue, Feb 20, 2018 at 1:17 PM, Zellyn <zel...@gmail.com> wrote:

A couple of initial questions:

• The "pretend you're importing github.com/foo/bar but actually pull it from localgit.host/foo/bar" functionality of dep was going to really simplify things for us. I can't quite tell whether vgo includes that functionality or not. We would want to do it at the monorepo level: ie. *everything* should use our forked version of the `glog` and `protobuf` packages.

You can do that. See research.swtch.com/vgo-tour and ^F for 'myfork'.

• For our (go monorepo) usecase: do you envision a single go.mod file being placed at the root, or would we sprinkle them as needed for each sub-project in the monorepo? Does vgo search upward for go.mod files?

Yes, it sounds like you'd want one in the root, which would apply to all subdirectories (until hitting another). Vgo searches upward.

• More details on the zipfile module representation will be welcome.

That'll be Thursday. Sorry, there's a lot to write, but it's on its way. :-)

If you want to poke around, look in $GOPATH/src/v/cache.

• Getting rid of GOPATH is nice. Where will `vgo install` drop binaries?
  

You don't have to work in GOPATH, but GOPATH will stick around as a way to

control where the cached downloaded source code lives. Most users won't

need to worry since it now defaults to $HOME/go, so no one has to set it.

For now I decided to try having vgo install commands to $GOPATH/bin,

since most people will already have it in their path and it is Go-specific.

As with just about everything, it's subject to change if we discover problems,

but it's been that way for, oh, 4 hours and it seems to be working OK so far.

Of course, if $GOBIN is set, then that wins, as it always has.

Best,

Russ

[Devon H. O'Dell]

With regards to minimum version selection, if I depend on some feature
present, this comes with two implicit dependencies: correctness and
safety. My knee-jerk reaction here is that the time folks spend
"telling the package manager, 'no, use at least Y,'" will largely be
in response to these sorts of issues. Maybe this is seen as "working
as intended;" my feeling is that this will become tiresome for
projects with large dependency chains. Might it be worthwhile (and
automatically possible) to pick the "maximally correct" minimum
version?

--dho

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[Russ Cox]

On Tue, Feb 20, 2018 at 1:55 PM, David Anderson <da...@natulte.net> wrote:

I love this. I want it now.

go get -u golang.org/x/vgo :-)

I've struggled with `glide` and `dep` dependency hell a lot in my Go projects (especially those that use the Kubernetes client library, which abuses "I, the library author, can define byzantine constraints on my users" to the extreme). The way I've described it informally is that traditional "arbitrary version selection" algorithms create the wrong incentive structure for library authors, in the sense that they have all the power to lock versions arbitrarily, but bear none of the pain associated with their decisions - instead, the binary author at the apex of the dependency tree gets that pain. "Authority without responsibility" is the death of many incentive structures.

Yes, I'll make that point again in tomorrow's posts elaborating minimal version selection, but I think this is probably the most important algorithmic policy detail. You get total control over your own build. You only get limited control over other people's builds that happen to import your code.

Questions that arose during reading:

• In the "defining Go modules" section, you say: "Although semantic versions are strongly preferred, referring to specific commits will be supported as well." Then, you specify that such commits order before v0.0.0. To me, this suggests that this feature will be useless, because any non-trivial example will have some semver constraint somewhere in the dependency tree, such that in practice commit-hash versions will always be overridden by minimal version selection. I don't have a solution to this (other than removing support for commit-hash versions), but it seems like it's something to think about

That's by design: if v1.2.3 is the latest tagged version and there's some experimental new stuff that you choose to depend on, and then vgo has to decide between those two requirements coming from two different dependencies in a larger build, I want it to pick the tagged one. That said, for builds of your own module, what you'd do is define that you want to replace the current latest (say, v1.2.3) with a different commit. In go.mod you'd say:

    replace "p.xyz/foo" v1.2.3 => "p.xyz/foo" commithash

and then the version selection would transparently drop in commithash as if it were really v1.2.3, so that it would win over v1.2.2, but not v1.2.4. I suppose you could even make up a very large number like v1.9999.0 and then depend on that. But again the replacement only applies to your build, not to other builds using your module. 

• Modules as zip archives: how do I discover which versions are available? Minimal version selection seems to rely on being able to list the version continuum for a module, so that constraints can be applied. What's the expected way to do that?

Yes, you have to have a way to list them. I'll write more about that Thursday.

• Writing module files sounds like a job for a machine, in cases where I don't care which version gets used. Can `goimports` (or a new sibling tool) be taught to update module definitions based on my code?

Goimports just needs to put the import in. "vgo build" will see the new import and take care of the rest. It rewrites go.mod for you (and formats it too, of course).

Best,

Russ

[Russ Cox]

On Tue, Feb 20, 2018 at 2:35 PM, Devon H. O'Dell <devon...@gmail.com> wrote:

With regards to minimum version selection, if I depend on some feature
present, this comes with two implicit dependencies: correctness and
safety. My knee-jerk reaction here is that the time folks spend
"telling the package manager, 'no, use at least Y,'" will largely be
in response to these sorts of issues. Maybe this is seen as "working
as intended;" my feeling is that this will become tiresome for
projects with large dependency chains. Might it be worthwhile (and
automatically possible) to pick the "maximally correct" minimum
version?

I am not sure what you mean by "maximally correct".

I think that's approximately what all the other package managers try to do.

It makes things very complex (literally: research.swtch.com/version-sat).

Note that it's still super easy to say "update this package to latest"
or "update everything to latest". It just only happens when you ask,

not when the tool feels like it.

Russ

[Devon H. O'Dell]

2018-02-20 11:39 GMT-08:00 Russ Cox <r...@golang.org>:
> On Tue, Feb 20, 2018 at 2:35 PM, Devon H. O'Dell <devon...@gmail.com>
> wrote:
>>
>> With regards to minimum version selection, if I depend on some feature
>> present, this comes with two implicit dependencies: correctness and
>> safety. My knee-jerk reaction here is that the time folks spend
>> "telling the package manager, 'no, use at least Y,'" will largely be
>> in response to these sorts of issues. Maybe this is seen as "working
>> as intended;" my feeling is that this will become tiresome for
>> projects with large dependency chains. Might it be worthwhile (and
>> automatically possible) to pick the "maximally correct" minimum
>> version?
>
> I am not sure what you mean by "maximally correct".
> I think that's approximately what all the other package managers try to do.
> It makes things very complex (literally: research.swtch.com/version-sat).

Thanks. I was thinking in terms of e.g. a "correctness counter" that
could be published. If I depend on e.g. foo.Bar, I automatically want
the least buggy version of foo.Bar. If foo.Bar appears in version 1,
the amount of work required to find the least buggy version is
directly proportional to the number of releases in which foo.Bar
appears, and is annotated as having some correctness fix. I don't
think this would necessarily be as complex as you point out in the
referenced post: this is a property of an individual package. Of
course, you want the same property for each package's dependencies,
but that can be decided without conflict using the same methodology
for each package. That is, I don't think there's a case for package X
with a dependency on package Y where the minimum version can't be
decided in linear time.

Upon further thought, I realize folks will occasionally rely on
particular forms of buggy behavior. I think that's probably fine if
you can always require a specific version. However, it'd probably
require some kinds of annotations on every portion of the API to work
well: if I rely only on foo.Bar, and there's no dependency on foo.Bat
which has a correctness change, that shouldn't leak into foo.Bar. That
is probably an unreasonable amount of work to require from developers,
so probably just ignore me :)

--dho

[Bakul Shah]

I had a somewhat similar reaction as Devon.

Reproducible builds are indeed very desirable but, as Devon
hints at, it is also the case that differences between
versions grow over time and implicit assumptions may break.
For example, updating from v1.1 to v1.5 and then v1.10 may be
easier than updating from v1.1 directly to v1.10. That is,
updating version depdendencies from time to time may be
considered part of regular maintenance.

*Once* you release a package you want its dependencies
explicitly stated for reproducible builds but when creating
the next release of the same package, you may want to update
to later versions of imported packages due to the above
considerations.

May be one can write a tool around or similar to "vgo test" to
test and update not to the latest but some earlier version.
For example, if pkg A currently imports C v1.x and B imports C
v1.y, through iterative testing one may find C v1.z that works
for both and update go.mod files for both A & B.

The underlying concern is that with better tooling to manage
versioning, there will also be more versions to manage as
there is less of an incentive to try to make things work with
the latest versions of imported packages. I wondered if there
is a way to encourage/improve the quality/correctness aspect.
One vague idea is to consider specifying tests (in go.mod) to
be run prior to an update. Sort of like asserts in code.

Apart from this concern vgo seems like a huge improvement.
Well done!

[Russ Cox]

On Tue, Feb 20, 2018 at 3:46 PM, Bakul Shah <ba...@bitblocks.com> wrote:

May be one can write a tool around or similar to "vgo test" to

test and update not to the latest but some earlier version.
For example, if pkg A currently imports C v1.x and B imports C
v1.y, through iterative testing one may find C v1.z that works
for both and update go.mod files for both A & B.

The underlying concern is that with better tooling to manage
versioning, there will also be more versions to manage as
there is less of an incentive to try to make things work with
the latest versions of imported packages.  I wondered if there
is a way to encourage/improve the quality/correctness aspect.
One vague idea is to consider specifying tests (in go.mod) to
be run prior to an update.  Sort of like asserts in code.

In the tour (research.swtch.com/vgo-tour) I show how "all" is redefined to be useful again. So it's completely reasonable to try an update, go test all, and if all the tests pass (and you trust your tests), then check in the go.mod file. Someone could build a bot to do this automatically, even. I don't think minimal version selection means you're always stuck with old versions. It just means you don't get new versions until you ask for them and are ready to evaluate how good they are, not just because you run 'go get' and a new version has appeared overnight.

Thanks.

Russ

[David Anderson]

As a counterpoint to this: with vgo, I plan to make all my binary modules specify that they want the latest versions of everything, and update that pinning every couple of weeks/months. If I encounter bugs, I'll report them, and move the pinning back a little, safe in the knowledge that my dependencies will tell me if I've moved it too far back.

I also believe the tooling around vgo should encourage/make default this behavior for binary modules (and maybe for library modules as well, though that's less clear to me). The default behavior when writing Go programs should be to use the latest of everything, unless you have a specific reason to hold back. Among other things, that ensures you are patched against CVEs and whatnot.

In my mind's eye, the version declarations that modules make for consumption by other modules is the tail end of the world's sliding window of "what set of stuff can build together". It should not be the default place that most Go binaries live.

- Dave

 

  I wondered if there
is a way to encourage/improve the quality/correctness aspect.
One vague idea is to consider specifying tests (in go.mod) to
be run prior to an update.  Sort of like asserts in code.

Apart from this concern vgo seems like a huge improvement.
Well done!

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[David Anderson]

On Tue, Feb 20, 2018 at 11:37 AM, Russ Cox <r...@golang.org> wrote:

On Tue, Feb 20, 2018 at 1:55 PM, David Anderson <da...@natulte.net> wrote:

I love this. I want it now.

go get -u golang.org/x/vgo :-)

I've struggled with `glide` and `dep` dependency hell a lot in my Go projects (especially those that use the Kubernetes client library, which abuses "I, the library author, can define byzantine constraints on my users" to the extreme). The way I've described it informally is that traditional "arbitrary version selection" algorithms create the wrong incentive structure for library authors, in the sense that they have all the power to lock versions arbitrarily, but bear none of the pain associated with their decisions - instead, the binary author at the apex of the dependency tree gets that pain. "Authority without responsibility" is the death of many incentive structures.

Yes, I'll make that point again in tomorrow's posts elaborating minimal version selection, but I think this is probably the most important algorithmic policy detail. You get total control over your own build. You only get limited control over other people's builds that happen to import your code.

Big +1. I'm curious about your ideas on how to manage the transition from a glide/dep world to a vgo world. I look forward to reading more about that, assuming you have an article queued to discuss that facet :).

 

Questions that arose during reading:

• In the "defining Go modules" section, you say: "Although semantic versions are strongly preferred, referring to specific commits will be supported as well." Then, you specify that such commits order before v0.0.0. To me, this suggests that this feature will be useless, because any non-trivial example will have some semver constraint somewhere in the dependency tree, such that in practice commit-hash versions will always be overridden by minimal version selection. I don't have a solution to this (other than removing support for commit-hash versions), but it seems like it's something to think about

That's by design: if v1.2.3 is the latest tagged version and there's some experimental new stuff that you choose to depend on, and then vgo has to decide between those two requirements coming from two different dependencies in a larger build, I want it to pick the tagged one. That said, for builds of your own module, what you'd do is define that you want to replace the current latest (say, v1.2.3) with a different commit. In go.mod you'd say:

    replace "p.xyz/foo" v1.2.3 => "p.xyz/foo" commithash

and then the version selection would transparently drop in commithash as if it were really v1.2.3, so that it would win over v1.2.2, but not v1.2.4. I suppose you could even make up a very large number like v1.9999.0 and then depend on that. But again the replacement only applies to your build, not to other builds using your module. 

That makes sense, thanks. What I was looking for was: it is impossible for libraries to "work around" this system by using, say, `dep` to resolve versions using more complex constraints, and then lock those versions in by commit hash for all its consumers. But I now see you've explicitly thought about that, and we can get the best of both worlds - apex modules can lock to commits in order to work around bugs or achieve non-technical goals, but library authors cannot impose arbitrary locks.

 

• Modules as zip archives: how do I discover which versions are available? Minimal version selection seems to rely on being able to list the version continuum for a module, so that constraints can be applied. What's the expected way to do that?

Yes, you have to have a way to list them. I'll write more about that Thursday.

One thought that I hope you'll cover there: immutability. Reading through the vgo tour, it seems to assume that once v5.6.7 has been released, that .zip and its declared dependencies will never change.

That is indeed what semver requires you to do, but I wonder how often "oopsie" reuploads happen at the same version, and how well/poorly vgo behaves under those conditions. IMO it's probably fine to say "wontfix, don't do that", as long as it's loudly documented.

 

• Writing module files sounds like a job for a machine, in cases where I don't care which version gets used. Can `goimports` (or a new sibling tool) be taught to update module definitions based on my code?

Goimports just needs to put the import in. "vgo build" will see the new import and take care of the rest. It rewrites go.mod for you (and formats it too, of course).

Excellent. I expected nothing less from Go tooling, but it's good to know for sure :).

 

Best,

Russ

[Bakul Shah]

Not automatically fetching the latest version is the right
choice. Minimal version selection is also something I agree
with. But an unintended consequence of this choice may be
binaries of large packages pulling in multiple versions. So I
was wondering what can be done to encourage periodic updates.

I just wanted to provide this feedback but it may be too early
to worry about this. As vgo is used more and more may be the
right tools will organically evolve!

Thanks,
Bakul

[Devon H. O'Dell]

2018-02-20 13:02 GMT-08:00 David Anderson <da...@natulte.net>:
> [snip]

>
> I also believe the tooling around vgo should encourage/make default this
> behavior for binary modules (and maybe for library modules as well, though
> that's less clear to me). The default behavior when writing Go programs
> should be to use the latest of everything, unless you have a specific reason
> to hold back. Among other things, that ensures you are patched against CVEs
> and whatnot.

AIUI, that is the opposite of what minimum version selection does;
active maintenance is required. This is at least no worse than any
other case where you need to make a determination on whether to
consume some library or tool based on whether it is actively
maintained. And when the tools are maintained, seems like it is
probably better.

--dho

[David Anderson]

I think we're agreeing, with different words (but please correct me if I'm wrong!).

Yes, active maintenance is required with vgo, as demonstrated in the tour. But crucially, I do not get held back by library writers that don't update, I'm only held back by my own laziness in failing to update at the apex of the dependency tree. That's something I can automate away. It probably means I'll be a beta tester for a new combination of versions more often than not. But `vgo test` helps with that, and generally encourages the writing of useful tests, so again I think the incentives all align. And when they don't, there's the replacement mechanism to fork in place.

- Dave

[Dave MacFarlane]

I really like this, except for the claim that it the blog post that it
will eliminate vendoring and deprecate GOPATH and the problems that
will cause for backwards compatibility for things that are currently
using them. If this is going to result in removing language features
(ie. vendoring), shouldn't it be in Go2, not Go 1.11?

I've abused vendoring in the past for things like fixing bugs in
upstream libraries that I use where the maintainer hasn't been
responsive to pull requests and, while the "replace = .." of vgo is
definitely a better solution, removing vendoring would mean that I
would need to go through every repo I have to ensure they keep
building once this goes in.

The other question I have is: where does the module name come from if
you don't have a "// import" comment on the package, since it can't be
filesystem based (because you're explicitly outside of GOPATH..)

- Dave

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to golang-nuts...@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
- Dave

[Bakul Shah]

On Tue, 20 Feb 2018 13:02:47 -0800 David Anderson <da...@natulte.net> wrote:
>
> As a counterpoint to this: with vgo, I plan to make all my binary modules
> specify that they want the latest versions of everything, and update that
> pinning every couple of weeks/months. If I encounter bugs, I'll report
> them, and move the pinning back a little, safe in the knowledge that my
> dependencies will tell me if I've moved it too far back.
>
> I also believe the tooling around vgo should encourage/make default this
> behavior for binary modules (and maybe for library modules as well, though
> that's less clear to me). The default behavior when writing Go programs

> *should* be to use the latest of everything, unless you have a specific

> reason to hold back. Among other things, that ensures you are patched
> against CVEs and whatnot.

The right choice here is not clear (at least to me). You do
want the latest security patches and you will want bug fixes
but you don't necessarily want the latest functional changes
as APIs may not have settled down or there may be more bugs
to be shaken out.

Usually more stable versions are tagged and "released" but you
may still choose to go with not the latest released version
but one or more earlier releases. Then there are issues such
as needing to run s/w on older versions of OS (not everyone
updates to the latest versions -- in large organizations the
delay can be very long) and often leading edge s/w will drop
support for older OS versions so ideally your s/w should work
on last 2-4 versions of an OS.

[Tom Mitchell]

On Tue, Feb 20, 2018 at 2:29 PM, Bakul Shah <ba...@bitblocks.com> wrote:

On Tue, 20 Feb 2018 13:02:47 -0800 David Anderson <da...@natulte.net> wrote:
>
> As a counterpoint to this: with vgo, I plan to make all my binary modules
> specify that they want the latest versions of everything,

..... 

The right choice here is not clear (at least to me). 

..... 

Usually more stable versions are tagged and "released"

My bias is to use the latest "released" tools libraries and source. 
Released code has more interactions and probably a better chance
for the community discovery of an issue you care about.

Build and test often as part of the process. 

It is not just the libraries but also the compiler and build system itself. 

I spent a month+ trying to sort out a customers application bug that 
resulted from promotion of an optimization to an intrinsic in a compiler

update.

Compilers and applications can be victims of a system lib change and more including system test 
resources.

This is a difficult problem and the answer involves knowing (best guess) what it takes to debug the

entire pile of worms not just the application.  It is turtles all the way down but one

possible solution is to put all the worms you can in a virtual machine that can be canned and archived.
A 4.7 GB DVD or a 25 GB Blu-Ray archival quality disk for the self contained 

single purpose built VM.   A 25GB system is worthy for a lot of projects but not all.

--

  T o m    M i t c h e l l

[s...@spenczar.com]

I'd like to understand the expectations of library maintainers better. I am particularly thinking about the reference to setting major versions through import paths, like github.com/go-yaml/yaml/v2.

Is the expectation that library owners would need to change the structure of their project like this when making breaking changes, or is it that vgo will handle this for users behind the scenes?

Second, I'd like to understand the proposed workflow when one repo has many subpackages that should be independently versioned. For example, a repo might have a exp/ subdirectory for experimental libraries that are available for adventurous users. Git tags can only identify the version of the whole repo; how will library maintainers signal that different packages have different versions?

I think the exp/ case is important but not the only case worth thinking about. It's also important in repos that provide both an installable main package and library code, like github.com/golang/protobuf. They might want to use tags and releases for distributing their binary, and might want to make breaking changes to their command-line API without a breaking version bump for the libraries, or vice-versa.

[Wojciech S. Czarnecki]

On Tue, 20 Feb 2018 12:20:19 -0500
Russ Cox <r...@golang.org> wrote:

> https://research.swtch.com/vgo.
> Russ

Amazing! The 'minimal version selection' approach is briliant.

Where it lacks for me is about keeping up with security patches.
If my module keeps libFoo at 1.2.3 it will not see a backported fix
that came with 1.2.98. At least not without me doing chores and
updating what I have to and leaving out what I don't.

So I'd like to see vgo to tackle also backported security patches.

Namely tags in the form of:

x.y.z+spN where N is a security patch level.

If there is no +spN meta in the chosen tag, sp0 level is assumed.
If there is +spN, vgo SHOULD update dependency to the
highest N it sees - unless forbidden by the go.mod.

So:
require (
"rsc.io/quote" v1.5.2
)

Will use v1.5.2+sp7 if there is a such tag in the repo and 7 is the
highest sp level.

For testing (or govt certification) purposes one may
intentionally keep dependency with a hole; e.g:

"rsc.io/quote" v1.5.2 noSecurityPatchesPlease

or even

"rsc.io/quote" v1.5.2+sp3 noSecurityPatchesPlease

to have quote pinned to exact sp3 level.

In corporate environments it is not unusual to have decades old
software in production and maintenance, where 'maintenance'
is strictly regulated. Above scheme would help with that. IMO.

As vgo promises builds with tags immersed into binary any
security audit of go binaries may just list versions and compare
sp levels with ones at the source repo. If there is higher spN,
we will know something is amiss immediately - without grepping
changelogs for a CVE lines.

m0.2c

--
Wojciech S. Czarnecki
<< ^oo^ >> OHIR-RIPE

[Jon Calhoun]

Would you be willing to create an example illustrating how a versioned package should look with major version changes? You have great examples for the minor/patch changes, but I don't see any concrete examples for a major version bump.

On Tuesday, February 20, 2018 at 12:20:54 PM UTC-5, Russ Cox wrote:

[Brian Slesinsky]

I'm wondering how to respond to security patches. After a patch, any go.mod file mentioning an older version of the library is a candidate for version-bumping: download the new version, test, and do a commit with the new version number if all goes well.

It's nice that it can be done in any order, but if it isn't done (either manually or automatically) nobody will use the new version except new customers of the patched library.

I suppose a minimal approach would be to encourage people to run "vgo get -u" periodically, test, and commit.

On Tuesday, February 20, 2018 at 9:20:54 AM UTC-8, Russ Cox wrote:

[Peter Bourgon]

As far as I understand it

- github.com/user/foo is expected to house package foo for SemVer 0.x.x – 1.x.x
- github.com/user/foo/v2 is expected to house package foo for SemVer 2.x.x
- github.com/user/foo/v3 is expected to house package foo for SemVer 3.x.x

And so on. Appending the additional path is optional in the sense that
consumers can still lock to package foo at v2.3.4 if it lives in plain
github.com/user/foo, but that lock will be expressed in the so-called
psuedoversion form "v0.0.0-timestamp-revhash".

[Dominik Honnef]

Hi,

I'd like your input on two concern I have, one as a tool developer and
one with regard to reproducible builds.

With go and go get, it is expected that code will not compile until
all dependencies have been explicitly downloaded. Hence, it is
acceptable for tools such as staticcheck to also fail if dependencies
are not present. Vgo seems to change this, by promising the user that
dependencies will be automatically downloaded. Running `vgo build` at
any point will "just work". Tools, however, do not have this luxury,
for multiple reasons:

- Multiple tools may run in parallel, either via helpers such as
gometalinter, or simply because editors like Emacs launch multiple
tools at once, for example when saving a file. This is likely prone to
race conditions. File locking may be an option, but file locking is
known to cause erratic problems on networked file systems, or be
outright unsupported.
- Fetching the dependencies, e.g. via 'vgo build', has the side-effect
of modifying the go.mod file. This is probably unacceptable for tools
that are run automatically, without the user's express wish to add new
dependencies to the go.mod file.
- If we do find a solution to these two problems, we'd probably still
want a 'vgo get' (with no arguments), to just fetch the dependencies,
without also incurring a costly build.

In my opinion, tools should behave as closely as possible to (v)go
build, so that users can operate on a single set of expectations.

My second concern is regarding the promise of getting rid of the
vendor directory. Reproducible builds not only depend on versioning,
but also on ensuring that code doesn't go away, be it due to server
downtime or more permanent reasons. Vendoring makes this rather easy:
add your dependencies to vendor and commit them, and your project is
now self-contained. Vgo seems to discourage this practice. Can you
suggest an alternative that is as straightforward as vendoring? All
other solutions - which amount to various forms of proxies and mirrors
- are significantly more laboursome.

I have noticed that currently, vgo downloads dependencies into GOPATH.
Are there any plans to instead store them with the module itself,
possibly in a way that facilitates committing them to version control?

[Chandru]

One shortcoming I see in this proposal compared to dep is the necessity of operating a proxy to achieve reasonable security. Without a proxy or a lock file, a repository take-over as it happened in the case of go-bindata will lead to vgo automatically downloading potentially malicious code.

In the case of dep the lock file ensures that the exact commit hash is fetched until "dep ensure -update" is run explicitly, even if the git tag itself has been changed point to a different commit. If the vendor directory is committed to VCS, the build continues to work safely irrespective of what happens in Github or any other repository host.

--
Chandra Sekar.S

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[je...@samsara.com]

Two more use cases of vendor that I would like to keep:

- A single audit trail for all go code going into production binaries in a git log.

- A single place to enforce upgrades for multiple binaries in a monorepo. Perhaps a top-level meta-module could do the trick?

[Henrik Johansson]

I am currently running `vgo build` on a decent sized project that has a few dependencies.

It bailed on import case it seems:

import "github.com/spf13/jwalterweatherman": module path of repo is github.com/spf13/jWalterWeatherman, not github.com/spf13/jwalterweatherman (wrong case)

This is a transitive dep and it seems it indeed mixed upper and lower case.

Does vgo assume lower case?

[Henrik Johansson]

Actually it seems viper imports it with lower case.
I don't remember if import paths are to be considered different if they only differ in case.

[sam boyer]

Unfortunately, the rules also create some new perverse incentives, with some, IMO, nastier failure modes and remediations. That's one of the things i'll be elaborating on in my doc (hopefully) next week.

That said, one of the points Russ and i agree on is that dep does allow more expressiveness than is wise.

On Tuesday, February 20, 2018 at 1:55:59 PM UTC-5, David Anderson wrote:

I love this. I want it now.

I've struggled with `glide` and `dep` dependency hell a lot in my Go projects (especially those that use the Kubernetes client library, which abuses "I, the library author, can define byzantine constraints on my users" to the extreme). The way I've described it informally is that traditional "arbitrary version selection" algorithms create the wrong incentive structure for library authors, in the sense that they have all the power to lock versions arbitrarily, but bear none of the pain associated with their decisions - instead, the binary author at the apex of the dependency tree gets that pain. "Authority without responsibility" is the death of many incentive structures.

The proposed rules around semver and minimal version selection address every pain point I've had so far, by aligning available authority with the responsibility of the library authors: tell me what versions definitely will not work, and I'll take it from there.

Questions that arose during reading:

• In the "defining Go modules" section, you say: "Although semantic versions are strongly preferred, referring to specific commits will be supported as well." Then, you specify that such commits order before v0.0.0. To me, this suggests that this feature will be useless, because any non-trivial example will have some semver constraint somewhere in the dependency tree, such that in practice commit-hash versions will always be overridden by minimal version selection. I don't have a solution to this (other than removing support for commit-hash versions), but it seems like it's something to think about

• One alternative I considered, but discarded, was to order all tags and commits by their date, and run minimal version selection on that. However, this doesn't work because semver tag dates don't grow monotonically - you might have 0.1.2 released after 0.2.0, which leads to confusing behavior to the user - why did 0.2.0 get selected when 0.1.2 was a "better" minimal version?

• Modules as zip archives: how do I discover which versions are available? Minimal version selection seems to rely on being able to list the version continuum for a module, so that constraints can be applied. What's the expected way to do that?

• Writing module files sounds like a job for a machine, in cases where I don't care which version gets used. Can `goimports` (or a new sibling tool) be taught to update module definitions based on my code?

• No GOPATH required: YES! Thank you. I share Zellyn's question on what happens to `go install` in this world. Does it go away? Does it install to some new $GOBINARYINSTALL or somesuch?

Haven't read the vgo tour yet, it may answer some of these. I may be back with more :)

- Dave

[Nicolas Grilly]

I like this!

One question:

Some of us commit their `vendor` folder, to be able to build the project without any network access after the initial checkout ("offline mode").

Since the `vendor` folder is gone, is vgo able to manage and use a kind of `mirror` folder, which would contain the .zip archives of the project dependencies? This folder could be committed with the project source code.

The yarn package manager, for example, has a similar feature:

https://yarnpkg.com/blog/2016/11/24/offline-mirror/

[matthe...@gmail.com]

Since the `vendor` folder is gone, is vgo able to manage and use a kind of `mirror` folder, which would contain the .zip archives of the project dependencies? This folder could be committed with the project source code.

I have the same question. Can we vendor modules? Code consumers may not have a way to get to my proxy or contact me and the project may become unbuildable due to lost dependencies.

Matt

[Sam Whited]

Something that is still unclear to me even after reading todays post is how I would manage the various package import paths created by the import compatibility rule.
The way this is currently done is through tools such as gopkg.in, will the import path be modified and rewritten by a package server in the future?
If not, do I maintain separate /v2 and /v3 trees in my directory and copy/paste changes between them when I want to release security fixes as a point release?
Do I maintain separate branches that have a single /v2 or /v3 tree (if so, cherry-picking changes between them will be very difficult)?
Do I maintain separate repos and cherry pick or copy/paste files between them?

None of these sound especially appealing except perhapse having a server handle it for me, but this makes hosting my own packages harder.

—Sam

--
Sam Whited
s...@samwhited.com

[Zellyn]

Maintaining fixes on two different branches doesn't seem much easier. I guess you could cherry-pick the same commit onto both branches, if it worked. With both versions visible in the same repository, you can send one PR that applies the same fix in two places, which is nice for review.

I guess every approach has pros and cons. vgo seems particularly goish :-)

Zellyn 

[Peter Bourgon]

On Wed, Feb 21, 2018 at 11:22 AM, Sam Whited <s...@samwhited.com> wrote:
> Something that is still unclear to me even after reading todays post is how I would manage the various package import paths created by the import compatibility rule.
> The way this is currently done is through tools such as gopkg.in, will the import path be modified and rewritten by a package server in the future?

No.

> If not, do I maintain separate /v2 and /v3 trees in my directory and copy/paste changes between them when I want to release security fixes as a point release?

Yes, this is what vgo expects.

> Do I maintain separate branches that have a single /v2 or /v3 tree (if so, cherry-picking changes between them will be very difficult)?
> Do I maintain separate repos and cherry pick or copy/paste files between them?

Both of these also seem possible, at first glance.

> None of these sound especially appealing except perhapse having a server handle it for me, but this makes hosting my own packages harder.
>
> —Sam
>
> --
> Sam Whited
> s...@samwhited.com
>

[Sam Whited]

On Wed, Feb 21, 2018, at 13:49, Peter Bourgon wrote:
> > If not, do I maintain separate /v2 and /v3 trees in my directory and copy/paste changes between them when I want to release security fixes as a point release?
>
> Yes, this is what vgo expects.

This (and the other alternatives) seem like they break most common source control work flows to me. Any time we force users to change their workflow, I suspect we risk having to back track later. Even in cases where it may be better (eg. GOPATH which, like many people, I fought against when I first started using Go, but later came to love) if you force developers to drastically change their work flow, many of them will either push back, or, if it's only convention as this is, simply won't do it.

—Sam

[Henrik Johansson]

But wait. Wasn't there a mention of archive downloads instead of relying on the different VCS's?

In the GitHub case I guess it amount to downloading releases (or any commit I guess) as a zip or tarball.

[Sam Whited]

On Wed, Feb 21, 2018, at 14:46, Henrik Johansson wrote:
> But wait. Wasn't there a mention of archive downloads instead of relying on
> the different VCS's?
>
> In the GitHub case I guess it amount to downloading releases (or any commit
> I guess) as a zip or tarball.

My understanding was that the path is still relavant, so you'd need a /v2 directory inside the zip file, which means you probably need it in your source code. However, if that is incorrect it fixes the problem.

—Sam

[Henrik Johansson]

That's not necessarily true.

The tool can understand from the tag and assert or even rewrite the imports to make the compiler happy.

[Russ Cox]

For people who haven't seen it yet: research.swtch.com/vgo-import is the post Sam's referring to.

My plan is that this is in one of tomorrows' post. Sorry for seeming like I'm withholding information. I'm posting as they are finished. :-)

Russ

[Sam Whited]

On Wed, Feb 21, 2018, at 14:57, Russ Cox wrote:
> My plan is that this is in one of tomorrows' post. Sorry for seeming like
> I'm withholding information. I'm posting as they are finished. :-)

Sounds good, I look forward to reading it and continuing to digest the proposal. I've had a hard time coming to grips with all the implications of this proposal (it being a radical departure from the current state of the art), but the detailed posts have been very helpful; thanks for writing about it!

—Sam

[Bakul Shah]

On Wed, 21 Feb 2018 20:53:20 +0000 Henrik Johansson <dahan...@gmail.com> wrote:
>
> That's not necessarily true.
> The tool can understand from the tag and assert or even rewrite the imports
> to make the compiler happy.
>
> On Wed, Feb 21, 2018, 21:51 Sam Whited <s...@samwhited.com> wrote:
>
> > On Wed, Feb 21, 2018, at 14:46, Henrik Johansson wrote:

> > > But wait. Wasn't there a mention of archive downloads instead of relyin=

> g
> > on
> > > the different VCS's?
> > >
> > > In the GitHub case I guess it amount to downloading releases (or any
> > commit
> > > I guess) as a zip or tarball.
> >
> > My understanding was that the path is still relavant, so you'd need a /v2
> > directory inside the zip file, which means you probably need it in your
> > source code. However, if that is incorrect it fixes the problem.

As v2 is only created due to some incompatible change, in my
view it is better to have v2 visible in the path. Otherwise
it is easy for a reader to get confused when foo.F() in one
package (using v1) behaves differently from foo.F() in another
package (using v2) as they both will have the same import path
and only the "bindings" in their respective go.mod change.

[Peter Bourgon]

On Wed, Feb 21, 2018 at 1:10 PM, Bakul Shah <ba...@bitblocks.com> wrote:
> On Wed, 21 Feb 2018 20:53:20 +0000 Henrik Johansson <dahan...@gmail.com> wrote:
>>
>> That's not necessarily true.
>> The tool can understand from the tag and assert or even rewrite the imports
>> to make the compiler happy.
>>
>> On Wed, Feb 21, 2018, 21:51 Sam Whited <s...@samwhited.com> wrote:
>>
>> > On Wed, Feb 21, 2018, at 14:46, Henrik Johansson wrote:
>> > > But wait. Wasn't there a mention of archive downloads instead of relyin=
>> g
>> > on
>> > > the different VCS's?
>> > >
>> > > In the GitHub case I guess it amount to downloading releases (or any
>> > commit
>> > > I guess) as a zip or tarball.
>> >
>> > My understanding was that the path is still relavant, so you'd need a /v2
>> > directory inside the zip file, which means you probably need it in your
>> > source code. However, if that is incorrect it fixes the problem.
>
> As v2 is only created due to some incompatible change, in my
> view it is better to have v2 visible in the path.

·Any· change to the exported API of a package except adding new
top-level identifiers can be considered an incompatible (breaking)
change.

https://blog.merovius.de/2015/07/29/backwards-compatibility-in-go.html

[Sebastien Binet]

Agreed but it seems a bit of a rather gratuitous constraint to require the version to be the last element:

https://github.com/golang/go/issues/23960

I kind of like Gonum's way of versioning:

gonum.org/v1/gonum/stat

gonum.org/v1/gonum/mat

gonum.org/v1/plot/vg/vgpdf

(And, in the future:

gonum.org/v4/plot

gonum.org/v4/gonum/deeplearn

...)

-s

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[Axel Wagner]

On Wed, Feb 21, 2018 at 10:17 PM Peter Bourgon <pe...@bourgon.org> wrote:

·Any· change to the exported API of a package except adding new
top-level identifiers can be considered an incompatible (breaking)
change.

https://blog.merovius.de/2015/07/29/backwards-compatibility-in-go.html

I should probably go on record that I acknowledge that post to be potentially unreasonably nitpicky :)

FWIW, I *believe* that if you only ever release v1.x.x you'll end up at essentially the same space as today, whether you actually end up adding breaking changes or not. Unless vgo somehow tracks API changes and verifies that your versioning is semantic, which I can't imagine working (which was my point then, basically). Of course this makes semantic versions largely useless, which was my other point then.

There is one significant difference, which is that people are currently automatically held back, whereas previously they where automatically updated, so it's somewhat more likely that people will sleep through necessary API changes and then break the build at an unpredictable point in the future.

Personally, that is (with the information currently published) what I intend to do, most likely: Tag all my stuff at v1.x.x (or maybe not tag at all, for a "pin on first use" kinda policy) and just be careful in if and how I break stuff.

[Bakul Shah]

On Wed, 21 Feb 2018 22:24:38 +0100 Sebastien Binet <seb....@gmail.com> wrote:
>
> On Feb 21, 2018 10:11 PM, "Bakul Shah" <ba...@bitblocks.com> wrote:
> >
> > As v2 is only created due to some incompatible change, in my
> > view it is better to have v2 visible in the path. Otherwise
> > it is easy for a reader to get confused when foo.F() in one
> > package (using v1) behaves differently from foo.F() in another
> > package (using v2) as they both will have the same import path
> > and only the "bindings" in their respective go.mod change.
>
> Agreed but it seems a bit of a rather gratuitous constraint to require the
> version to be the last element:

Something like
import "foo" v2
would be mildly preferable to me as
import "foo/v2"
can also refer to version 1 of a package named v2 (as opposed
to version 2 of foo) and the current naming rules continue to
work.

[Zellyn]

On Wednesday, February 21, 2018 at 4:45:25 PM UTC-5, Bakul Shah wrote:

Something like
        import "foo" v2
would be mildly preferable to me as
        import "foo/v2"
can also refer to version 1 of a package named v2 (as opposed
to version 2 of foo) and the current naming rules continue to
work.

As I understood it, the fundamental philosophical point of today's rsc article was that you shouldn't use the the same "name" (identifier) for two different "things". If you look at it from that point of view, the "v2" should absolutely be part of the package URL. In fact, having the concept of version stored in some lookaside metadata outside of the name (eg. git tag) would be a mistake.

Zellyn

[Henrik Johansson]

Why would that be a mistake? Working with tags and branches is what the VCS is for. Extracting a given version and using that per the convention that the import ends with the major version sounds very reasonable.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[Aram Hăvărneanu]

[+cc golang-nuts, it was lost at some point]

I find it unexpected that to compute a version you need information
not from one, but from two places that use very similar but ultimately
different syntax.

In vgo the major version has to be specified in Go import paths,
and the minor version needs to be specified in go.mod file, which
is not Go code but looks very similar to Go code.

Why not just use Go code throughout? We could extend Go syntax in
a backwards-compatible way to be able to express all the information
contained in the go.mod file. The bulk of the code would continue
to import old-style, "bare", imports, while a single, or perhaps a
small number of Go files would encode the information required for
version management using new-style imports. Perhaps this file, or
files could have standardized names, zmod.go, or whatever (please
don't bikeshed the name).

These files could be guarded by build-tags, so old compilers ignore
them, just like how old compilers ignore the go.mod file.

--
Aram Hăvărneanu

[Russ Cox]

> [+cc golang-nuts, it was lost at some point]

BCC golang-nuts. Please do NOT mix the two threads. I expect that the two different lists with different people will have different focuses for their discussions.

[Russ Cox]

Replying to this thread to drop golang-dev. Please try to keep the two threads separate. I should not have used the same subject for each. Sorry.

[David Anderson]

Reading the latest post, https://research.swtch.com/vgo-mvs , a question...

It feels to me like there's a missing 5th operation, in additions to the one you proposed: "upgrade all my direct dependencies to their latest version, but keep using minimal versions below that." I don't believe there is an analog to this operation in the worlds of `go get` or `dep`, but I think it might be an interesting middle-ground. I as the apex module developer want to say "I'd like to use the latest versions of the things I'm programming against, but I don't care about their dependencies – please defer to the library authors's opinions for those."

First of all, am I reading correctly that this is not currently one of the options you described? If so, do you think this additional option has merit, compared to "upgrade the entire transitive closure to the absolute newest things" ?

I'm still completely in love with this overall approach, and so far the only quibbles I might have would be over syntax or equivalently unimportant details. I sincerely hope that this becomes Go's package management solution of choice, I'm already finding it unpleasant to operate in a world where this isn't the way to do things :).

- Dave

On Tue, Feb 20, 2018 at 9:20 AM, Russ Cox <r...@golang.org> wrote:

Hi everyone,

I have a new blog post you might be interested in.

https://research.swtch.com/vgo.

I'll try to watch this thread to answer any questions.

Best,

Russ

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[Axel Wagner]

On Wed, Feb 21, 2018 at 11:55 PM David Anderson <da...@natulte.net> wrote:

Reading the latest post, https://research.swtch.com/vgo-mvs , a question...

It feels to me like there's a missing 5th operation, in additions to the one you proposed: "upgrade all my direct dependencies to their latest version, but keep using minimal versions below that." I don't believe there is an analog to this operation in the worlds of `go get` or `dep`, but I think it might be an interesting middle-ground. I as the apex module developer want to say "I'd like to use the latest versions of the things I'm programming against, but I don't care about their dependencies – please defer to the library authors's opinions for those."

First of all, am I reading correctly that this is not currently one of the options you described? If so, do you think this additional option has merit, compared to "upgrade the entire transitive closure to the absolute newest things" ?

I think this operation is algorithmically trivial: Just bump the versions in A's list to the newest ones available of each direct dependency. I understand Russ' Article to justify that these operations can be implemented efficient and that thus a trivial operation isn't interesting to mention.

If the "import compatibility rule" is true, there shouldn't be a downside to do a full, transitive closure upgrade either and if this is the standard/most common/only way to do upgrades, that would at least build *some* force into the system, to progress through upgrades regularly.

[David Anderson]

On Wed, Feb 21, 2018 at 3:55 PM, Axel Wagner <axel.wa...@googlemail.com> wrote:

On Wed, Feb 21, 2018 at 11:55 PM David Anderson <da...@natulte.net> wrote:

Reading the latest post, https://research.swtch.com/vgo-mvs , a question...

It feels to me like there's a missing 5th operation, in additions to the one you proposed: "upgrade all my direct dependencies to their latest version, but keep using minimal versions below that." I don't believe there is an analog to this operation in the worlds of `go get` or `dep`, but I think it might be an interesting middle-ground. I as the apex module developer want to say "I'd like to use the latest versions of the things I'm programming against, but I don't care about their dependencies – please defer to the library authors's opinions for those."

First of all, am I reading correctly that this is not currently one of the options you described? If so, do you think this additional option has merit, compared to "upgrade the entire transitive closure to the absolute newest things" ?

I think this operation is algorithmically trivial: Just bump the versions in A's list to the newest ones available of each direct dependency. I understand Russ' Article to justify that these operations can be implemented efficient and that thus a trivial operation isn't interesting to mention.

Agreed, from the algorithmic perspective it simplifies down to the existing operation add/remove operations, so the algorithms and properties are unchanged.

What I'm arguing for (or at least, asking about), is whether `vgo get -u` should only set direct dependencies to the latest version, or the entire transitive closure. It's purely a UI/UX question, not an algorithms one.

Currently, per the vgo tour, the entire transitive closure gets updated, which, per the article's definition, results in a low-fidelity build... If you assume that the intent of `vgo get -u` was "the libraries I'm calling have new features/fixes, please use them," and not "I want to use the latest possible code the universe can offer me, throughout the stack."

I believe both operations have their place, and I could be convinced either way on which should be the default for "update my stuff." I'm wondering if I'm alone in thinking that "upgrade only direct dependencies, minimal versions elsewhere" is desirable as a "suggested" commandline action.

- Dave

[dc0d]

I'm looking forward to see this in official releases too!

Also I would like to:

- Have a mechanism for safe dependency packages (as in Safe-Tcl - this implies it would be possible to have meta-data other than versions for packages, too).

- This one looks like a minor change in import syntax and might bring in some security concerns: being able to use env-vars in imports: import "$MY_OTHER_PKG".

- I love managing major versions in git by having different branches for major versions because it makes patching fixes much more logical and cleaner. It would be great to be able to perform go get based on branch name too - the concept is common in major source controls of today.

[dc0d]

And also I would like the $GOPATH to stay. I clean it up from time to time - also $GOPATH/bin which go install uses.

[Axel Wagner]

On Thu, Feb 22, 2018 at 4:21 AM David Anderson <da...@natulte.net> wrote:

Currently, per the vgo tour, the entire transitive closure gets updated, which, per the article's definition, results in a low-fidelity build... If you assume that the intent of `vgo get -u` was "the libraries I'm calling have new features/fixes, please use them," and not "I want to use the latest possible code the universe can offer me, throughout the stack."

I don't think this is the correct interpretation of what Russ calls "low-fidelity". "low-fidelity build" means, that you install something I built and gets a poor reproduction of what I intended you to get. This is different from me upgrading the transitive closure of dependencies of my stuff and you then installing that; this would still be a good reproduction of my intent and thus high-fidelity.

 

I believe both operations have their place, and I could be convinced either way on which should be the default for "update my stuff." I'm wondering if I'm alone in thinking that "upgrade only direct dependencies, minimal versions elsewhere" is desirable as a "suggested" commandline action.

I think it may have its place but it should be strongly discouraged (to the degree that I'm not sure whether I want it to exist at all). I think upgrading the transitive closure can be a valuable driver to prevent bitrot in the Go community at large - but that requires that a critical mass of projects actually use it.

[Axel Wagner]

On Thu, Feb 22, 2018 at 7:10 AM dc0d <kaveh.sh...@gmail.com> wrote:

I'm looking forward to see this in official releases too!

Also I would like to:

- Have a mechanism for safe dependency packages (as in Safe-Tcl - this implies it would be possible to have meta-data other than versions for packages, too).

- This one looks like a minor change in import syntax and might bring in some security concerns: being able to use env-vars in imports: import "$MY_OTHER_PKG".

Please don't. This is the antithesis to what vgo is trying to do. vgo tries very hard to marry reproducible builds with simplicity. Having environment variables determine what gets built destroys that completely.

The occasional edge-case that might benefit from this should write their own tool which generates an appropriate package.

[dc0d]

Thanks Axel,

You are right about env-vars and as I've noted, there might be security concerns. The main goal here was to represent an idea for the source of the import. It can be the module descriptor of vgo.

And the first idea is about having packages that does not harm the environment (like by importing reflect or executing external commands), and seems to be a feasible goal.

[Axel Wagner]

On Thu, Feb 22, 2018 at 9:50 AM dc0d <kaveh.sh...@gmail.com> wrote:

And the first idea is about having packages that does not harm the environment (like by importing reflect or executing external commands), and seems to be a feasible goal.

Apologies, I was ambiguous in my quote, I wasn't trying to say anything about the first point at all.

[jimmy frasche]

Is there a story for a module that changes domains, like when google
code shutdown or moving to a custom domain for vanity import paths?

Having directives to say "find earlier versions here" or "find later
versions" here seems like it would help downgrades/upgrades. If one of
those old domains stops existing entirely the chain would be broken,
certainly, but it still seems like it would be useful.

[Russ Cox]

On Tue, Feb 20, 2018 at 4:07 PM, David Anderson <da...@natulte.net> wrote:

On Tue, Feb 20, 2018 at 11:37 AM, Russ Cox <r...@golang.org> wrote:

On Tue, Feb 20, 2018 at 1:55 PM, David Anderson <da...@natulte.net> wrote:

I love this. I want it now.

go get -u golang.org/x/vgo :-)

I've struggled with `glide` and `dep` dependency hell a lot in my Go projects (especially those that use the Kubernetes client library, which abuses "I, the library author, can define byzantine constraints on my users" to the extreme). The way I've described it informally is that traditional "arbitrary version selection" algorithms create the wrong incentive structure for library authors, in the sense that they have all the power to lock versions arbitrarily, but bear none of the pain associated with their decisions - instead, the binary author at the apex of the dependency tree gets that pain. "Authority without responsibility" is the death of many incentive structures.

Yes, I'll make that point again in tomorrow's posts elaborating minimal version selection, but I think this is probably the most important algorithmic policy detail. You get total control over your own build. You only get limited control over other people's builds that happen to import your code.

Big +1. I'm curious about your ideas on how to manage the transition from a glide/dep world to a vgo world. I look forward to reading more about that, assuming you have an article queued to discuss that facet :).

I don't have an article about this. There are two parts to the answer. The first is that vgo reads all these tools' lock files and records the things listed there are minimum requirements, as a starting point. I expect this will be good enough for many projects. For example, on Tuesday when vgo was first released, before anything had time to convert, you could git clone upspin or dep, cd into the checkout (anywhere, not just in GOPATH), and start working. Vgo seeded go.mod with the Gopkg.lock file. The second part is that, as Sam posted, we are trying to work out a way to contract him to build tools to help with more complex conversions.

One thought that I hope you'll cover there: immutability. Reading through the vgo tour, it seems to assume that once v5.6.7 has been released, that .zip and its declared dependencies will never change.

You've probably seen this by now, but: research.swtch.com/vgo-repro.

Best,

Russ

[Russ Cox]

On Tue, Feb 20, 2018 at 5:14 PM, Dave MacFarlane <dri...@gmail.com> wrote:

I really like this, except for the claim that it the blog post that it
will eliminate vendoring and deprecate GOPATH and the problems that
will cause for backwards compatibility for things that are currently
using them. If this is going to result in removing language features
(ie. vendoring), shouldn't it be in Go2, not Go 1.11?

The proposal is only to remove vendoring when working with modules.

Non-module repos with vendor directories will keep building at least past Go 1.11,

perhaps even forever.

 

The other question I have is: where does the module name come from if
you don't have a "// import" comment on the package, since it can't be
filesystem based (because you're explicitly outside of GOPATH..)

It comes from go.mod. It only has to come from somewhere else if go.mod is missing. That was a cute trick to ease transition more than anything else. The places that vgo looks to try to guess a module path when filling in the initial go.mod are:

    - // import comments in the top-level directory or immediate subdirectories

    - Godeps/Godeps.json

    - vendor/vendor.json

    - .git/config

Perhaps this is not worth keeping in the real integration, but it was a nice way to get off the ground.

Best,

Russ

[David Anderson]

This is an experience report onboarding vgo for a relatively complex project (multiple binaries, vast import tree, tricky unidiomatic imports). I leave it here in the hopes that it guides other people, and potentially illustrates places where vgo falls short of great. TL;DR it went pretty well, modulo a couple of UX speed bumps.

The result is the `vgo-test` branch of https://github.com/google/metallb/ . Cloning that repository should allow you to `vgo test ./...`, `vgo build ./controller`, `vgo build ./speaker`, and a few others. I don't make any representation that the code does what it should, merely that it builds and passes tests.

The resultant go.mod, https://github.com/google/metallb/blob/vgo-test/go.mod , is quite messy. This is mostly due to the number of dependencies that have no semver at all, forcing vgo to use commit hash "versions". The result is a lot of visual noise in the file, but hopefully that will improve over time as both vgo and dep nudge people towards semver releases.

I encountered two major roadblocks on the road to `vgo test ./...`: the Kubernetes client library, and mixed case packages. These are https://github.com/golang/go/issues/24032 and https://github.com/spf13/jWalterWeatherman/issues/22 respectively.

The Kubernetes client library is a worst case scenario for vgo. It releases a new major version every 3 months under the same module name, with real incompatibilities between versions; and it relies extensively on a transitive version lock to force a specific package selection on its dependents. Making this library usable from vgo required the following:

• Fix up client-go in a fork
• Fork github.com/kubernetes/client-go to github.com/danderson/client-go
• Add a go.mod to the fork, containing only: module "k8s.io/client-go/v6"
• Update all internal self-references within client-go: perl -pi -e 's#"k8s.io/client-go#"k8s.io/client-go/v6#g' **/*.go
• Commit the result, https://github.com/danderson/client-go/commit/7d8481153a9edbf8eacc176ce29f8d58949c3a77 . Tag as v0.0.0-vgo-prototype-compat and push.
• Make my repository use my fork of client-go:
• Update all uses of client-go to the new versioned package name: perl -pi -e 's#"k8s.io/client-go#"k8s.io/client-go/v6#g' **/*.go
• Require the right version in mod.go: require "k8s.io/client-go/v6" v6.0.0
• Replace upstream with my fork in mod.go: replace "k8s.io/client-go/v6" v6.0.0 => "github.com/danderson/client-go" v0.0.0-vgo-prototype-compat

I'm curious how we could upstream the changes I had to make to client-go. I had to rename module-internal imports, which will break existing non-vgo uses of the library, but vgo offers no alternative to this renaming that I'm aware of. It looks to me like there's no graceful upgrade path for this module. The repo structure works either for pre-vgo clients, or for vgo, but not both at once.

The UX was lacking to explain the reason for failures, before I made any of the changes. Given just my repository, vgo inferred that I wanted v1.5.2 of client-go (3+ years old), continued resolving dependencies, and eventually barfed when it found an import for a client-go subpackage that didn't exist in 1.5.2. The error was simply a bunch of "no such file or directory", and required some head-scratching to understand what had happened. Once I understood why vgo was making that decision, and how to correct it, vgo provided the right tools (mostly replace-with-fork) to correct the issue without waiting on the library to fix itself.

The second issue I hit is github.com/spf13/jwalterweatherman. The actual repository name in github is "jWalterWeatherman", but everyone (including the package's author :) ) imports it lowercase. vgo disallows this casing mismatch.

To solve that, I just added a require + replacement to "fix" the case: replace "github.com/spf13/jwalterweatherman" v0.0.0-20180109140146-7c0cea34c8ec => "github.com/spf13/jWalterWeatherman" v0.0.0-20180109140146-7c0cea34c8ec

The version to use in that replacement was hard to come by. Since vgo was unable to compute the right dependency list without this replacement, it bailed without giving me any version I might use. The date+hash format is irritating to construct by hand, but could be trivially lifted into a tool (perhaps even vgo itself). Instead, I opted to create a separate dummy module, with a dummy main.go that imported and used the camelCased package, and run vgo on that. This produced a tiny go.mod with a constructed commit hash version, which I then used in MetalLB's go.mod. It would have been nice for vgo to help me a bit more there.

- Dave

On Tue, Feb 20, 2018 at 9:20 AM, Russ Cox <r...@golang.org> wrote:

Hi everyone,

I have a new blog post you might be interested in.

https://research.swtch.com/vgo.

I'll try to watch this thread to answer any questions.

Best,

Russ

[David Anderson]

I should add: even though I spent several hours working through these issues, I feel very happy about the process and outcome. I have high confidence that I understand what vgo is doing with my module spec, and that I know how to tweak its thinking in future. I would contrast this with my experience with Glide. Last weekend, I tried to `glide up` this same project, and spent 4 hours trying to resolve the resulting mayhem of diamond dependencies and build errors. I failed, rolled back the change, and decided MetalLB could stay on old code for a while longer. I still have very low confidence that I understand what glide will do when I `glide up`, or that it will produce a working result.

Additionally, some of the time spent is likely the learning curve. As https://github.com/golang/go/issues/24032 illustrates, I was initially confused and had to re-read some of rsc's writing to formulate my plan of action. Despite that, I still spent less time end to end than with glide, and I had a working build at the end of it.

- Dave

[Henrik Johansson]

A sidenote is that there seems to always be issues with using kubernetes client.

Is it structured very un Go-ish?

Very nice writeup!

NB: I was also hit by the casing issue with the same viper dependency.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[Nicolas Grilly]

On Fri, Feb 23, 2018 at 8:20 AM, David Anderson <da...@natulte.net> wrote:

The date+hash format is irritating to construct by hand, but could be trivially lifted into a tool (perhaps even vgo itself).

Very true. I suffered from the same issue while trying vgo. This could be improved to ease the adoption of vgo by projects with many dependencies that are not already compatible with vgo semver.

[Nic Pottier]

Thanks for the great write up Dave, you've inspired me to "try harder" to use it on a few of my projects after running into similar issues.

As you noted, my remaining big question mark is how existing libraries that are above v1 transition to a vgo style while remaining backwards compatible during the transition. That seems like the biggest wrench to adoption. (if we can convince everyone to start using v# in import paths) I was preparing to open some PRs with dependencies of mine to introduce go.mod files, but as I understand it that won't be enough for those above v1.

[roger peppe]

On 23 February 2018 at 12:31, Nic Pottier <nicpo...@gmail.com> wrote:
> Thanks for the great write up Dave, you've inspired me to "try harder" to
> use it on a few of my projects after running into similar issues.
>
> As you noted, my remaining big question mark is how existing libraries that
> are above v1 transition to a vgo style while remaining backwards compatible
> during the transition.

As Russ pointed out to me, you can work around that by using a
0.0.0-20000101234543-4f34f456eeefdcba version in your go.mod require
section. If you've got that, it ignores the version tags.

[Zellyn]

Fantastic post. One small answer below:

On Friday, February 23, 2018 at 2:22:32 AM UTC-5, David Anderson wrote:

The version to use in that replacement was hard to come by. Since vgo was unable to compute the right dependency list without this replacement, it bailed without giving me any version I might use. The date+hash format is irritating to construct by hand, but could be trivially lifted into a tool (perhaps even vgo itself).

git log -1 ae16178c028329cb0b979cd63bbd3921ddaf0945 --date='format:%Y%m%d%H%M%S' --format='v.0.0.0-%cd-%h'

v.0.0.0-20180222211942-ae16178c02

Zellyn

 

[Nic Pottier]

On Fri, Feb 23, 2018 at 7:59 AM, roger peppe <rogp...@gmail.com> wrote:
> As Russ pointed out to me, you can work around that by using a
> 0.0.0-20000101234543-4f34f456eeefdcba version in your go.mod require
> section. If you've got that, it ignores the version tags.

Right, that works for clients of the library (and I did manage to get
my projects working with that and it now feels magical) but authors of
modules above v1 are posed with a bit of a quandary as to how to give
their users a nice vgo version without breaking current users on plain
go.

From Russ's latest post I think the answer is you put a v3 subdir in
your repo with a copy of the code, add a mod.go and update both the
root and v3 until vgo becomes adopted everywhere.

-Nic

[roger peppe]

Good point. Ugh.

[Zellyn Hunter]

On Fri, Feb 23, 2018 at 10:01 AM Nic Pottier <nicpo...@gmail.com> wrote:

Right, that works for clients of the library (and I did manage to get
my projects working with that and it now feels magical) but authors of
modules above v1 are posed with a bit of a quandary as to how to give
their users a nice vgo version without breaking current users on plain
go.

 Why not just commit a mod.go with v3 in it at the root, and tag it 3.{y+1}?, assuming you're already on v3.y? Adding mod.go doesn't break existing users. vgo users will use different import paths. If you like, you can branch off of your 1.x tag, add a mod.go with v1 in it there, and tag it 1.{x+1}. Same for v2.

Oh wait, is it a problem because your self-import paths need to change (or not) depending on whether the client is using vgo? I think I'm unclear on that detail :-)

Zellyn

[Nic Pottier]

Right, any self import needs to change for the vgo case. (but not for
the go case unless you force everybody currently using your library to
start using semantic versioning paths)

-Nic

[Jon Calhoun]

Not ideal, but you *could* release a new major version where the "breaking" change is the import path change. Eg if you are on v3 now, make a v4 folder and the upgrade to v4 process would involve updating import paths. I wouldn't do this right away since vgo is just a prototype and could change to handle this situation better, but it is an option. 

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/jFPz5yZCPcQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts+unsubscribe@googlegroups.com.

[Richard Wilkes]

Overall, I really like the direction vgo is headed.

One thing that seems to be difficult, if not impossible to do (I've yet to figure out how to do so, anyway) is to work with code that has yet to be committed and resides outside of your module. This is a frequent occurrence for me, as I often have to work on groups of changes that interact between multiple repos. Yes, I'll eventually check those changes in, but usually not while actively working on the problem.

Related to this, it took a few minutes to realize that you really can't "downgrade" to a commit-level entry with the tool when a tag is present in the repo -- you have to manually edit it to get the desired behavior. I put downgrade in quotes because what I was actually trying to do was to get the actual latest (HEAD) code, but could not without manually futzing with the go.mod file. This is also something that I frequently need to do -- and I'm relatively sure I'm not alone -- so it seems like the tool should make this easy, not hard.

Despite the two problems above, an inability to work with GitHub Enterprise (filed in a bug report), and concern over how IDEs will work with library source, I'm already enjoying working with vgo.

- Rich

[David Anderson]

On Fri, Feb 23, 2018 at 3:34 PM, Richard Wilkes <raw.so...@gmail.com> wrote:

Overall, I really like the direction vgo is headed.

One thing that seems to be difficult, if not impossible to do (I've yet to figure out how to do so, anyway) is to work with code that has yet to be committed and resides outside of your module. This is a frequent occurrence for me, as I often have to work on groups of changes that interact between multiple repos. Yes, I'll eventually check those changes in, but usually not while actively working on the problem.

I think a local replace will do that. Something like: replace "foo.com/my/pkg" v1.5.2 => ../local-clone-with-changes

Put that in the top-level module, and it should override that import throughout the build with your local copy, pending changes and all. Or is that not what you were trying to do here?

 

Related to this, it took a few minutes to realize that you really can't "downgrade" to a commit-level entry with the tool when a tag is present in the repo -- you have to manually edit it to get the desired behavior. I put downgrade in quotes because what I was actually trying to do was to get the actual latest (HEAD) code, but could not without manually futzing with the go.mod file. This is also something that I frequently need to do -- and I'm relatively sure I'm not alone -- so it seems like the tool should make this easy, not hard.

Despite the two problems above, an inability to work with GitHub Enterprise (filed in a bug report), and concern over how IDEs will work with library source, I'm already enjoying working with vgo.

- Rich

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[Richard Wilkes]

Ah… yes, that would probably let me work on it. Didn’t occur to me — thanks for pointing it out. It seems unpleasant to deal with, though, especially since there is a non-zero chance that you then commit your go.mod file with that change inadvertently.

[Aram Hăvărneanu]

> Ah… yes, that would probably let me work on it. Didn’t occur to me
> — thanks for pointing it out. It seems unpleasant to deal with,
> though, especially since there is a non-zero chance that you then
> commit your go.mod file with that change inadvertently.

See my proposal here:
https://github.com/golang/go/issues/23972#issuecomment-368077927

--
Aram Hăvărneanu

[David Collier-Brown]

Some forms of attempted versioning don't do well with vgo (:-))

I tried a main named "consumer" that imported "gopkg.in/Shopify/sarama.v1", and got

``` 

[davecb@miles consumer]$ vgo build

can't load package: import cycle not allowed

package gopkg.in/Shopify/sarama.v1

imports gopkg.in/Shopify/sarama.v1

import cycle not allowed

package gopkg.in/Shopify/sarama.v1

imports gopkg.in/Shopify/sarama.v1

 

[davecb@miles consumer]$ cat go.mod 

module "gopkg.in/Shopify/sarama.v1"

```

The main.go started with 

```

package main 

import (

samara "gopkg.in/Shopify/sarama.v1"

"os"

"log"

"os/signal"

)

func main() {

consumer, err := samara.NewConsumer([]string{"10.5.70.111:9092"}, nil)

if err != nil {

panic(err)

}

```

I didn't expect it to work, but the nature of the error might be interesting 

--dave

[bit...@gmail.com]

Hi Russ, I had a question about this:

> We expect that most developers will prefer to follow the usual “major branch” convention, in which different major versions live in different branches. In this case, the root directory in a v2 branch would have a go.mod indicating v2, like this:

It seems like there's subdirectories and this major branch convention that are both supported by vgo. In my anecdotal experience no repositories follow this convention in Go or other languages (can't actually think of a single one other than the ones forced to by gopkg.in which seems relatively unused these days). Master branch is whatever latest is and has v2.3.4 tags in it's history. Tags exist to separate everything (not just minor versions). If it's necessary to patch an old version, a branch is temporarily created off the last v1 tag, commits pushed, a new tag pushed, and the branch summarily deleted. There is no branch for versions, it's just current master/dev/feature branches + version tags.

Having said that, I've tested the above described workflow with vgo (just having tags that say v2.0.0, v2.0.1 and no branches) and it does seem to work (though as an aside it is pretty confusing/annoying to have to change the require statement to say both /v2 as well as v2.0.0 when you upgrade the dependency in go.mod, not sure why that's required). So my question is: Although this works now, is it intended? As it doesn't seem as thoroughly described as the other two workflows in the blog, and I want to ensure that working without a v2/v3... branch is not accidental functionality that will disappear since as I explained above I've never seen this (or the other) described workflow in the post to be massively adopted by anyone (especially outside the Go community).

Of course my argument is coming down to preference and anecdotes, so I'd be willing to do some repo-scraping to prove this across all languages if needed. So far I've really liked the proposal posts and am generally on board with the changes, will continue to follow along and play with vgo.

Thanks for all your efforts.

Aaron

[Sean Russell]

Russ,

I'm working on adding support for Bitbucket, per your comment in #23950.  I'm struggling a bit with executing the tests in x/vgo -- should I expect vgo test to work?  I feel like I'm missing something obvious.

I added unit tests first and they're failing (as expected) but not for the reasons I'd expect:

➜  vgo git:(master) ✗ vgo test ./vendor/cmd/go/internal/modfetch/              
vgo: import "github.com/serussell/vgo/vendor/cmd/go/internal/modfetch" ->
        import "cmd/go/internal/modconv" [/usr/local/go/src/cmd/go/internal/modconv]: open /usr/local/go/src/cmd/go/internal/modconv: no such file or directory

Many similar messages follow this. Why is it looking in GOROOT?  The project directory structure still follows the legacy $GOPATH/src/github/serussell/vgo pattern.

I'm hacking on fetch.go, but I'd feel more comfortable if I could execute the unit tests.

Thanks,

--- SER

[Maxim Ivanov]

I am replying mostly to https://resea rch.swtch.com/vgo-import article.

It is proposed that "moauth" author should do double work, even if his code works absolutely fine with either verion of oauth. This viral effect will be causing chain reaction and a lot of people will be creating unnecessary major semver releases. "moauth" author doesn't need to do anything, but once vgo is out , he'll be receiving requests from "aws" to do theese releases and "aws" itself will be blocked on somebody else without a way to move forward with "oauth/v2".

It will also cause combinatorial explosion, just imagine if "moauth" depends on "oauth" and "samlauth", would it need to release 4 major semver versions for all combinations of dependencies visible in API? Looks like it is inevitable, as users of "moauth" can decide to use types from either version of "oauth" and "samlauth".

IMHO right solution vgo should promote is to apply semver trick described here: https://github.com/dtolnay/semver-trick/blob/master/README.md . In a nutshell it suggests that "oauth/v1" imports "oauth/v2" and type aliases all common types visible in API. Then "moauth" doesn't need a new semver release at all, it stops being viral and whole dependency chain becomes much much simpler.

Go community is going to to do pretty much whatever Russ tells them with imminent vgo release, if smever trick indeed solves this viral problem, it should be encouraged from day 0.

[Spencer Nelson]

I'd like to better understand how multiple-module repositories work if tagged commits are used for minor and patch versioning.

My understanding is that major version increments are controlled with import paths, while minor and patch version increments are controlled with commit tags.

Is it possible for me to have different minor/patch versions for different modules in one repository? For example, suppose I have this:

    github.com/my/thing/go.mod

    github.com/my/thing/blue/go.mod

And suppose my current commit is tagged with v1.0.0.

Now I fix a bug in github.com/my/thing/blue. How do I indicate that github.com/my/thing/blue is at v1.0.1 without changing github.com/my/thing's version?

[ivanov...@gmail.com]

The proposed rules around semver and minimal version selection address every pain point I've had so far, by aligning available authority with the responsibility of the library authors: tell me what versions definitely will not work, and I'll take it from there.

Maybe I miss something, but it is exactly what vgo doesn't let library authors to do. Sometimes "too new" is definitely not going to work for multiple reasons, but there is no way to convey that knowledge to library users.

[Sam Whited]

On Sat, Feb 24, 2018, at 21:44, Sean Russell wrote:
> I'm working on adding support for Bitbucket, per your comment in #23950

> <https://github.com/golang/go/issues/23950>. I'm struggling a bit with

> executing the tests in x/vgo -- should I expect vgo test to work? I feel
> like I'm missing something obvious.

FYI, there is an active CL for this here: https://golang.org/cl/95578

—Sam

[Alex Efros]

Hi!

On Fri, Feb 23, 2018 at 06:23:33PM -0800, Spencer Nelson wrote:
> Is it possible for me to have different minor/patch versions for different
> modules in one repository? For example, suppose I have this:
>
> github.com/my/thing/go.mod
> github.com/my/thing/blue/go.mod
>
> And suppose my current commit is tagged with v1.0.0.
>
> Now I fix a bug in github.com/my/thing/blue. How do I indicate that
> github.com/my/thing/blue is at v1.0.1 without changing
> github.com/my/thing's version?

Russ already answered this in one of articles - use tag "blue/v1.0.1".

--
WBR, Alex.

[Sean Russell]

On Sunday, February 25, 2018 at 1:08:31 AM UTC-5, Sam Whited wrote:

FYI, there is an active CL for this here: https://golang.org/cl/95578

Thanks Sam -- I found it, and it's working fine.

I'm still curious as to what I'm doing wrong in executing the tests in x/vgo.

--- SER 

[David Collier-Brown]

On 24/02/18 04:05 PM, Maxim Ivanov wrote:
> I am replying mostly to https://resea rch.swtch.com/vgo-import article.

> ...

> IMHO right solution vgo should promote is to apply semver trick
> described here:
> https://github.com/dtolnay/semver-trick/blob/master/README.md . In a
> nutshell it suggests that "oauth/v1" imports "oauth/v2" and type
> aliases all common types visible in API. Then "moauth" doesn't need a
> new semver release at all, it stops being viral and whole dependency
> chain becomes much much simpler.

This is one of the two approaches that the Multics folks used when they
ran into the version-skew problem in a heavily versioned collection of
shared libraries.

They wrote "updaters" and "downdaters", that respectively took an old
call and called the new code, or took a new call and mapped it into old
code.

Also used in Solaris and inside Linux glibc, so we've independently
re-invented it at least four times (;-))

> Go community is going to to do pretty much whatever Russ tells them
> with imminent vgo release, if smever trick indeed solves this viral
> problem, it should be encouraged from day 0.

I think the mechanics of doing multiple versions deserves a blog
article, as it's been touched on in the group a couple of times.

--dave
[I also think Russ was just using Mo writing extra libraries as an
illustration]
[see also "Dependency hell is NP-complete" at
https://research.swtch.com/version-sat and
“DLL Hell”, and avoiding an NP-complete problem, at
https://leaflessca.wordpress.com/2017/02/12/dll-hell-and-avoiding-an-np-complete-problem/
]

--
David Collier-Brown, | Always do right. This will gratify
System Programmer and Author | some people and astonish the rest
dav...@spamcop.net | -- Mark Twain

[David Collier-Brown]

I'm following the discussion and using the blog descriptions, but where are the "man pages"?

A search at golang.org for vgo only points me to /src/crypto/x509/root_darwin_armx.go (:-))

Seriously, though, I'll happily help develop and/or edit reference material: I was an O'Reilly author in a previous life.

--dave

[David Collier-Brown]

And I later found https://godoc.org/golang.org/x/vgo/vendor/cmd/go, which looks like where you'll be developing

new material from the old docs.

--dave

[Robert Carlsen]

Before with the naive GOPATH workflow and also with vendoring, it was super easy for me to debug problems with and contribute to upstream packages.  With vgo, I can't just cd into vendor/github.com/foo/bar to test out fixes for bugs, etc.  Without vendoring I could just cd into $GOPATH/src/github.com/foo/bar, test a patch - commit, push - and submit a PR to the 3rd party project.  With vgo, I now have to fork+clone that repository, modify my go.mod file to point to my fork, patch my clone, commit and push to my fork before I can rebuild and test the patch.  Yuck.

[Daniel Theophanes]

Hi Robert, you still can do that. Just add a "replace" directive in the local go.mod.

Please see https://research.swtch.com/vgo-tour for more details.

[Robert Carlsen]

I read through quite a bit of those docs, but somehow missed that - thanks Daniel.

[Göcs Jëss]

Is it possible to restrain of having 2 files in my workspace/project while keeping 2 versions of the same repo?

[thepud...@gmail.com]

Hi Göcs,

Could you expand on your question, including a bit more about the scenario you are in, what you are seeing currently, and what you would like to see instead?

It would probably help to include a simplified example that is representative of your question or issue. For example (and making something up):

  "I have module 'foo' located in a remote git server, and the latest commit is tagged v1.2.3. I just did a fresh git clone to /tmp/scratchpad/foo, and now I want to do ... but instead I am seeing ..."

Also, it might be good to start a new thread with a new subject, which will make it easier for people to find your question. (This thread has many people on it, and was primarily about reaction to the initial versioned modules blog series from back in February).

Best,

--thepudds