x/vgo: instructions for package maintainers?

[robfig]

Hi all,
I maintain a couple packages for which I'm interested in providing vgo support. Despite the large amount written about it, It wasn't that easy to find instructions on exactly what to do.

From what I understand:
1. go get -u golang.org/x/vgo
2. vgo build
3. Commit the resulting go.mod file
4. Tag that commit with a semantic version

Is that correct / comprehensive? Is using "v1.0.0" for a stable project the expected thing to do? I haven't previously used any versions, just tried to maintain backwards compatibility.

As a bit of feedback, I think it might help adoption if there were authoritative articles where maintainers or consumers can go to get started, or maybe a "vgo help" module. As more unsolicited feedback, vgo seems like a huge advance for Go, the "different version at different import path" struck me as an elegant and best solution, and I'm glad that Russ & the Go team had the inventiveness to design it and fortitude to realize it despite the detractors. I would be very happy to see it rolled out and adopted more widely.

Thank you,
Rob

[Fred Simon]

From my experience:

After the go.mod is created, you may need to verify the tags used in your dependencies.

Many time some old tags with semantic versioning exists in projects, and the latest tag will be chosen instead of the master branch that you may need/use.

If some tag or specific version exists, it may be a good idea to lock it also in your go.mod.

[Peter Waller]

I agree with Rob's call for instructions-for-maintainers. It really isn't clear, even to me as someone having spent a bit of time exploring vgo.

First question: what should the instruction to users be to obtain my package? For example, github.com/pwaller/docker-show-context - in the old days, I would say `go get github.com/pwaller/docker-show-context`, but the "obvious thing", `vgo get github.com/pwaller/docker-show-context` doesn't work. It complains about a lack of a go.mod in the current working directory.

In the old days it was necessary to put your package in a specific place ($GOPATH/<url>), so go get helped a lot for creating those directories under $GOPATH. I guess this is less necessary, but it still feels like a pretty good practice,

Should I update the instructions to be "clone the repository to wherever you feel like it, then run `vgo install`"?

Or is it intended that `vgo get github.com/pwaller/docker-show-context` should work?

[Peter Waller]

+Bryan C. Mills, any thoughts?

What should my instructions be to a user to let them install github.com/pwaller/docker-show-context ?

My best guess:

- first, obtain vgo

- then, run `vgo install github.com/pwaller/docker-show-context`

But doesn't work, because vgo expects to be run from a module, is that correct?

So second best guess:

- obtain vgo

- `git clone https://github.com/pwaller/docker-show-context`

- `cd docker-show-context`

- `vgo install`

Is there a command which will install a main package with no other environment/workdir setup? If not, can there be one?

[Sameer Ajmani]

We indeed need an integration guide for vgo, both for package maintainers and end users. One of the important cases to document is what maintainers must do when they have packages that are already tagged with a major version greater than 1, as vgo requires the module import path for such packages include the version number. Related to this is documentation on module size and granularity, as modules can be as small as a single Go package or as large as a repository.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bryan C. Mills]

On Fri, Jul 13, 2018 at 9:08 AM Sameer Ajmani <sam...@golang.org> wrote:

We indeed need an integration guide for vgo, both for package maintainers and end users. One of the important cases to document is what maintainers must do when they have packages that are already tagged with a major version greater than 1, as vgo requires the module import path for such packages include the version number.

We might not require the import path to include the version number for existing major-version tags: it's an open question.

(See discussion in https://golang.org/issue/26238.)

 

Related to this is documentation on module size and granularity, as modules can be as small as a single Go package or as large as a repository.

I expect that module size and granularity will be one of the more interesting things we learn from the 1.11 module experiment.

[Bryan C. Mills]

On Fri, Jul 13, 2018 at 7:39 AM Peter Waller <pe...@pdftables.com> wrote:

+Bryan C. Mills, any thoughts?

What should my instructions be to a user to let them install github.com/pwaller/docker-show-context ?

My best guess:

- first, obtain vgo

We don't really want `vgo` to be the long-term name of the command, so step 1 is probably more like:

- first, build the Go toolchain from head

(Soon, that will hopefully become “install Go 1.11beta2”.)

- then, run `vgo install github.com/pwaller/docker-show-context`

But doesn't work, because vgo expects to be run from a module, is that correct?

`go get github.com/pwaller/docker-show-context` should continue to work as it used to.

Installing at a specific version, or installing without `go get`, is a more interesting question: at some point you might expect `go get github.com/pwaller/docker-sh...@v1.1.0` or `go install github.com/pwaller/docker-sh...@v1.1.0` to work, without downloading code to $GOPATH or requiring a go.mod file in the working directory.

That's https://golang.org/issue/24250, which is still open (and labeled NeedsDecision).

[Paul Jolly]

> What should my instructions be to a user to let them install github.com/pwaller/docker-show-context ?

Good question.

I think an important part of the answer to this question is whether
the user is in module mode or not (see
https://groups.google.com/d/msg/golang-dev/a5PqQuBljF4/61QK4JdtBgAJ)

And part of understanding that is the extent to which users will
know/care about module mode.

During the "transition" that is Go 1.11, I think it will be a somewhat
nuanced answer therefore.

But I agree, having a "template" of that nuanced answer will be
useful. But I think that needs the discussion around GO111MODULE to
settle first.


Paul

[thepud...@gmail.com]

Hi all,

As has been pointed out in this thread, there has been much written about vgo / versioned Go modules, but it is a little tough to know what the *current* advice is on how to use it.

As an example, if I just read this relatively recent thread here including, I would think "OK, Bryan Mills recommends the best thing to do currently is build from tip" (where he made that recommendation in this thread exactly one week ago).

However, with new release of 1.11 beta2, I suspect the official advice might now be switching to "use beta2"?

BUT... I don't really know if that's true.

All of this is currently a moving target and official docs are still being created and/or polished, and hence it's not easy for a member of the Go community to synthesize that topic A from blog post B from 2 months ago was overridden by github issue C, but that was then later overridden by google group post D from 4 weeks ago and github issue E from 2 weeks ago.

I think something that would help would be a very brief wiki page that covered:

  * a very, very brief introduction

  * a statement of which software is currently recommended as the typical choice for an early adopter (vgo vs. tip vs. betaN vs. ___)

  * a few pointers to other resources for obtaining additional information

I know vgo and versioned Go modules are still targeted at early adopters, but especially in light of some of the "fractious" discussions and comments within the community during the proposal process, slightly smoothing things out a little more for the early adopters might lead to a larger number of "happy" tweets/blog posts/comments regarding early vgo/versioned modules experience from those early adopters (where those early adopters are of course influencers on the viewpoints that then spread within the larger community, including influencing some viewpoints of "late adopters" who often have not have looked at any of this in detail yet).

As official doc gets better (and includes documents or sections specifically targeted at package maintainers vs. end users), this wiki page could get progressively smaller with pointers to the official doc (probably until this wiki page hits a terminal state of something close to "Go 1.11 has been released! Please see official introductory documentation _here_".)

A wiki page of course has an advantage that it is easier to update, and people's expectations are of course lower regarding level of polish compared to the official doc (and no one would raise an eyebrow at a wiki page pointing to a Google Groups post or Dave Cheney blog post for details, but that might be odd in official doc...).

And I'm not complaining -- I know this is a complex topic, a big project, currently a bit of a moving target, software sometimes gets ahead of 'introductory' docs, etc., etc.

--thepudds

[Ian Lance Taylor]

Thanks. This is something we need to aim for for the 1.11 release.
If you don't want to keep up with all the changes, which I agree is
very hard and I'm not trying to do it myself, then please wait for the
1.11 release. And even in 1.11 the versioning support is going to be
experimental, but I hope that at least it will have some reasonable
documentation.

Ian

[thepud...@gmail.com]

Hi Ian, all,

I'm sure the official docs for the 1.11 release will make all of this much easier to consume, but I still suspect there is an opportunity to help the early adopters with a lightweight wiki page?

In the interests of possibly helping others, I tried just now to put together a DRAFT / WIP of a possible wiki page that I just recommended in my prior post to this thread ...

I'm not an expert here, but rather this is the type of info I'd personally like to see, and I suspect might help others (IF someone more knowledgeable than me was to tweak, correct, and/or completely re-write this draft ;-) and hopefully it is lightweight enough that it is not too burdensome to create?

The biggest thing I'm actually making up in this draft below is the advice of whether or not right now it is better to use vgo vs. tip vs. beta2, but I'm hoping someone could comment on that.

The points #1 and #2 in 'Advice for package maintainers' section below are taken from (slightly re-arranged) conclusion of https://research.swtch.com/vgo-module.

Much of rest of the text in 'Advice for package maintainers' section is cribbed from tip documentation, and obviously what I put in this draft below is much (much!) shorter than the full doc, but my section below ends with a pointer back to tip documentation for full documentation. The intent of including that material is to give someone a flavor of what they would need to do, which hopefully leads to a feeling of "this doesn't seem to be too burdensome; let me try this out or learn more"... but without overwhelming them with all the details (and the full doc can cover more details).

In general, I tried to sprinkling in a few phrases along lines of "as of date X" to slightly help a future reader be more suspicious of older info (and to slightly increase the odds of triggering someone to delete stale info or update with more recent info).

In any event, here is a quick/rough DRAFT of a possible wiki page for comments (including perhaps "we don't need this", or whatever other feedback people might have):

###########################################################################

== Advice for early adopters of vgo / versioned Go modules ==

=== Introduction ===

Package versioning in Go is a rich topic, and this wiki page is intended to just give:

 * a very brief introduction

 * a statement of which software is currently recommended as the typical choice for an early adopter (vgo vs. tip vs. betaN, etc.)

 * a few pointers to other resources for obtaining additional information

The recent work by the Go team on versioned Go modules started outside of the main repository with the vgo tool, but as of 2018-07-12 support for versioned Go modules has landed in the official main go repository [1]. 

As of 2018-07-12, there are various minor known issues, but quite a lot works very well. 

[1] https://groups.google.com/forum/#!topic/golang-dev/a5PqQuBljF4

=== Which software should I use? ===

Now that modules have landed in the main go repository, vgo is no longer recommended. 

As of 2018-07-20, the recommendation instead is to please _get 1.11 beta2_ [2].  Alternatively, you can build from tip. [3]

[2] https://groups.google.com/forum/#!topic/golang-dev/A6TCp2kCoss 

[3] https://golang.org/doc/install/source

=== Brief advice for package maintainers ===

The most important pieces of advice for package mainters:  

1. Please start tagging your packages with release tags that follow semantic versioning

2. Add go.mod files if that makes sense for your project. Today's go.mod files will be understood by any future tooling.

Go 1.11 includes experimental support for Go modules, including a new module-aware 'go get' command. We intend to keep revising this support, while preserving compatibility, until it can be declared official (no longer experimental).

The quickest way to take advantage of the new Go 1.11 module support is to check out your repository into a directory outside GOPATH/src, create a go.mod file, and run go commands from within that file tree.

To start a new module, simply create a go.mod file in the root of the module's directory tree, containing only a module statement. The 'go mod' command can be used to do this:

go mod -init -module example.com/m

In a project already using an existing dependency management tool like godep, glide, or dep, 'go mod -init' will also add require statements matching the existing configuration.

Once the go.mod file exists, no additional steps are required: go commands like 'go build', 'go test', or even 'go list' will automatically add new dependencies as needed to satisfy imports.

For more information, please see the full godoc documentation at tip [4] and the output of 'go help modules'. 

[4] https://tip.golang.org/cmd/go/#hdr-Modules__module_versions__and_more

== Some Additional Resources ==

An important short blog from Russ Cox that summarizies how versioned modules are currently an experimental opt-in feature for go 1.11 (2018-05-29):  https://research.swtch.com/vgo-accepted

The initial "Go & Versioning" series of blog posts by Russ Cox: https://research.swtch.com/vgo

Current tip documentation: https://tip.golang.org/cmd/go/#hdr-Modules__module_versions__and_more

Introductory blog post on 'Taking Go Modules for a Spin' by Dave Cheney (2018-07-16): https://dave.cheney.net/2018/07/14/taking-go-modules-for-a-spin

Introductory blog post on how to build go from tip and try go modules by Carolyn Van Slyck (2018-07-16): https://carolynvanslyck.com/blog/2018/07/building-go-from-source/

Official 'Versioned Go Modules' Proposal (last updated 2018-03-20): https://github.com/golang/proposal/blob/master/design/24301-versioned-go.md

_List of open Go modules bugs_

_Submitting a new Go modules bug_

###########################################################################

END DRAFT

###########################################################################

--thepudds

[thepud...@gmail.com]

Hi all,

Here is slightly updated draft of a possible new wiki page.  (Only relatively minor deltas from the first draft: corrected a couple typos; moved some of the content from the 'Advice for package maintainers' section into a 'Defining a Go module' section; added some slightly stronger language at the end to suggest reading the official tip doc, etc.). 

Again, this is a WIP draft written by a non-expert.

######################################################

START DRAFT 2

######################################################

Wiki page title: Advice for early adopters of vgo / versioned Go modules

=== Introduction ===

Package versioning in Go is a rich topic, and this wiki page is intended just to supply:

 * a very brief introduction

 * a statement of which software is currently recommended as the typical choice for an early adopter (vgo vs. tip vs. betaN, etc.)

 * a few pointers to other resources for obtaining additional information

The recent work by the Go team on versioned Go modules started outside of the main repository with the vgo tool, but as of 2018-07-12 support for versioned Go modules has landed in the official main Go repository [1]. 

As of 2018-07-12, there are various minor known issues, but quite a lot works very well. 

[1] https://groups.google.com/forum/#!topic/golang-dev/a5PqQuBljF4

=== Should I use vgo, tip, or a beta 1.11 release? ===

Now that modules have landed in the main Go repository, vgo is no longer recommended. 

As of 2018-07-20, the recommendation instead is to please _get 1.11 beta2_ [2].  Alternatively, you can build from tip. [3]

[2] https://groups.google.com/forum/#!topic/golang-dev/A6TCp2kCoss 

[3] https://golang.org/doc/install/source

=== Brief advice for package maintainers ===

Go 1.11 includes experimental support for Go modules, including a new module-aware 'go get' command. The Go team intends to keep revising this support, while preserving compatibility, until it can be declared official (no longer experimental).

The most important pieces of advice for package maintainers:  

1. Please start tagging your packages with release tags that follow semantic versioning

2. Add go.mod files if that makes sense for your project. Today's go.mod files will be understood by any future tooling.

=== Defining a Go module ===

As of approximately 2018-07-20, according to the _tip documentation_[4] the quickest way to take advantage of the new Go 1.11 module support is to check out your repository into a directory outside GOPATH/src, create a go.mod file, and run go commands from within that file tree.

To start a new module, simply create a go.mod file in the root of the module's directory tree, containing only a module statement. The 'go mod' command can be used to do this:

go mod -init -module example.com/m

In a project already using an existing dependency management tool like godep, glide, or dep, 'go mod -init' will also add require statements matching the existing configuration.

Once the go.mod file exists, no additional steps are required: go commands like 'go build', 'go test', or even 'go list' will automatically add new dependencies as needed to satisfy imports.

However, this wiki page is only an extremely high-level introduction. For more information, please see the full godoc documentation at tip [4] and the output of 'go help modules'.

[4] https://tip.golang.org/cmd/go/#hdr-Modules__module_versions__and_more

== Some Additional Resources ==

An important short blog from Russ Cox that summarizes how versioned modules are currently an experimental opt-in feature for go 1.11 (2018-05-29):  https://research.swtch.com/vgo-accepted

The initial "Go & Versioning" series of blog posts by Russ Cox: https://research.swtch.com/vgo

Current tip documentation: https://tip.golang.org/cmd/go/#hdr-Modules__module_versions__and_more

Introductory blog post on 'Taking Go Modules for a Spin' by Dave Cheney (2018-07-14): https://dave.cheney.net/2018/07/14/taking-go-modules-for-a-spin

Introductory blog post on how to build go from tip and try go modules by Carolyn Van Slyck (2018-07-16): https://carolynvanslyck.com/blog/2018/07/building-go-from-source/

Official 'Versioned Go Modules' Proposal (last updated 2018-03-20): https://github.com/golang/proposal/blob/master/design/24301-versioned-go.md

_List of open Go modules bugs_

_Submitting a new Go modules bug_

######################################################

END DRAFT 2

######################################################

[Jeevanandam M.]

Now go1.11 beta 2 is out. I believe it could be good start. So I thought to post here.

$ go1.11beta2 help modules

Help content has sections -

* Defining a module

* The main module and the build list

* Maintaining module requirements

* Pseudo-versions

* Module queries

* Module code layout (this section is currently TODO, has mentioning to https://research.swtch.com/vgo-module, everyone knows this link)

* Module downloading and verification

* Modules and vendoring

- Jeeva

On Wednesday, June 20, 2018 at 6:08:41 AM UTC-7, robfig wrote:

[thepud...@gmail.com]

Hi all,

Apologies for circling back to this again, but FYI:

It looks like a few days ago Bryan Mills created the following wiki page on versioned Go modules:

  https://github.com/golang/go/wiki/Modules

The starting point was that he took the somewhat stale content from the older 'vgo user guide' wiki page. He then updated the introduction and instructions (including to be less about older vgo information and instead to be more about the 'mod' subcommands).

That seems like a useful page, including perhaps for some of the people reading this thread.

Today, I added a short 'current status' section near the top of that wiki page and also an 'additional resources' section at the bottom (including some pointers to official doc as well as some recent introductory blogs/videos). I won't be offended if anyone wants to revert or improve my changes.

(Finally, in case anyone is not already aware, there is also a wiki paged named 'vgo', but it seems to have more focus on the reactions to the vgo announcement and some specific issues, and is less focused on "I want to use versioned modules -- now what do I do?").

--thepudds