$ mkdir /tmp/hello$ cd /tmp/hello$ echo module hello >go.mod$ cat >hello.go <<EOFpackage mainimport "rsc.io/quote"func main() {println(quote.Glass())}EOF$ go run .I can eat glass and it doesn't hurt me.$ cat go.modmodule hellorequire rsc.io/quote v1.5.2$ go list -m allhellogolang.org/x/text v0.0.0-20170915032832-14c0d48ead0crsc.io/quote v1.5.2rsc.io/sampler v1.3.0$
--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Hi Russ, a quick note. My $GOPATH is $HOME, and I develop all my code in $HOME/src.If I don't set GO111MODULE, I will have to put code somewhere else, which I'd rather not do. But if I set GO111MODULE=on globally, all non-module aware code will break. So my only option is to set GO111MODULE=on on a case-by-case basis, which is not great.Why not make auto mode work within GOPATH as well when a go.mod file is present? I feel like it would also avoid another layer of GOPATH confusion, given "we don't want the repository location to matter" is a common complaint that module support is meant to address.
Since go.mod appears to become now a part of the official go tool, I gave it a run. It appeared to work well, but there are three observations that I would like to share:
- I can't do rm -fr $GOPATH/src/mod because the directory with the @<hexstring> (in my case golang.org/x/crypto@<hexstring>) is not writable for the user.
- If GOPATH is not set $HOME/go/src/mod is created, which is not nice, when my go command comes actually from $HOME/go1.11 and there is another version in $HOME/go. I guess in that scenario I will have to keep GOPATH.
- goimports will have to support go.mod. Since I'm using an older version, some work might have already gone into it.
Hello,
this looks really cool. It seems that it will make really easy to follow good engineering practices regarding creating releases. Also it seems that this will make vendoring effectively obsoleted/not needed any more, which I perceive as really good.
Also do you plan to tackle the issues with API stability/compatibility i.e. will vgo be able to detect and report API breakages?
On side note, is there a place that one can follow and see the development progress and development discussions of the vgo/dep and contribute to them? IIRC I have read somewhere sometime ago that you are using some private slack for the discussions, right?
Awesome to hear that :). So you are now under go not vgo under golang@github? And the CL workflow is the same as for Go, right?
--
I couldn't follow your response to Fillipo... what exactly are the problems for people who have GOPATH=$HOME, and what steps do we need to take to keep working? I think a lot of gophers still set GOPATH this way (myself included).
- I can't do rm -fr $GOPATH/src/mod because the directory with the @<hexstring> (in my case golang.org/x/crypto@<hexstring>) is not writable for the user.
Yes, this is really annoying: when I made the directories unwritable I thought that rm -rf would still work on them, and I'm disappointed that it doesn't. Even so, I do think it's important to keep them unwritable to prevent accidental modification. I intend to add 'go clean -modcache' or something like that to blow away that tree for you (chmod + rm), so even though rm -rf fails, there will be an easy command.
The problem is that we just finished telling people that "it just builds because it uses vendoring, but you have to clone it within GOPATH/src", which broke an assumption about the location of a repository being irrelevant, and now we have to tell people that "it just builds because it uses modules, but you have to clone it outside GOPATH/src", adding confusion instead of solving that very common adoption issue.
Also, this will require most CI systems to be reconfigured, as they clone in GOPATH and run "go build" by default.
I don't think I understand the nuances enough to judge the tradeoff or the alternatives, but I am very familiar with the off-putting impact GOPATH has on new developers, and this only barely helps.
On Thu, Jul 12, 2018 at 5:57 PM Filippo Valsorda <fil...@golang.org> wrote:
If I don't set GO111MODULE, I will have to put code somewhere else, which I'd rather not do. But if I set GO111MODULE=on globally, all non-module aware code will break. So my only option is to set GO111MODULE=on on a case-by-case basis, which is not great.
It's a great question, and I've added golang-dev back because I think lots of people will have this question, so it's worth sharing the answer. I like to use GOPATH=HOME too, and keeping that case in mind is why the GOPATH carveout when GO111MODULE=on is only GOPATH/src not all of GOPATH. ...
Thanks,
Paul
Today, while debugging, I make some changes to a couple of dependent packages: cloud.google.com/go/pubsub and google.golang.org/grpc. Added some logging, changed a bit of error handling logic. The usual.How do I do the same thing with modules?The code in $GOPATH/src/mod is read-only, so apparently not there.
--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
On Mon, 2018-07-16 at 11:36 -0400, Russ Cox wrote:
> My priority here is to ensure a smooth transition for *late*
> adopters, ...
The documentation for go mod falls down in this area. I was looking
through `go help mod` today trying to figure out how to resolve an
issue filed by one of our users and came to the conclusion that the
easiest way to deal with the problem was just manually edit the go.mod
file and then hope. This is not what I usually expect from Go docs.
An additional data point on this highlighting the non-orthonality; I
now have GO111MODULE=on as part of my env to avoid the problems that I
saw before, but this breaks just being able to do go get -u somepkg.
-j
The property, that import paths can be mechanically translated to file paths under $GOPATH/src and vice versa is IMHO a valuable one.
--Is it really unavoidable to lose this property (while providing and/or consuming Go modules in 1.11)? Or am I, which is far more probable, completely missing something again?Thanks in advance for any enlightenment. It will be very appreciated.-j
--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
On Jul 24, 2018, at 4:49 AM, 'Axel Wagner' via golang-dev <golan...@googlegroups.com> wrote:Hi Jan,On Mon, Jul 23, 2018 at 5:17 PM Jan Mercl <0xj...@gmail.com> wrote:The property, that import paths can be mechanically translated to file paths under $GOPATH/src and vice versa is IMHO a valuable one.I don't think this property ever held in any sense of generality. $GOPATH could have mulitple elements, $GOPATH can change between build-time and runtime (and usually™ does), vendoring can break the mapping and the source might not even be available at runtime. So you already had to make a bunch of assumptions to make this work, now you just have to slightly modify them.
> email to golang-dev+unsubscribe@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
Dave, thanks for your support. That to check in in the results of go generate is the way to work is not just your opinion but part of the design and documentation. The client of your code might not have the tools to run the command that generates it, so the design and intent is to assume that they never do, to avoid problems.-rob
On Fri, Jul 27, 2018 at 12:39 PM, Dave Cheney <da...@cheney.net> wrote:
> Will `go modules` support `go:generate` in the future?
No, for the same reason that go get does not execute go generate as
part of checking out a repo, I imagine that go modules will not
support this either.
IMO go:generate code should be checked into the repo by the author. It
should not be the responsibilty of those who depend on that code as
clients to be able to execute their go:generate steps.
On 27 July 2018 at 11:58, Ewan Chou <coo...@gmail.com> wrote:
> I tried to build a project but failed because it depends on a repo that has
> some of the generated code ignored in git.
>
> In GOPATH mode, I can generate the code first, then build, but in modules
> mode, I can't do that.
>
> Will `go modules` support `go:generate` in the future?
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-dev+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
[...] Note that it would be entirely reasonable to start using modules and Go 1.11 but also run 'go mod -vendor' to populate the vendor directory for users on older Go releases. In that case you'd want the CI system to keep testing the vendor directory. [...]
$ (cd module_path; go install -v -mod vendor ./...)
net
# net
exec: "gcc": executable file not found in $PATH
As of CL 123580, submitted a few minutes ago, the development copy of the go command, in the main go repository, has support for Go modules. There are various minor known issues, but quite a lot works very well.
There's still more to be done but this is a key milestone: if you run on the bleeding edge, you can play with modules just by using "go". Work on the module support will now be done exclusively in the main repo. We will automatically export the main repo to the x/vgo repo when we have a snapshot we want to make available to people using "vgo".The open issues have moved from the vgo milestone to the Go 1.11 milestone, but labeled with the new 'modules' label.A note on terminology: Go 1.11 is not shipping with "vgo support". It is shipping with "Go module support". Vgo was only ever the prototype/demo of that module support. Just like all the other features we've introduced to the go command over the past ten years, the goal for this one is to fit so well into the rest of the go command fabric that the seams are invisible - module support is just part of "go", the same way that, say, race detector support or GOPATH support is.Speaking of GOPATH, the go command automatically chooses between using modules and using GOPATH depending on where it is run. If a go command runs outside GOPATH/src and there is a go.mod file in the current directory or one of its parent directories, then modules are enabled (otherwise not):$ mkdir /tmp/hello$ cd /tmp/hello$ echo module hello >go.mod$ cat >hello.go <<EOFpackage mainimport "rsc.io/quote"func main() {println(quote.Glass())}EOF$ go run .I can eat glass and it doesn't hurt me.$ cat go.modmodule hellorequire rsc.io/quote v1.5.2$ go list -m allhellogolang.org/x/text v0.0.0-20170915032832-14c0d48ead0crsc.io/quote v1.5.2rsc.io/sampler v1.3.0$For more information, see 'go help modules', which I've included at the end of this mail.
Many thanks to everyone in the community who has helped us get to this point, but especially to Paul Jolly for really hammering on the module support and filing tons of great bugs, and to Baokun Lee for a steady sequence of helpful CLs. Thanks also to Bryan Mills for helping work out a lot of important details in the past few weeks.Thanks again, and keep the bug reports coming.Best,Russgo help modulesA module is a collection of related Go packages.Modules are the unit of source code interchange and versioning.The go command has direct support for working with modules,including recording and resolving dependencies on other modules.Modules replace the old GOPATH-based approach to specifyingwhich source files are used in a given build.Experimental module supportGo 1.11 includes experimental support for Go modules,including a new module-aware 'go get' command.We intend to keep revising this support, while preserving compatibility,until it can be declared official (no longer experimental),and then at a later point we may remove support for workin GOPATH and the old 'go get' command.The quickest way to take advantage of the new Go 1.11 module supportis to check out your repository into a directory outside GOPATH/src,create a go.mod file (described in the next section) there, and rungo commands from within that file tree.For more fine-grained control, the module support in Go 1.11 respectsa temporary environment variable, GO111MODULE, which can be set to oneof three string values: off, on, or auto (the default).If GO111MODULE=off, then the go command never uses thenew module support. Instead it looks in vendor directories and GOPATHto find dependencies; we now refer to this as "GOPATH mode."If GO111MODULE=on, then the go command requires the use of modules,never consulting GOPATH. We refer to this as the command beingmodule-aware or running in "module-aware mode".If GO111MODULE=auto or is unset, then the go command enables ordisables module support based on the current directory.Module support is enabled only when the current directory is outsideGOPATH/src and itself contains a go.mod file or is below a directorycontaining a go.mod file.Defining a moduleA module is defined by a tree of Go source files with a go.mod filein the tree's root directory. The directory containing the go.mod fileis called the module root. Typically the module root will also correspondto a source code repository root (but in general it need not).The module is the set of all Go packages in the module root and itssubdirectories, but excluding subtrees with their own go.mod files.The "module path" is the import path prefix corresponding to the module root.The go.mod file defines the module path and lists the specific versionsof other modules that should be used when resolving imports during a build,by giving their module paths and versions.For example, this go.mod declares that the directory containing it is the rootof the module with path example.com/m, and it also declares that the moduledepends on specific versions of golang.org/x/text and gopkg.in/yaml.v2:module example.com/mrequire (golang.org/x/text v0.3.0gopkg.in/yaml.v2 v2.1.0)The go.mod file can also specify replacements and excluded versionsthat only apply when building the module directly; they are ignoredwhen the module is incorporated into a larger build.For more about the go.mod file, see https://research.swtch.com/vgo-module.To start a new module, simply create a go.mod file in the root of themodule's directory tree, containing only a module statement.The 'go mod' command can be used to do this:go mod -init -module example.com/mIn a project already using an existing dependency management tool likegodep, glide, or dep, 'go mod -init' will also add require statementsmatching the existing configuration.Once the go.mod file exists, no additional steps are required:go commands like 'go build', 'go test', or even 'go list' will automaticallyadd new dependencies as needed to satisfy imports.The main module and the build listThe "main module" is the module containing the directory where the go commandis run. The go command finds the module root by looking for a go.mod in thecurrent directory, or else the current directory's parent directory,or else the parent's parent directory, and so on.The main module's go.mod file defines the precise set of packages availablefor use by the go command, through require, replace, and exclude statements.Dependency modules, found by following require statements, also contributeto the definition of that set of packages, but only through their go.modfiles' require statements: any replace and exclude statements in dependencymodules are ignored. The replace and exclude statements therefore allow themain module complete control over its own build, without also being subjectto complete control by dependencies.The set of modules providing packages to builds is called the "build list".The build list initially contains only the main module. Then the go commandadds to the list the exact module versions required by modules alreadyon the list, recursively, until there is nothing left to add to the list.If multiple versions of a particular module are added to the list,then at the end only the latest version (according to semantic versionordering) is kept for use in the build.The 'go list' command provides information about the main moduleand the build list. For example:go list -m # print path of main modulego list -m -f={{.Dir}} # print root directory of main modulego list -m all # print build listMaintaining module requirementsThe go.mod file is meant to be readable and editable by bothprogrammers and tools. The go command itself automatically updates the go.mod fileto maintain a standard formatting and the accuracy of require statements.Any go command that finds an unfamiliar import will look up the modulecontaining that import and add the latest version of that moduleto go.mod automatically. In most cases, therefore, it suffices toadd an import to source code and run 'go build', 'go test', or even 'go list':as part of analyzing the package, the go command will discoverand resolve the import and update the go.mod file.Any go command can determine that a module requirement ismissing and must be added, even when considering only a singlepackage from the module. On the other hand, determining that a module requirementis no longer necessary and can be deleted requires a full view ofall packages in the module, across all possible build configurations(architectures, operating systems, build tags, and so on).The 'go mod -sync' command builds that view and thenadds any missing module requirements and removes unnecessary ones.As part of maintaining the require statements in go.mod, the go commandtracks which ones provide packages imported directly by the current moduleand which ones provide packages only used indirectly by other moduledependencies. Requirements needed only for indirect uses are marked with a"// indirect" comment in the go.mod file. Indirect requirements areautomatically removed from the go.mod file once they are implied by otherdirect requirements. Indirect requirements only arise when using modulesthat fail to state some of their own dependencies or when explicitlyupgrading a module's dependencies ahead of its own stated requirements.Because of this automatic maintenance, the information in go.mod is anup-to-date, readable description of the build.The 'go get' command updates go.mod to change the module versions used in abuild. An upgrade of one module may imply upgrading others, and similarly adowngrade of one module may imply downgrading others. The 'go get' commandmakes these implied changes as well. If go.mod is edited directly, commandslike 'go build' or 'go list' will assume that an upgrade is intended andautomatically make any implied upgrades and update go.mod to reflect them.The 'go mod' command provides other functionality for use in maintainingand understanding modules and go.mod files. See 'go help mod'.Pseudo-versionsThe go.mod file and the go command more generally use semantic versions asthe standard form for describing module versions, so that versions can becompared to determine which should be considered earlier or later than another.A module version like v1.2.3 is introduced by tagging a revision in theunderlying source repository. Untagged revisions can be referred tousing a "pseudo-version" of the form v0.0.0-yyyymmddhhmmss-abcdefabcdef,where the time is the commit time in UTC and the final suffix is the prefixof the commit hash. The time portion ensures that two pseudo-versions canbe compared to determine which happened later, the commit hash identifesthe underlying commit, and the v0.0.0- prefix identifies the pseudo-versionas a pre-release before version v0.0.0, so that the go command prefers anytagged release over any pseudo-version.Pseudo-versions never need to be typed by hand: the go command will acceptthe plain commit hash and translate it into a pseudo-version (or a taggedversion if available) automatically. This conversion is an example of amodule query.Module queriesThe go command accepts a "module query" in place of a module versionboth on the command line and in the main module's go.mod file.(After evaluating a query found in the main module's go.mod file,the go command updates the file to replace the query with its result.)A fully-specified semantic version, such as "v1.2.3",evaluates to that specific version.A semantic version prefix, such as "v1" or "v1.2",evaluates to the latest available tagged version with that prefix.A semantic version comparison, such as "<v1.2.3" or ">=v1.5.6",evaluates to the available tagged version nearest to the comparison target(the latest version for < and <=, the earliest version for > and >=).The string "latest" matches the latest available tagged version,or else the underlying source repository's latest untagged revision.A revision identifier for the underlying source repository,such as a commit hash prefix, revision tag, or branch name,selects that specific code revision. If the revision isalso tagged with a semantic version, the query evaluates tothat semantic version. Otherwise the query evaluates to apseudo-version for the commit.All queries prefer release versions to pre-release versions.For example, "<v1.2.3" will prefer to return "v1.2.2"instead of "v1.2.3-pre1", even though "v1.2.3-pre1" is nearerto the comparison target.Module versions disallowed by exclude statements in themain module's go.mod are considered unavailable and cannotbe returned by queries.For example, these commands are all valid:go get github.com/gorilla/mux@latest # same (@latest is default for 'go get')go get github.com/gorilla/m...@v1.6.2 # records v1.6.2go get github.com/gorilla/mux@e3702bed2 # records v1.6.2go get github.com/gorilla/mux@c856192 # records v0.0.0-20180517173623-c85619274f5dgo get github.com/gorilla/mux@master # records current meaning of masterModule compatibility and semantic versioningThe go command requires that modules use semantic versions and expects thatthe versions accurately describe compatibility: it assumes that v1.5.4 is abackwards-compatible replacement for v1.5.3, v1.4.0, and even v1.0.0.More generally the go command expects that packages follow the"import compatibility rule", which says:"If an old package and a new package have the same import path,the new package must be backwards compatible with the old package."Because the go command assumes the import compatibility rule,a module definition can only set the minimum required version of oneof its dependencies: it cannot set a maximum or exclude selected versions.Still, the import compatibility rule is not a guarantee: it may be thatv1.5.4 is buggy and not a backwards-compatible replacement for v1.5.3.Because of this, the go command never updates from an older versionto a newer version of a module unasked.In semantic versioning, changing the major version number indicates a lackof backwards compatibility with earlier versions. To preserve importcompatibility, the go command requires that modules with major version v2or later use a module path with that major version as the final element.For example, version v2.0.0 of example.com/m must instead use module pathexample.com/m/v2, and packages in that module would use that path astheir import path prefix, as in example.com/m/v2/sub/pkg. Including themajor version number in the module path and import paths in this way iscalled "semantic import versioning". Pseudo-versions for modules with majorversion v2 and later begin with that major version instead of v0, as inv2.0.0-20180326061214-4fc5987536ef.The go command treats modules with different module paths as unrelated:it makes no connection between example.com/m and example.com/m/v2.Modules with different major versions can be used together in a buildand are kept separate by the fact that their packages use differentimport paths.In semantic versioning, major version v0 is for initial development,indicating no expectations of stability or backwards compatibility.Major version v0 does not appear in the module path, because thoseversions are preparation for v1.0.0, and v1 does not appear in themodule path either.As a special case, for historical reasons, module paths beginning withgopkg.in/ continue to use the conventions established on that system:the major version is always present, and it is preceded by a dotinstead of a slash: gopkg.in/yaml.v1 and gopkg.in/yaml.v2, notfor more information.Module verificationThe go command maintains, in the main module's root directory alongsidego.mod, a file named go.sum containing the expected cryptographic checksumsof the content of specific module versions. Each time a dependency isused, its checksum is added to go.sum if missing or else required to matchthe existing entry in go.sum.The go command maintains a cache of downloaded packages and computesand records the cryptographic checksum of each package at download time.In normal operation, the go command checks these pre-computed checksumsagainst the main module's go.sum file, instead of recomputing them oneach command invocation. The 'go mod -verify' command checks thatthe cached copies of module downloads still match both their recordedchecksums and the entries in go.sum.Modules and vendoringWhen using modules, the go command completely ignores vendor directories.By default, the go command satisfies dependencies by downloading modulesfrom their sources and using those downloaded copies (after verification,as described in the previous section). To allow interoperation with olderversions of Go, or to ensure that all files used for a build are storedtogether in a single file tree, 'go mod -vendor' creates a directory namedvendor in the root directory of the main module and stores there all thepackages from dependency modules that are needed to support builds andtests of packages in the main module.To build using the main module's top-level vendor directory to satisfydependencies (disabling use of the usual network sources and localcaches), use 'go build -getmode=vendor'. Note that only the main module'stop-level vendor directory is used; vendor directories in other locationsare still ignored.