package is not in goroot

[Alexander Mills]

i am getting this weird error message:

 package twitch/go-scripts/dynamo-creators-to-s3/lib is not in GOROOT (/usr/local/Cellar/go/1.15.6/libexec/src/twitch/go-scripts/dynamo-creators-to-s3/lib)

my GOPATH=$PWD/scripts/go

so there is only 1 GOPATH

[Dan Kortschak]

On Tue, 2021-01-19 at 13:54 -0800, Alexander Mills wrote:
> i am getting this weird error message:
>
> package twitch/go-scripts/dynamo-creators-to-s3/lib is not in GOROOT
> (/usr/local/Cellar/go/1.15.6/libexec/src/twitch/go-scripts/dynamo-
> creators-to-s3/lib)
>
> my GOPATH=$PWD/scripts/go
>
> so there is only 1 GOPATH

Packages that are not in GOROOT must have a dot in the first path
element. Your import path doesn't; it's "twitch".

[Alexander Mills]

I have never seen the dot needed, I have used plenty of packages that
are in GOPATH but not GOROOT

> --
> You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/yetRzPu19eQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/3e1a5f86d1cb6724deeb48cdc4017b3a94f459ef.camel%40kortschak.io.



--
Alexander D. Mills
New cell phone # (415)730-1805
linkedin.com/in/alexanderdmills

[Dan Kortschak]

On Tue, 2021-01-19 at 16:08 -0800, Alexander Mills wrote:
> I have never seen the dot needed, I have used plenty of packages that
> are in GOPATH but not GOROOT

https://go.googlesource.com/go/+/go1.15.6/src/cmd/go/internal/search/search.go#542

This is why it is saying that it's expecting it to be in GOROOT. Why it
can't find it? I don't know, but you haven't said what the error was in
response to.

[Marvin Renich]

* Alexander Mills <alexande...@gmail.com> [210119 16:54]:

The error message is really incomplete, and therefore confusing. It
should be "package ... is not in GOROOT or any element of GOPATH".

I had written an answer based on an environment using GOPATH before
carefully looking at your screenshot (please paste text rather than
using screenshots). It seems you have a go.mod file, so GOPATH is not
relevant.

If using modules rather than GOPATH, the error message is probably
correct, as GOPATH is not used except as a place to cache downloaded
modules under the "$GOPATH/pkg" directory.

As best as I can tell (and I am still using GOPATH, so I may be
completely wrong), a Go program in a module can only use web-aware
import paths (except for standard library packages). Perhaps someone
else can clarify that, and how to set up your directory structure. I
think there was recent thread with a similar problem, but I did not pay
much attention to it.

Personally, I think that requiring (as opposed to allowing) web-aware
import paths is a terrible (no, I mean really, really terrible) design.

On the other hand, if you were intending to use GOPATH, but it was
incorrectly set and so the go tool kindly (???) created a go.mod file
for you, I can probably help you figure out your problem. First, erase
go.mod. Then, ensure GOPATH is what you expect (use "go env GOPATH",
rather than "echo $GOPATH"). If that doesn't lead you to an answer,
post the output from go env and the go command you are using that gives
the error (go build, go test, etc.).

...Marvin

[Jan Mercl]

On Wed, Jan 20, 2021 at 2:58 PM Marvin Renich <mr...@renich.org> wrote:

> The error message is really incomplete, and therefore confusing. It
> should be "package ... is not in GOROOT or any element of GOPATH".

_That_ would be incorrect and confusing. Reserved import paths are
never searched for in GOPATH.

[Marvin Renich]

* Jan Mercl <0xj...@gmail.com> [210120 09:32]:

I don't understand what you are saying. What is a "reserved import
path" and where is it defined?

When using GOPATH (not modules), when the go toolchain encounters

import "path/to/package"

it first looks for a directory obtained from (roughly)
filepath.Join(GOROOT, "src", "path/to/package"), if it does not find the
package there, it does the same thing, replacing GOROOT with,
successively, each element of GOPATH. So, if it cannot find the given
import path, then the package is not in GOROOT, nor is it in any element
of GOPATH. What is confusing about that?

...Marvin

[Marvin Renich]

* Marvin Renich <mr...@renich.org> [210120 09:58]:

> * Jan Mercl <0xj...@gmail.com> [210120 09:32]:
> > On Wed, Jan 20, 2021 at 2:58 PM Marvin Renich <mr...@renich.org> wrote:
> >
> > > The error message is really incomplete, and therefore confusing. It
> > > should be "package ... is not in GOROOT or any element of GOPATH".
> >
> > _That_ would be incorrect and confusing. Reserved import paths are
> > never searched for in GOPATH.

Perhaps you missed my third paragraph. The first paragraph was written
when I was still thinking the OP was using GOPATH, not modules.

I still don't know what you mean by "reserved import path".

...Marvin

[Jan Mercl]

On Wed, Jan 20, 2021 at 3:57 PM Marvin Renich <mr...@renich.org> wrote:

> I don't understand what you are saying. What is a "reserved import
> path" and where is it defined?

See https://github.com/golang/go/issues/32819

[Marvin Renich]

* Jan Mercl <0xj...@gmail.com> [210120 10:46]:

???

Thanks for pointing this out!

My first paragraph was written when I thought the OP was using GOPATH
mode, not modules (and that the error message he received was from that
mode), and so my suggestion would have been correct and more clear. In
fact, if using go build in GOPATH mode, it does, indeed give that
information and even more, enumerating the elements of GOPATH that it
searched.

I still cannot find any mention of reserved import paths in any
documentation (go help modules/importpath/gopath). It's just an issue
on the tracker.

Import paths without a dot in the first component (which is how I read
what this issue is trying to claim as "reserved") have been part and
parcel of the GOPATH way of building user packages since the beginning
of the go tool, and this still works perfectly. Not everyone wants or
needs to put their go source code in a web-accessible location, or even
in a version control system, for that matter. Changing this for GOPATH
mode would, in my mind, be a major breaking change.

I have never seen any mention of this issue on golang-nuts, though I
certainly could have missed it. Perhaps some discussion took place on
golang-dev. For a change this important, I would expect it to be
publicized much better. If this is only going to affect module mode,
that is a different matter.

...Marvin

[Jan Mercl]

On Wed, Jan 20, 2021 at 9:03 PM Marvin Renich <mr...@renich.org> wrote:

> I still cannot find any mention of reserved import paths in any
> documentation (go help modules/importpath/gopath). It's just an issue
> on the tracker.

When an important part of the implementation is undocumented then it's
a documentation bug, hence the issue.

> Import paths without a dot in the first component (which is how I read
> what this issue is trying to claim as "reserved") have been part and
> parcel of the GOPATH way of building user packages since the beginning
> of the go tool, and this still works perfectly. Not everyone wants or
> needs to put their go source code in a web-accessible location, or even
> in a version control system, for that matter.

Import path is not an URL. The Go build system in GOPATH does not
interpret import paths as URLs. It's only the go get command that uses
a well defined function for going from import path to URL to
download/update repositories.

From the POV of the build system, Import paths refer to file system
locations, again using a well defined function for that.

> Changing this for GOPATH
> mode would, in my mind, be a major breaking change.

IMO it's a security issue that needs to be fixed. If anything puts a
replacement of some stdlib package in your $GOPATH then you may have a
problem.

Even on an uncompromised system, using the reserved path may in future
clash with a new package of the same name added to stdlib, producing a
log of confusion about why this bunch of packages build just fine and
the rest does not.

> I have never seen any mention of this issue on golang-nuts, though I
> certainly could have missed it. Perhaps some discussion took place on
> golang-dev. For a change this important, I would expect it to be
> publicized much better. If this is only going to affect module mode,
> that is a different matter.

I don't recall where I became aware of the linked issue. It's too long ago.

[Marvin Renich]

* Jan Mercl <0xj...@gmail.com> [210120 15:27]:

> On Wed, Jan 20, 2021 at 9:03 PM Marvin Renich <mr...@renich.org> wrote:
>
> > I still cannot find any mention of reserved import paths in any
> > documentation (go help modules/importpath/gopath). It's just an issue
> > on the tracker.
>
> When an important part of the implementation is undocumented then it's
> a documentation bug, hence the issue.

Absolutely. I note the bug has been open for more than 18 months.

> Import path is not an URL. The Go build system in GOPATH does not
> interpret import paths as URLs. It's only the go get command that uses
> a well defined function for going from import path to URL to
> download/update repositories.
>
> From the POV of the build system, Import paths refer to file system
> locations, again using a well defined function for that.

Agreed. However, what I seem to be hearing about module mode is
different. Perhaps I am misunderstanding.

> > Changing this for GOPATH
> > mode would, in my mind, be a major breaking change.
>
> IMO it's a security issue that needs to be fixed. If anything puts a
> replacement of some stdlib package in your $GOPATH then you may have a
> problem.
>
> Even on an uncompromised system, using the reserved path may in future
> clash with a new package of the same name added to stdlib, producing a
> log of confusion about why this bunch of packages build just fine and
> the rest does not.

GOPATH build mode searches GOROOT first, so no security issue. Name
clash with future stdlib package is an issue, but much easier to deal
with. Public discussion about reserving an import namespace is still
important. If we had had this discussion prior to Go 1, I would have
suggested using one, well-known name for the first component of standard
library packages; perhaps "_stdlib". Too late for that now.

> > I have never seen any mention of this issue on golang-nuts, though I
> > certainly could have missed it. Perhaps some discussion took place on
> > golang-dev. For a change this important, I would expect it to be
> > publicized much better. If this is only going to affect module mode,
> > that is a different matter.
>
> I don't recall where I became aware of the linked issue. It's too long ago.

I did not intend for this to sound like a complaint directed at you. I
appreciate your pointing it out to me. If the developers working on
this intend to make GOPATH mode fail for a large number of existing
local packages, I would appreciate it if they would say something on
golang-nuts. This is somewhat different from deprecating the use of
first-level, non-dot names, but still allowing them if they don't clash
with stdlib packages.

...Marvin