godoc does not follow links: feature or not?

[Petar Maymounkov]

I have a symlink directory in my gopath, like so

 $GOPATH/src/this_dir_is_a_symlink

and I noticed that godoc does not include the symlink directory in the documentation.

I am wondering if this is intended.

Thanks,

Petar

[Andrew Gerrand]

I think it was deliberately avoided, which is somewhat problematic on
debian systems where the GOROOT is split between /usr/share and
/usr/lib. (cc Ondřej Surý)

The argument against following symlinks is that it confuses import paths.

If you search golang-dev for "godoc symlink" you'll find more details.

Andrew

[Marvin Renich]

* Andrew Gerrand <a...@golang.org> [120516 21:47]:

I've offered to help Ondřej and am looking at this right now.

godoc in the current Debian package is pretty useless. The split is
done to follow the FHS. I'll look for the above reference and do some
reading.

...Marvin

[Andrew Gerrand]

I suggest consolidating the golang-src and golang-doc packages. Treat
the source code as documentation - which is what it is used for,
nobody is going to use the golang-src package to build the go tree.
This should still be within the rules of the FHS.

Then you can set godoc's GOROOT to the documentation root (and
preserve the existing go doc/ lib/godoc/ and pkg/ directories).

Andrew

[Marvin Renich]

* Andrew Gerrand <a...@golang.org> [120517 19:28]:

> On 18 May 2012 00:56, Marvin Renich <mr...@renich.org> wrote:
> > * Andrew Gerrand <a...@golang.org> [120516 21:47]:

> >> The argument against following symlinks is that it confuses import paths.
> >>
> >> If you search golang-dev for "godoc symlink" you'll find more details.

I didn't find anything that I thought was relevant here. Can you point
me to a specific thread?

I can imagine how symlinks in the middle of the import path might cause
problems, but my proposal below only allows the symlinks at the top
level, where they would not show up in any import statement.

> I suggest consolidating the golang-src and golang-doc packages. Treat
> the source code as documentation - which is what it is used for,
> nobody is going to use the golang-src package to build the go tree.
> This should still be within the rules of the FHS.
>
> Then you can set godoc's GOROOT to the documentation root (and
> preserve the existing go doc/ lib/godoc/ and pkg/ directories).

Combining these packages is completely orthogonal to the problem. It
would not solve the symlink issue, and putting the contents of these
packages directly under /usr/lib without symlinks can be done without
combining the packages (but not without violating the FHS).

The FHS issue is that $GOROOT/lib must be somewhere under /usr/lib (or
/lib, which isn't appropriate here), but the contents of golang-src and
golang-doc belong under /usr/share (or potentially under /usr/src for
golang-src, but I think it is better where Ondřej has put it).

You already allow an exception to the not-following-symlinks rule for
the explicit arguments provided on the godoc command line. Would you
entertain a patch that allows the immediate children of GOROOT to be
symbolic links?

What I have in mind is in godoc main() after

fs.Bind("/", OS(*goroot), "/", bindReplace)

loop through the contents of the goroot directory and bindReplace each
symlink with the directory to which it points.

There is the potential for a minor inconvenience with this approach in
what I believe is a very rare circumstance. If an administrator
(someone with permission to make changes under GOROOT) changes one of
these directories from a real directory to a symlink or vice versa, any
currently running instances of godoc will need to be restarted.

Alternatively, if GOROOT were /usr/share/go with a symlink for
$GOROOT/lib pointing to /usr/lib/go, would the go tool handle this
correctly for building and installing packages? Note that the files
under GOROOT only change on package upgrade (for Debian).

I am not sure if a symlink in /usr/share that points to somewhere in
/usr/lib that is not architecture independent is acceptable; I would
have to ask Ondřej or someone who understands this better than I.

...Marvin

(Does this thread belong on golang-dev? I have not subscribed to
that group yet, but I will.)

[Kyle Lemons]

On Fri, May 18, 2012 at 12:09 PM, Marvin Renich <mr...@renich.org> wrote:

* Andrew Gerrand <a...@golang.org> [120517 19:28]:

> On 18 May 2012 00:56, Marvin Renich <mr...@renich.org> wrote:
> > * Andrew Gerrand <a...@golang.org> [120516 21:47]:

> >> The argument against following symlinks is that it confuses import paths.
> >>
> >> If you search golang-dev for "godoc symlink" you'll find more details.

I didn't find anything that I thought was relevant here.  Can you point
me to a specific thread?

I can imagine how symlinks in the middle of the import path might cause
problems, but my proposal below only allows the symlinks at the top
level, where they would not show up in any import statement.

> I suggest consolidating the golang-src and golang-doc packages. Treat
> the source code as documentation - which is what it is used for,
> nobody is going to use the golang-src package to build the go tree.
> This should still be within the rules of the FHS.
>
> Then you can set godoc's GOROOT to the documentation root (and
> preserve the existing go doc/ lib/godoc/ and pkg/ directories).

Combining these packages is completely orthogonal to the problem.  It
would not solve the symlink issue, and putting the contents of these
packages directly under /usr/lib without symlinks can be done without
combining the packages (but not without violating the FHS).

I think his point was that in Go, the code is documentation, and thus it wouldn't violate the FHS.

[Marvin Renich]

* Kyle Lemons <kev...@google.com> [120518 17:30]:

> On Fri, May 18, 2012 at 12:09 PM, Marvin Renich <mr...@renich.org> wrote:
> > Combining these packages is completely orthogonal to the problem. It
> > would not solve the symlink issue, and putting the contents of these
> > packages directly under /usr/lib without symlinks can be done without
> > combining the packages (but not without violating the FHS).
>
> I think his point was that in Go, the code is documentation, and thus it
> wouldn't violate the FHS.

I'm sorry, my message incorrectly stated $GOROOT/lib must be under
/usr/lib, but it is $GOROOT/pkg that contains the *.a files that must be
under /usr/lib. I was confusing the correlation between directory names
under $GOROOT and their intended use, as lib in the FHS is for "object
files, libraries, and internal binaries that are not intended to be
executed directly by users or shell scripts." Although I knew that the
.a files were under pkg, I was still thinking lib as in the FHS
definition.

The point is that the .a files for the pre-compiled standard libraries
(used by go build and go install) must be under /usr/lib, while the
.html files used by godoc must be under /usr/share in order to satisfy
the FHS and Debian policy. Combining the golang-src and golang-doc
packages is not relevant to this.

Ondřej, if my understanding of Debian policy or the FHS is wrong, please
let me know.

...Marvin

[John Asmuth]

On Friday, May 18, 2012 7:36:06 PM UTC-4, Marvin Renich wrote: while the 

.html files used by godoc must be under /usr/share 

What .html files? 

[Marvin Renich]

* John Asmuth <jas...@gmail.com> [120518 21:44]:

> On Friday, May 18, 2012 7:36:06 PM UTC-4, Marvin Renich wrote: while the
> > .html files used by godoc must be under /usr/share
>
> What .html files?

The ones found in $GOROOT/doc.

...Marvin

[Andrew Gerrand]

[taking this conversation to golang-dev]

On 19 May 2012 05:09, Marvin Renich <mr...@renich.org> wrote:
> I can imagine how symlinks in the middle of the import path might cause
> problems, but my proposal below only allows the symlinks at the top
> level, where they would not show up in any import statement.
>

[snip]

>
> You already allow an exception to the not-following-symlinks rule for
> the explicit arguments provided on the godoc command line.  Would you
> entertain a patch that allows the immediate children of GOROOT to be
> symbolic links?

That sounds reasonable to me.

Andrew

[joe....@gmail.com]

I know I'm resurrecting a really old thread, but I wanted to bring this up for discussion again, since godoc still does not follow symbolic links.

I too would like the ability for godoc to follow (top level) symbolic links.  The reason for this is that our organization all of our code bases exist in a few different repos and are organized in such a way that makes it impractical to make our go code "go get-able".  What we tend to do is check out the entire repo in a central location (ex. ~/home/src/repo004/trunk) and use symbolic links to map our go libraries into our GOPATH (ln -s $REPO/some/deep/path/to/golibs $GOPATH/src/).

Since godoc doesn't follow links, the documentation for our in-house stuff does not show up.

- Joe

[sna...@gmail.com]

Sounds to me like your setup is going directly against expected idioms, and are using symlinks to try and dance around that. I find it unlikely for something to be implemented to support an unsupported setup. You specifically even say your setup makes it impractical to make your code "go get-able".

[Joe Sylve]

There is little reason to make non-publicly available libraries "go get-able".  I don't feel like our setup is very uncommon for non open source projects.  The other requests earlier in this thread show that it's more than just us that are requesting this feature.

[Nate Finch]

You don't have to make your code go-gettable to make local godoc work.  You just need to have package "foo/bar/baz" at $GOPATH/src/foo/bar/baz  ... this should be completely doable without needing symlinks.

It seems like the answer is just to put one or more src/ directories in your repo with the Go code under it, and add that path to your gopath

so, for your example:

$REPO/some/deep/path/to/golibs/src/foo/bar/baz

if you just make $GOPATH=$REPO/some/deep/path/to/golibs/

then import "foo/bar/baz" will work.

If you have more than one directory of repos, remember that GOPATH can be multiple paths:

$GOPATH=$REPO/some/deep/path/to/golibs/:$REPO/some/other/path/

It's all a matter of setting up GOPATH correctly, and then making sure your go code imports correctly.