go doc vs. godoc

1,505 views
Skip to first unread message

Rob Pike

unread,
Apr 17, 2015, 4:40:49 PM4/17/15
to golan...@googlegroups.com

Now that the type checker and related components are moving into the main repository from x/tools, it's time to consider the correct way to build, install, and deploy vet and godoc, both of which have resided in x/tools since the type checker arrived. (They had to live there rather than in the main repository because the main repository can have no outside dependencies.) Having these tools outside the main repository is annoying and complicates the process of producing binary releases for download.


For Go 1.5, it is clear that vet can move into the main repository. The old location will still exist, for compatibility reasons, until at least the release of 1.6, but it will not be actively maintained. Vet is coming home.


The story with godoc is messier. It has a much larger list of dependencies, many of which will stay in the x/tools repository for the foreseeable future. Thus godoc as it is today must stay where it is. But there is a way to make things better.


Proposal


I propose a way to clean up the release process and improve the situation for all users.


The essence is to resurrect the go tool's doc command (go doc as opposed to godoc) but as an all-new tool that provides command-line level functionality only. This means it need have none of the complex dependencies that burden the full godoc. (It might even be simple enough to be built into the go tool directly rather than as a separate binary, but that is an implementation detail.) The releases would ship with that tool (only), and would therefore be buildable directly from the main repository. People who want to run a local web server or want access to the facilities of the oracle and so on would be required to run go get golang.org/x/tools/cmd/godoc, as they do today when they want to update, but that is a small matter.


There is actually a more interesting reason for a new go doc: it provides the opportunity to design a new, easy to use, more expressive UI for a completely command-line-oriented tool.


The new go doc would be nothing more than a convenient, fast way to see the documentation, as plain text, for a package or one of its components. Several people have already built such tools precisely because of the inconvenient UI of godoc.


Say we wish to check the documentation for net/http's Server.ListenAndServeTLS method to verify the order of its two string arguments. It is not even possible to get it from godoc without getting the rest of the Server type's methods, but even that request has an odd interface:


    godoc net/http Server # maybe piped through grep ListenAndServeTLS


(Godoc's syntax is a package path followed by a space and a package-level name). A beginner might expect a more Go-like syntax, and that is in fact what at least one of the new-style command-line-oriented documentation tools does. My own tool godoc.org/robpike.io/cmd/doc has some deep flaws but it does allow the user to ask the specific question more directly and somewhat naturally:


   doc http.ListenAndServeTLS


(The syntax here is packagename.exportedname, where exportedname could be an exported method of an exported type.) Actually, it's case insensitive; this works too:


  doc http.listenandservetls


and one can even use a regular expression. If we can't remember how to spell it we can ask for all the listen methods (again, with a case-insensitive match):


  doc http.listen.*


I am not proposing we adopt the existing doc tool; it has its own idiosyncrasies that should not exist in a generally available and supported program. But as a command-line tool it is nicer to use than godoc for most things already.


I propose we decide to create a new tool to ship with Go 1.5 and spend some time figuring out what its interface should be. The tool itself will be easy to write once the command-line interface is clear.


Here is a proposal for the command-line interface to include with 1.5. It is sure to be refined and extended but I want the starting point to be simple and intuitive. The list may seem long but the examples that follow will show they work well.


  • Ignore case in symbols: for symbols (not package specifiers) printf matches Printf.

  • If there are no arguments, show the full documentation for the package in the current directory.

  • If there is a single argument:

    • If the argument has no periods after any slash that may be present:

      • If it has a slash, show the full documentation for the package whose path is exactly the argument (encoding/binary, etc.)

      • Otherwise it is no slash or period; it is just alphanumeric:

        • If it matches an exported symbol in the current directory's package, show its documentation.

        • If it starts with a capital, it must be a symbol in the current directory, but matching is still case-independent (Html will match symbol HTML).

        • Otherwise if there is a package in $GOPATH with that package name, show its documentation.

        • Otherwise fail (there is room to do things here; we hold off for now).

    • Otherwise it has periods. Treat the first period not followed by a slash as the boundary between package and symbol. Use the rules above (package path vs. package name; an empty package specifier refers to the current package) to find the package, then treat the rest of the argument as a symbol, which must be non-empty:

      • If the symbol has a (second) period, it matches type.method (ignoring whether there is a * in the actual receiver),

      • Otherwise it matches a package-level symbol

  • For now, only one argument is accepted. I think doing multiple arguments confuses the matching rules because we would expect consistent heuristics across the arguments. Avoid the issue by allowing only one argument; it's a minor hardship.


Flags:

-list: the argument must specify a package. List its contents in a convenient greppable format.

-pkg: using the rules above, the argument (if any) must match a package; show its package doc comment only.

-std: treat the argument as a package in the standard library to disambiguate arguments that might refer to a local symbol (all packages outside the standard library can be disambiguated using a slash in the package specifier).


Examples:


go doc

Show the current package's documentation.

go doc -list

Show a compact representation of the exported symbols in the current package.

go doc fmt

Show the fmt package's documentation (unless there is a Fmt in the current directory).

go doc -std fmt

Show the fmt package's documentation.

go doc .fmt

Show the documentation for symbol Fmt (etc.) in the current directory.

go doc Fmt

Show the documentation for symbol Fmt in the current directory.

go doc -pkg fmt

Show the package doc comment for package fmt.

go doc -pkg

Show the package doc for the current directory.

go doc encoding/gob

Show the documentation for encoding/gob (and any other package with that package name).

go doc gob

Show the documentation for encoding/gob (only).

go doc gob.Decoder

Show the documentation for encoding/gob's Decoder type.

go doc gob.Decoder.Decode

Show the documentation for encoding/gob's Decoder type's method Decode.

go doc encoding/gob.Decoder.Decode

Equivalent to the previous.


As for what documentation to show, the output will be comparable to the existing godoc rules, for example to show factory-like functions if the symbol is a type.


The program will be installed as go tool doc and will have source in cmd/doc.


-rob


Yongjian Xu

unread,
Apr 17, 2015, 5:20:58 PM4/17/15
to Rob Pike, golan...@googlegroups.com
Can you give an example of what "compact representation" looks like?
 

go doc fmt

Show the fmt package's documentation (unless there is a Fmt in the current directory).

What's the difference b/w "go doc fmt" and "go doc -pkg fmt"?
 

go doc -std fmt

Show the fmt package's documentation.

go doc .fmt

Show the documentation for symbol Fmt (etc.) in the current directory.

go doc Fmt

Show the documentation for symbol Fmt in the current directory.

go doc -pkg fmt

Show the package doc comment for package fmt.

go doc -pkg

Show the package doc for the current directory.

go doc encoding/gob

Show the documentation for encoding/gob (and any other package with that package name).

go doc gob

Show the documentation for encoding/gob (only).

The current godoc seems to not allow any sub-pkg names to be specified directly. e.g. "godoc http" would not work. Does this example means that we will allow this behavior in the new "go doc"? What if we have a "foo/fmt" and a "bar/fmt"?

go doc gob.Decoder

Show the documentation for encoding/gob's Decoder type.

go doc gob.Decoder.Decode

Show the documentation for encoding/gob's Decoder type's method Decode.

go doc encoding/gob.Decoder.Decode

Equivalent to the previous.


As for what documentation to show, the output will be comparable to the existing godoc rules, for example to show factory-like functions if the symbol is a type.


The program will be installed as go tool doc and will have source in cmd/doc.


-rob


--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Fatih Arslan

unread,
Apr 18, 2015, 6:34:56 AM4/18/15
to Rob Pike, golan...@googlegroups.com
I would be happy if the proposal could also have a way of defining the
output format, like Oracle's `-format` flag. For example oracle has
three output formats: plain, json and xml. The first one is
editor-friendly and it's the one that vim-go uses for displaying the
output in the editor.

For "godoc", currently vim-go shows the whole output and then tries to
simply search for the package identifier (such as Println for the case
fmt.Println). It works but it's not perfect and ther are some work to
improve this situation: https://github.com/fatih/vim-go/issues/361

The current proposal is a welcome one, at least the output can be
modified via the input, such as "go doc gob.Decoder". However it would
be also great if we had the choice of output format too, so we could
plug it into other tools (such as editors).





--
Fatih Arslan

meta keule

unread,
Apr 18, 2015, 4:41:29 PM4/18/15
to golan...@googlegroups.com
I find the proposal much more confusing and harder to remember and understand (concerning all the rules regarding the interpretation of the single argument and the miriad of ways to specify a package) than the current cli options.

Why not just have *one* way to specify a package, like say -pkg or -p and if missing, take the package inside the current directory, if no package is specified and just have one argument that is the symbol that is searched for. Then have a default output mode and an optional -mode flag for other modes like "list", "doc" (documentation only) "sig" (signature only)? Maybe add a special flag -c to search for a combined input of pkg path and symbol. 

...

Matt Sherman

unread,
Apr 18, 2015, 9:09:20 PM4/18/15
to golan...@googlegroups.com
Navigating to the package directory, and simply typing go doc, should be the main use case. Doing so obviates a lot of the disambiguation that's required when packages/symbols are arguments.

And hey, shells offer a bit of autocomplete for paths. :)

...

Dmitri Shuralyov

unread,
Apr 19, 2015, 2:31:25 AM4/19/15
to golan...@googlegroups.com
I use and rely on godoc.org and Rob's doc tool (robpike.io/cmd/doc) quite a bit (and much less on go doc/godoc). Although I use the doc tool with only one form:

doc http.ListenAndServeTLS

So the general direction sounds great, as it'd be an improvement over the current go doc/godoc. But I'm not sure if that many options/modes are needed; I'd honestly be happy with just packagename.ExportedName mode (it doesn't even have to be case insensitive).
...

Steve McCoy

unread,
Apr 20, 2015, 5:01:08 PM4/20/15
to golan...@googlegroups.com
This is almost exactly how I think it should work. My only complaint is that "go doc" vs. "godoc" is something that'll take me until Go 3.0 to remember the distinction between, so it would be nice if the names could be more distinct. I can't think of any better name, though, so it's probably the best and I can deal with it, especially because I default to trying the subcommand first anyhow (a habit from the old "go doc" days).

Also, are these explanations swapped?

go doc encoding/gob

Show the documentation for encoding/gob (and any other package with that package name).

go doc gob

Show the documentation for encoding/gob (only).




On Friday, April 17, 2015 at 4:40:49 PM UTC-4, Rob Pike wrote:
...

Joe Shaw

unread,
Apr 23, 2015, 3:21:04 PM4/23/15
to golan...@googlegroups.com
Hi,

Would this proposal allow for the removal of the special cases in which vet and godoc are installed by "go get" into $GOROOT rather than $GOPATH?  That would leave only golang.org/x/tools/cmd/cover.

Thanks,
Joe


On Friday, April 17, 2015 at 4:40:49 PM UTC-4, Rob Pike wrote:
...

Russ Cox

unread,
Apr 24, 2015, 9:44:22 AM4/24/15
to Joe Shaw, golang-dev
On Thu, Apr 23, 2015 at 3:21 PM, Joe Shaw <j...@joeshaw.org> wrote:
Would this proposal allow for the removal of the special cases in which vet and godoc are installed by "go get" into $GOROOT rather than $GOPATH?  That would leave only golang.org/x/tools/cmd/cover.

Yes, vet and cover would move back into the main tree and be built during all.bash; no more need to go get those when working off-release.

Russ

Andrew Gerrand

unread,
May 3, 2015, 7:03:44 PM5/3/15
to Rob Pike, golang-dev
I'm generally in favour of this proposal.

Reply all
Reply to author
Forward
0 new messages