Who killed the docs?

[eve...@fireproofsocks.com]

Sorry to rant in my first post here, but my god, who killed the Golang docs?  Clearly this "documentation" style is intentionally idiomatic, but why?!?  The official docs @ https://golang.org/doc/ are categorically terrible. Full-blooded examples are nearly extinct there.  Why is it so hard for the Go cult to write docs that TEACH instead of TELL?    

I want to like Go, but it's really hard to forgive the needless amounts of time it wastes when we have to tinker around for hours trying to get any single task to work. There is simply no valid excuse for it.

It would do wonders for the language if the Go contributors could find it in their hearts to clean up the minefields of the cratered landscape and plant some real examples there instead.  The failures outlined in this article http://stevelosh.com/blog/2013/09/teach-dont-tell/ were frighteningly on the nose, and I can only guess how much productivity has suffered.

</rant>

[Ian Lance Taylor]

I'm sorry you don't like the docs. They need to serve multiple
purposes. The current docs are designed to work as a reference: you
know more or less how it works, and you want to see the details. This
plays into the new "go doc" command as well.

The docs do contain examples, as you probably know. We know that we
need more of them (e.g., http://golang.org/issue/5757). Want to write
some?

Ian

[Caleb Spare]

In addition to what Ian said: if you want to be taught, that's what
things like https://tour.golang.org/ and http://blog.golang.org/ are
for.

I, for one, appreciate that resources like the specification and the
stdlib docs are clear, concise, and precise, allowing me to quickly
locate the information I need. In fact, I much prefer Go's
documentation to any other language's that I've used.

On Tue, Jun 16, 2015 at 1:04 PM, <eve...@fireproofsocks.com> wrote:

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[John Weldon]

On Tue, Jun 16, 2015 at 2:05 PM, Caleb Spare <ces...@gmail.com> wrote:

In fact, I much prefer Go's
documentation to any other language's that I've used.

I agree.  The concise and clear docs on the official golang.org pages are some of my favourite technical docs of all time.

[Everett Griffiths]

Whether I like the docs or not is irrelevant: if their purpose is to educate or provide meaningful context for the functions they describe and ultimately to save time, then they are a failure.  Seldom do they provide more information than I get out of my IDE's code completion.  I'm hopping in between 3 or 4 languages over the past couple months and hands down, the docs for Go stand out as the least helpful, sparse to a fault, and contain by far the fewest examples.  The examples are few and seldom complete -- instead of demonstrating usage, at best they demonstrate only a fragment that sometimes takes significant time to figure out how it fits into an actual use-case. The playground is great, but feel like it's used as a crutch because you end up needing to use it to figure out how to actually get the function in question to work instead of simply having that demonstrated in a few lines.  I'm deliberately making a mischievous comment here, but in the area of documentation, PHP blows Go out of the water.

I would love to write some more examples, but seeing responses like those from Caleb and John I get the feeling that the types of docs I would write would not be welcome.  I cannot easily convey how frustrating it is to see docs like this -- it is a style that needlessly wastes so much time (I think the time wasted by myself and by my developers is not unusual).  Concise is fine, but that's what a summary/synopsis is for.  That should not come at the expense of omitting descriptions and relevant examples or fielding common errors.  Smokers shouldn't be bothered by a non-smoking section, and instead I feel like all windows and doors are being sealed.  The existing docs seem largely helpful only if you are already familiar with the functions.  tour.golang.org is helpful as a learning tool, but it's not built to be a reference: the take-away examples need to be in the docs, not relegated to some other site.  I would love to see more effort to integrate meaningful examples into the docs so that they are as well thought out as the Go language itself.  Even having a clear feedback loop would be a step in the right direction (e.g. a link where we could fork the doc repo or make comments).

[alb.do...@gmail.com]

I'm not OP, but I somewhat share his feelings.

Il giorno martedì 16 giugno 2015 23:06:15 UTC+2, Caleb Spare ha scritto:

In addition to what Ian said: if you want to be taught, that's what

things like https://tour.golang.org/ and http://blog.golif you want to be taught, that's what
things like https://tour.golang.ang.org/ are
for.

Yes, but the tour is very simple, and the blog doesn't cover everything:
it's simply no substitute for real code examples.

Take for example the math/big package. There's no mention of it in
the tour (obviously, it cannot cover every package in the standard
library), and there's no blog post about it.

Now what? The documentation is our last option. A 15-20 lines
code snippet that *does somethig real* would be tremendously
useful to someone that has never used math/big.Int. Something
that shows a whole *something* from start to end. A snippet where

a) you initialize a few big.Int (NewInt, etc.. )
b) you use operators on them (Add, Mul, Cmp)
c) you query Ints (Bits, Sign)
d) you print them (Format, ..)

But we don't have anything like that. A newcomer will have to settle for this
example of the SetString( ) function:

i := new(big.Int)
i.SetString("644", 8) // octal
fmt.Println(i)

There's another one (4 lines) for Scan( ). And that's it.

This is virtually useless to someone that wants to start using the
math/big package. He'll have to google it hoping that someone
on stackoverflow.com has written a 15 lines complete example
that uses the package.

I, for one, appreciate that resources like the specification and the
stdlib docs are clear, concise, and precise, allowing me to quickly
locate the information I need. In fact, I much prefer Go's
documentation to any other language's that I've used.

I agree, but examples are hidden by default, and having better examples
won't bloat the documentation. It'll remain "clear and concise"
 

[Michael Jones]

I’ve written many (~100) little programs of this sort. 

For big, for example, a Fibonacci tool and test suite:

Code: fibonacci.go

http://play.golang.org/p/An0BtNAsme

Tests: fibonacci_test.go

http://play.golang.org/p/DqpDtR2vT3

I’ve been thinking that we could have a “cookbook” for Go.

— 

Michael Jones, CEO  •  mic...@wearality.com  •  +1 650 656-6989 

Wearality Corporation  •  289 S. San Antonio Road  •  Los Altos, CA 94022

[Ian Lance Taylor]

On Tue, Jun 16, 2015 at 3:34 PM, Michael Jones <m...@wearality.com> wrote:
> I’ve written many (~100) little programs of this sort.
>
> For big, for example, a Fibonacci tool and test suite:
>
> Code: fibonacci.go
> http://play.golang.org/p/An0BtNAsme
>
> Tests: fibonacci_test.go
> http://play.golang.org/p/DqpDtR2vT3
>
> I’ve been thinking that we could have a “cookbook” for Go.

That would probably be good.

There is also http://gobyexample.com/ .

Ian

[alb.do...@gmail.com]

A cookbook would be nice but this kind of stuff (a Fibonacci
function a little simpler than yours would be the perfect example)
should be in the documentation, too, IMHO. Something
like those 4 examples on the top for the sort package:

http://tip.golang.org/pkg/sort/

[John Weldon]

On Tue, Jun 16, 2015 at 2:57 PM, Everett Griffiths <eve...@fireproofsocks.com> wrote:

I would love to write some more examples, but seeing responses like those from Caleb and John I get the feeling that the types of docs I would write would not be welcome.

I'm sorry for conveying that - contributions of examples would be very welcome.  I was mostly responding to the statement that the official docs were categorically terrible.   I think they're fantastic at what they do provide.  Sure we would love some more examples.  Sure we'd love more documentation to teach new users how to use the language.  I just don't think either of those ideals mean that the existing docs are not good.

[Ian Lance Taylor]

On Tue, Jun 16, 2015 at 2:57 PM, Everett Griffiths
<eve...@fireproofsocks.com> wrote:
>

> I would love to write some more examples, but seeing responses like those
> from Caleb and John I get the feeling that the types of docs I would write
> would not be welcome.

There are many different opinions on this mailing list--after all, it
has over 15,000 members. Please don't give up merely because some
people disagree with you. Still, it would probably be best to give an
example of what you would like to do before putting too much effort
into it.

Ian

[Benjamin Measures]

On Tuesday, 16 June 2015 21:47:57 UTC+1, Everett Griffiths wrote:

categorically terrible. Full-blooded examples are nearly extinct there.  Why is it so hard for the Go cult to write docs that TEACH instead of TELL?

I find it ironic that your rant is lacking in examples. Perhaps you too can teach instead of tell? 

[Andrew Gerrand]

It's funny, because our docs clearly work well for a particular kind of programmer. I regularly receive high praise for our docs. But then there are substantial criticisms like these, which we take seriously. 

The web site definitely needs work.

Part of that relates to package documentation; for instace, the math/big package's inscrutability could be easily fixed (I filed a bug to do so). If there are other packages that need specific attention, please (anyone) let me know by filing bugs.

The other part relates to teaching new users. For that we don't have a great story, apart from the tour. We have some open issues to improve the tour, branching into other areas. There are also blog key articles that need writing. But broader than that, we could structure the site better to funnel users toward the content they need; it's on my plate to sort that out over the next few months.

I appreciate any constructive feedback. Everett, can you tell us what what particular parts of the docs were lacking? What did you have trouble achieving? 

Andrew

​
​

[Caleb Spare]

I too was responding to hyperbolic claims in the original email about
the docs being "categorically terrible", and did not mean to suggest
that more examples would be bad.

A nice feature of the documentation site is that examples are
collapsed by default, so for the most part adding lots of examples
doesn't clutter the page too much.

[Dan Kortschak]

On Tue, 2015-06-16 at 14:57 -0700, Everett Griffiths wrote:
> Whether I *like* the docs or not is irrelevant:

It's entirely relevant. How a person responds to reference material
depends entirely on their preferred style of communication, to the
extent that some styles of writing completely miss some readers.

> if their purpose is to educate or provide meaningful context for the
> functions they describe and ultimately to save time, then they are a
> failure.

I'd strongly disagree here. I think the Go docs are some of the best I
have ever seen. They say what they need to without additional fluff and
provide links to the code (sometimes I wish there were a link directly
to the tests, but that's just a little typing away).

I have found the documentation in the most part to be refreshingly to
the point, making it easy to pick of the language and start with new
packages easily.

> Seldom do they provide more information than I get out of my IDE's
> code completion. I'm hopping in between 3 or 4 languages over the
> past couple months and hands down, the docs for Go stand out as the
> least helpful, sparse to a fault, and contain by far the fewest
> examples.

As has been stated, this is a work in progress.

> The examples are few and seldom complete -- instead of demonstrating
> usage, at best they demonstrate only a fragment that sometimes takes
> significant time to figure out how it fits into an actual use-case.

As is often the case in other languages, the test suites and other
people's code will provide the best resource for learning usage and
idiom.

> I'm deliberately making a mischievous comment here, but in the area of
> documentation, PHP blows Go out of the water.

Responding in kind, maybe that's because they need to have extensive
documentation to explain how to use a double clawed hammer... and they
don't have a history of properly testing.

>
> That should not come at the expense of omitting descriptions and
> relevant examples or fielding common errors.

Can you provide some examples where you think this has not been done?

> tour.golang.org is helpful as a learning tool, but it's not built to
> be a reference: the take-away examples need to be in the docs, not
> relegated to some other site.

The art of programming is taking generalised tools and specifying their
use. Yes, examples can help, but in the cases of truly generalised
tools, it is not trivial in terms of space to truly example all uses. It
is the programmer's job to understand the general behaviour without
being led through by the hand, and bend the tools to their service. It
seems that having really extensive examples would be treading down the
path to including how-to video links in the documentation
</tongue-in-cheek>.

[Benjamin Measures]

On Tuesday, 16 June 2015 22:57:44 UTC+1, Everett Griffiths wrote:

 tour.golang.org is helpful as a learning tool, but it's not built to be a reference

Indeed it is not, and as you point out neither is a reference a learning tool (at least not the kind you want).

http://golang.org/doc/ should be taken as a whole comprising two parts of http://golang.org/doc/#learning and http://golang.org/doc/#references .

It is akin to the first part teaching the language, and the second providing a dictionary. That you prefer the dictionary to be ordered by simple words first is another matter. For those that have learned the language, having a terse dictionary ordered alphabetically turns out to be the most useful reference.

[Benjamin Measures]

On Wednesday, 17 June 2015 00:28:47 UTC+1, Benjamin Measures wrote:

It is akin to the first part teaching the language, and the second providing a dictionary. That you prefer the dictionary to be ordered by simple words first is another matter. For those that have learned the language, having a terse dictionary ordered alphabetically turns out to be the most useful reference.

s/the language/a language/g 

[Volker Dobler]

Am Dienstag, 16. Juni 2015 23:57:44 UTC+2 schrieb Everett Griffiths:

[...] the docs for Go stand out as the least helpful, sparse to a fault, and contain by far the fewest examples.  The examples are few and seldom complete -- instead of demonstrating usage, at best they demonstrate only a fragment that sometimes takes significant time to figure out how it fits into an actual use-case.

Could you give specific examples?

Go's stdlib is broad and maybe there are packages which would

benefit from "larger" examples or from additional package level

examples. encoding/json comes to my mind but dismissing

all packages as having no useful documentation and lacking

relevant examples is a very strong statement. The package strings

and regexp do have several examples which I think do show

clearly how to use these packages.

V.

[Volker Dobler]

Am Mittwoch, 17. Juni 2015 00:19:02 UTC+2 schrieb alb.do...@gmail.com:

I'm not OP, but I somewhat share his feelings.

Il giorno martedì 16 giugno 2015 23:06:15 UTC+2, Caleb Spare ha scritto:

In addition to what Ian said: if you want to be taught, that's what
things like https://tour.golang.org/ and http://blog.golif you want to be taught, that's what
things like https://tour.golang.ang.org/ are
for.

Yes, but the tour is very simple, and the blog doesn't cover everything:
it's simply no substitute for real code examples.

Take for example the math/big package. There's no mention of it in
the tour (obviously, it cannot cover every package in the standard
library), and there's no blog post about it.

Now what? The documentation is our last option. A 15-20 lines
code snippet that *does somethig real* would be tremendously
useful to someone that has never used math/big.Int. Something
that shows a whole *something* from start to end. A snippet where

a) you initialize a few big.Int (NewInt, etc.. )
b) you use operators on them (Add, Mul, Cmp)
c) you query Ints (Bits, Sign)
d) you print them (Format, ..)

Assume there would be such a "real" example which includes NewInt, Add,

Mul, Cmp, Bits, Sign and Format. What would you tell someone who

feels this example is pointless as it does not explain how to use Quo

and Rsh?

 

But we don't have anything like that. A newcomer will have to settle for this
example of the SetString( ) function:

i := new(big.Int)
i.SetString("644", 8) // octal
fmt.Println(i)

There's another one (4 lines) for Scan( ). And that's it.

I _never_ used math/big so I am a newcomer but these examples make it

pretty clear to me how to use SetString and Scan. Do you really think it would

be helpful for a newcomer to add an example showing how to use SetString

in base 29?

 

This is virtually useless to someone that wants to start using the
math/big package. He'll have to google it hoping that someone
on stackoverflow.com has written a 15 lines complete example
that uses the package.

I would disagree (remember I am a newcomer to math/big, I never used it).

V. 

[Everett Griffiths]

Perhaps saying the docs are categorically terrible is too critical to elicit anything other than emotional responses, but put that in the context of me and my team having spent hours laboring over what should have been trivial tasks, and when developers are turning in huge time sheets AND scripts written in other languages (completed on their own time) that accomplish the same tasks that we have still failed to accomplish in Go, it's demoralizing and it indicates problems with the docs.

I'm happy to elaborate with specific examples, but first, let me frame this around the concept of education and teaching.  The Go docs are good at being a dictionary reference, but as anyone who has travelled abroad and learned to speak another language can tell you, a dictionary reference is not the most important thing to have: the most important reference is a phrasebook.  Nothing wrong with the dictionary, but it is in fact secondary (especially to newcomers). 

So in addition to #learning and #references being the core roles of the docs, I would include #usage as well (i.e. the phrasebook).  In practice, the phrasebook functionality usually gets merged in with the reference material.  To clarify my frustration, it is the features of the phrasebook that I feel are largely missing from the current Go docs, and that's what I feel is a huge impediment to learning the language. 

Having experience as a technical writer, a translator, and an instructor at the university level, I cannot overstate the importance of having these "phrases" to work with.  They become "thinking units", and they cement the cursory lessons outlined in the #learning articles AND they form the foundation of everything else that follows.  They are the underlying currency that enables learning of any language or skill -- some parts of the Go docs are bankrupt. 

Imagine the absurdity of teaching a baby to speak by reading it dictionary definitions A to Z.  And who has ever had a music lesson where instead of learning to play the instrument, you get your ears talked off about chromaticism, major scales, and enharmonic equivalents?  The article I linked to was especially apt about this music analogy (http://stevelosh.com/blog/2013/09/teach-dont-tell/).  And who doesn't remember those futile days from elementary school when you would go to look up a word like "solipsistic" in the big dead-tree dictionary, and finding the definition would be a wild goose chase starting with "a person who follows the principles of solipsism" etc.  While living abroad, I was so glad when I discovered a dictionary by Pon's that gave real functional definitions of each word. It's such a simple idea, but it's so effective.  I would even put money on it: give one set of developers the old-school dictionary documentation, give another the "phrasebook" style docs and I have little doubt which group would complete their tasks faster.  It's important to consider that virtually no programmer ever undertakes to learn every function in a language -- it's nearly always a hunt for "how do I do this?"  I.e. it's a phrasebook question, not a dictionary one, so to me it seems inappropriate to devote docs to a dictionary structure when that isn't representative of their primary usage.

Two common mistakes that I would warn against here were covered in Act 1 and Act 2 of the article: Source Code and Tests cannot substitute for documentation.  No harm in linking to it, but it fills a different need and fundamentally it's ineffective as a teaching strategy for beginners.  Why? Because most often, the source code and the tests are MORE complex with more moving parts, so to understand what's going on, you need to have a broader background.  Maybe the user doesn't understand exactly how tests work or maybe it's using some construct that's unfamiliar to the specific problem at hand.  Sure, having that perspective can help, but in order to provide a functional example (i.e. a phrasebook entry), the code needs to be reduced to the simplest possible example, where complexity is REMOVED, not added.

I'm not suggesting anything new: it's a formula that has worked successfully for many other languages.  E.g. consider your humble man page, and that it's no accident that many languages include the same basic segments in their docs.  Broadly speaking, Perl, PHP, Java docs have these sections: 

• NAME
  
• SYNOPSIS - function usage + signature
  
• DESCRIPTION 
  
• Parameters/Options
• Examples
• See Also

Go follows some of this at the package level, but zooming into the function level gets grainy.  By contrast, PHP implements this very well in my opinion (and I contend it has absolutely nothing to do with its architecture): the PHP docs include a detailed description of the PARAMETERS and output value (which is less important in Go), but they include the all important section for EXAMPLES, SEE ALSO, and USER CONTRIBUTED NOTES.  Go misses out a lot of explanations by being too concise with its parameters: knowing an argument's data type is not enough information to know how to use it.

Consider the popularity and ease of use of jQuery issuing Ajax requests (https://api.jquery.com/jQuery.get/) vs. the convoluted docs for Ext JS (http://docs.sencha.com/extjs/4.2.3/#!/api/Ext.Ajax).  The options are THERE in the docs, but it's a much larger mental leap to figure out how to connect the to the original method call in Ext JS.  Its usage suffers drastically.

So, now that I've laid out a bit more of the philosophy behind useful documentation, I can include some general comments that apply to everything in the Go docs:

1. Include a link for feedback on the documentation.  If it's managed via a github repo, include a link to it there, so at the very least, interested users could fork it and submit pull requests.  It's not clear at the moment where one could submit changes if they wanted to.

2. Explain any abbreviations.  I understand the desire to shorten code, but when abbreviations are used in package, function, and variable names, it would prevent a lot of mental friction to be clear as to what those terse tidbits were referring to.

3. Describe the parameters in meaningful ways.  It is not enough to know the data types: it takes several trips to the playground to do variations on the inputs to see what affect they have.  Why not just save the time and include the inputs and outputs next to different options?  

4. Include examples for each function.  If there's not an example for a function, there probably should be.  This one thing alone would revolutionize the docs.

Now, let's look at a specific package, e.g. https://golang.org/pkg/encoding/hex/

Already, we can see there are no examples here, making it tough for a newcomer to follow along with.  For example, it's not clear that the functions that are listed here need to be prefixed with the "hex." prefix.  Yes, that's a simple language thing, but if it's not re-enforced in the documentation with a few examples, it takes that much longer to learn.

Consider this (BAD/CONFUSING):

var ErrLength = errors.New("encoding/hex: odd length hex string")

What is that?  My first assumption is that this is some type of package variable because it's under the "Variables" heading.  It's only by comparing this to other packages that I can guess that this is a reference to a particular error message.  But without an example to see how it might arise, I have no idea what to do with this info or what exactly it pertains to.

BETTER: include an example of working with the returned variables to see how/why they might be relevant or omit this entirely and just list the message strings.  

BAD: func Dumper

This "crosses the beams" by referencing another package: "io".  Without an example to work off of, it's not clear how one might instantiate an io.Writer or what might be done with an io.WriteCloser.  We end up banging our heads on a different wall over at the "io" docs.

BETTER: include an example here so we aren't on a wild goose chase to figure out how to instantiate io.Writer objects.

BAD: func DecodeString

Again, no example, so immediately I have to pay the toll of fiddling with this to see how it works.  Perhaps an example like this:

    bytes, _ := hex.DecodeString("AE")
    fmt.Println(hex.EncodeToString(bytes)) // ae (lowercase)

It might be worth mentioning that the output is always lowercase, and it would be good to explain the behavior that pops up if your input is out of range, e.g. "XX".  This is why it would be useful to describe the parameters and output in their own dedicated sections.

BAD: func Decode

The explanation includes "Decode decodes src into DecodedLen(len(src)) bytes" -- nesting like that is more concise, but it's harder to follow.  I'm not entirely sure I understand what this function does.  Again, an example would help clarify this more than paragraphs of text.

BETTER: ???

Similar comments would apply to https://golang.org/pkg/strconv/ (or indeed, to every single bit of Go documentation I have looked at over the past 2 weeks). 

In conclusion, I think that the current organization and terseness of the docs really does affect usability of Go, and I would love to see the organizational strategy changed to something with more meat to it (e.g. the man page format) so that those developers who need additional explanations and examples can have them to work with.  That's really the best way to ensure idiomatic code writing: via repetitive demonstrations of it.

[Dan Kortschak]

I think you are still conflating documentation intended to learn from
with documentation intended to refer to. It is unreasonable to expect
that a person can go into reference material and get from it all the
things you are asking for (this from another educator).

Note also that one stage there was a link to feedback on each page, now
it is assumed that the user can navigate to the issues page. There is
also a Project page which describes how to file issues and how to
contribute (not by pull request).

[Everett Griffiths]

No, I am not conflating those at all -- there will always be a place for tutorials, how-to's, et al, and that's not what I'm requesting.  Asking for examples is not too much to ask from the docs -- it is, I contend, the absolute bare minimum.  Look at the docs for any bash function, PHP, Perl, Java, and Python: they all include coding examples which demonstrate functionality, provide clarifications, and reinforce coding standards and provide without exception exactly what I am asking for, so it's hardly unreasonable, it is downright common.  If there's any disadvantage to including that extra clarification, it is far outweighed by the advantages.  Please share your examples of a documentation syntax which is the most appropriate and useful for everyday "active reference" of a programming language.

[Henrik Johansson]

I come from the Java world and sure the conceptual docs structures you mention exists there but I still think the Go documentation is superior simply because of its simplicity. The Java eco system has a slew of out dated docs that never kept up to speed with the code and where this is not the case often the docs is missing or reduced to autogenerated JavaDoc with all that entails. This is what Go does but the Java format and tradition seems to actively discourage a good use. There is also the code and test code which are often neat examples. I agree that code and tests alone does not encompass all that a verbose and detailed documentation can bring but imho the idea that the go doc tool brings, to mix them, is clearly the way to go.

And, to be frank, it is an open source project and you are free to contribute. Please do! It may sound harsh but we were all noobs at Go at one time or another and we learned through what doc and examples and mailing lists there was. It getting expanded with more pedagogical doc (or alternative location) can only make it better.

ons 17 juni 2015 kl 08:12 skrev Everett Griffiths <eve...@fireproofsocks.com>:

No, I am not conflating those at all -- there will always be a place for tutorials, how-to's, et al, and that's not what I'm requesting.  Asking for examples is not too much to ask from the docs -- it is, I contend, the absolute bare minimum.  Look at the docs for any bash function, PHP, Perl, Java, and Python: they all include coding examples which demonstrate functionality, provide clarifications, and reinforce coding standards and provide without exception exactly what I am asking for, so it's hardly unreasonable, it is downright common.  If there's any disadvantage to including that extra clarification, it is far outweighed by the advantages.  Please share your examples of a documentation syntax which is the most appropriate and useful for everyday "active reference" of a programming language.

--

[Everett Griffiths]

How and where to contribute here is not at all obvious -- I'm still looking for any link re issues or project pages pertaining to docs. I've never spent so much time googling simple solutions in my life as I have the past 2 weeks wrestling with Go, and I reject the thought of this being a hallmark of "superior documentation".  As I said, there needs to be a clear link for how/where to contribute to the docs.

[Dan Kortschak]

(and now for the list)

Many examples *do* exist, many of the responders here have indicated
that more examples would be welcomed, there is even an issue for this.

However, you have asked for "Include examples for each function", you
have also pointed to things like "how one might instantiate an
io.Writer or what might be done with an io.WriteCloser" which are
clearly dealt with in the io package documentation (linked to in the
godoc wherever they are referred to except in comments). It seems that
all these things really should have been picked up during learning
(maybe using the tour or some of the other excellent tutorials).

Much of this comes down to preference obviously, but I (as an example)
far prefer the style of documentation for functions and methods use in
godoc over those in java, C/C++ or perl (python's is not bad).
Particularly lists of arguments with terse and barely English
annotation. You will find that all the arguments in a function usually
*are* described, just in flowing prose rather than dot points, with the
added advantage that if you do want to see exactly what is going on, the
source is one click away.

The first google hit for "golang contribute" for me is
https://golang.org/doc/contribute.html which a pretty extensive
explanation of how to contribute, the third hit is
https://golang.org/project/ which also points to how to raise issues.
The first hit for "golang issues" is
https://github.com/golang/go/issues.

[roger peppe]

It would be nice to have one or more examples for almost every
exported symbol in the Go standard library. Perhaps we
could
a) make a list of all the symbols that don't have examples
(perhaps in priority order, prioritising functions and methods
over values and constants)
b) make it trivial for people to submit examples.

The latter is problematic, as the Go contributor hoop
is a high one to jump through for Jo Random Gopher;
I wonder if a web page allowing people to enter
an example and click to sign the CLA might work.

cheers,
rog.

[Dan Kortschak]

Almost all or even most, yes - all seems excessive.

The priority thing is important (I had intended to mention it but forgot, so thanks for filling that omission).

[tech...@gmail.com]

Oh, and may I ask for a language tab switcher for those of us

who speak in different languages, i.e. Python. 

[Chris Hines]

Although I am an experienced Go programmer now, I remember the day I found https://gobyexample.com/. The side by side style to document simple examples about the language and some basic programming tasks really helped the learning process. I still believe that is one of the best formats for helping programmers coming from other languages to get started with Go.

Chris

On Tuesday, June 16, 2015 at 4:47:57 PM UTC-4, Everett Griffiths wrote:

Sorry to rant in my first post here, but my god, who killed the Golang docs?  Clearly this "documentation" style is intentionally idiomatic, but why?!?  The official docs @ https://golang.org/doc/ are categorically terrible. Full-blooded examples are nearly extinct there.  Why is it so hard for the Go cult to write docs that TEACH instead of TELL?    

[Tieson Molly]

I like the docs, but I would like to see a synopsis section like in Perl's plain old documentation (POD).  I find it very useful to have a practical example of how to use the code.   Here is an example from the Perl world   

https://metacpan.org/pod/Moo

[Ian Lance Taylor]

On Tue, Jun 16, 2015 at 10:22 PM, Everett Griffiths
<eve...@fireproofsocks.com> wrote:
>
> Now, let's look at a specific package, e.g.
> https://golang.org/pkg/encoding/hex/

Thanks for the specific suggestions. I opened
http://golang.org/issue/11254 .

I also opened http://golang.org/issue/11255 to consider letting people
make comments directly from doc pages.

Ian

[Lars Seipel]

On Wed, Jun 17, 2015 at 07:41:18AM -0700, Ian Lance Taylor wrote:
> I also opened http://golang.org/issue/11255 to consider letting people
> make comments directly from doc pages.

You already mention spam in the issue. There's another thing, though:
well-meaning, but wrong, comments. The PHP docs are plagued by this.
Curating the comments might turn out to be a huge amount of work.

Personally, I don't think it would pay off. The contribution process is
a better place for improving documentation. But that's just me.

[Ian]

While I wouldn't say the Go's docs are "terrible", I do notice they often don't explain things as well as they could. Part of it is the 1996-era stylesheet, but much of it is that for far too many functions and methods, delving into the source and heading to the playground are required! And, quite often, while reading the source, figuring out what "that variable" or "what this parameter" is doing, what it represents, etc, is an exercise in bafflement. That's due to the fondness for cryptically short names and an almost-always complete lack of explanation! If the source is going to be a part of doc, it needs better explanations within it! How anyone can expect to maintain some of that code after 6 months is beyond me, but that's another topic.

The concern about style might seem trivial, but after reading Apple's doc, or Microsoft's, I feel I am stepping backward in time with the Go docs. Apple's, in particular, are quite gorgeous and usable (there are, to be sure, rough spots). Take Symfony, for example. Its doc is complete, easy to read and comes with examples that make sense. I don't need to head to a REPL or the source code to figure what something does; I certainly don't need to build an example app to develop an understanding of it! (Well, I once did.) Django's doc is good; Rails doc has some problem areas, usually with the complex and abstract functionality, but it is generally really good, very usable and is strikingly good in its presentation. Sinatra is more primitive; its doc is almost as good [sic] as Go's. As any magazine or newspaper editor, or any decent web designer, will tell you - good style aids readability. It's essential for impressing someone, too! And as Go becomes more popular, creating a good first impression becomes all the harder when the doc looks like it came from two decades ago. It simply doesn't foster confidence in the language. And telling people to head to the source *and* all-but requiring the playground to understand something is not really an acceptable solution! Such things should be of last resort and a rather pungent "doc smell", if you will. 

Now, I'll be the first to admit that writing good doc is a challenging task; personally speaking, I'm dreadful at the job. One firm I worked at, years ago, hired a technical writer to help with such tasks; their code was easier to maintain than most! The coders didn't like her much, and they hated her meetings where they had to explain to her the whats and whys, but they were more receptive to her after a month or two when they had to go back and fix some bug or other in their own code! As I've frequently noted to my wife - she's a copywriter and truly understands the power of the written word to explain and convince - Go really does need a technical writer. Asking the community for help in improving the doc might help (you'll need a different contributor's agreement for that, by the way), but if they're anything like me, they've got a zillion other things that need doing first. Not least of which is documenting my own code! Getting the person writing the code to be more comprehensive is a start, but that often comes down to a choice of "code or doc?". Doc usually gets neglected because the code is needed and the doc can be done later... While I'm not about telling Google how to run its business, (he says as he does exactly that...) Google should provide a technical writer or two, a web designer or two, a stated goal and a deadline. While there's no business case to be made (or at least not one I'm willing to write), creating good, usable documentation should be considered essential for the continued popularity of Go. 

/Ian

[Serge Hulne]

If you like to learn with examples, here is a book (moreover a free PDF e-book) with plenty of good examples:

http://www.miek.nl/downloads/Go/Learning-Go-latest.pdf

Also, the documentation of a language is never the same as a manual (see, e.g. Java, Python, the Gnome glibc doc, etc...) 

Serge.

[Manlio Perillo]

Il giorno mercoledì 17 giugno 2015 20:05:39 UTC+2, Ian ha scritto:

> [...]

 

The concern about style might seem trivial, but after reading Apple's doc, or Microsoft's, I feel I am stepping backward in time with the Go docs. Apple's, in particular, are quite gorgeous and usable (there are, to be sure, rough spots). Take Symfony, for example. Its doc is complete, easy to read and comes with examples that make sense. I don't need to head to a REPL or the source code to figure what something does; I certainly don't need to build an example app to develop an understanding of it! (Well, I once did.) Django's doc is good; Rails doc has some problem areas, usually with the complex and abstract functionality, but it is generally really good, very usable and is strikingly good in its presentation. Sinatra is more primitive; its doc is almost as good [sic] as Go's.

Note, however, that Django (just to make an example) documentation is in a separate set of documents and this make it feasible to hire a technical writer.

Go standard (and only, AFAIK) documentation, on the other hand, is generated from the source code.

 

Regards  Manlio

[Josh Bleecher Snyder]

> It would be nice to have one or more examples for almost every
> exported symbol in the Go standard library. Perhaps we
> could
> a) make a list of all the symbols that don't have examples
> (perhaps in priority order, prioritising functions and methods
> over values and constants)

As a crude first pass at priority, I looked at what symbols from the
standard library show up in the text in a large corpus of open source
code. There are many problems with this as a priority metric, but
perhaps it could help as a very rough guide.

Here's the top of the list:

7.85% 448975 testing.T
4.80% 274738 fmt.Sprintf
4.35% 248967 fmt.Errorf
3.83% 219284 unsafe.Pointer
2.64% 150849 errors.New
1.58% 90341 net/http.Request
1.54% 87862 reflect.ValueOf
1.40% 79820 fmt.Fprintf
1.26% 72115 fmt.Println
1.14% 65303 time.Now
1.09% 62554 net/http.ResponseWriter
1.09% 62507 fmt.Printf
1.07% 61032 bytes.Buffer
1.00% 56924 time.Time

The output for everything with at least 50 uses is
https://gist.github.com/josharian/67e2c1e04479113c01a3. To reproduce
with your own corpus, grab http://play.golang.org/p/GUXUUlPLJv and run
'find . -name "*.go" | xargs -n 1 ~/stdlibusage | pct', where pct is
https://github.com/josharian/pct or your favorite output aggregator.

-josh

[Ian]

There's nothing to say that Go's doc *has* to be generated from the source code. Or that the doc that is included in the source can't be improved. Or even that separate documentation can't be written. It's a convention that developed into a (highly restrictive) rule for some reason; it *should* be changed! 

On Wednesday, June 17, 2015 at 2:49:04 PM UTC-4, Manlio Perillo wrote:

Il giorno mercoledì 17 giugno 2015 20:05:39 UTC+2, Ian ha scritto:

...

[Brendan Tracey]

Is there a clear guide on what the standard library documentation is supposed to be? I have found a lot of the documentation to be great once you already know what the function does, but if you don't it is frequently hard to learn from. An example from the sync package:

"A Mutex is a mutual exclusion lock. Mutexes can be created as part of other structures; the zero value for a Mutex is an unlocked mutex."

This documentation was not helpful the first time I attempted to use one. It was especially confusing with the mantra of "don't communicate by sharing memory". Of course, once I saw how it was used it's obvious memory has to be shared and I no longer understood what I was thinking before. Now that I know how it's used, the documentation is sufficient.

[Volker Dobler]

Am Mittwoch, 17. Juni 2015 20:05:39 UTC+2 schrieb Ian:

While I wouldn't say the Go's docs are "terrible", I do notice they often don't explain things as well as they could. Part of it is the 1996-era stylesheet, but much of it is that for far too many functions and methods, delving into the source and heading to the playground are required! And, quite often, while reading the source, figuring out what "that variable" or "what this parameter" is doing, what it represents, etc, is an exercise in bafflement. That's due to the fondness for cryptically short names and an almost-always complete lack of explanation! 

Could you give concrete examples in the standard library?

Fixing a specific problem (and some parts of the standard library

may have a too terse documentation) is much easier than

"just improve everything".

V.  

 

[yob...@gmail.com]

On Wednesday, June 17, 2015 at 12:12:05 AM UTC-6, Everett Griffiths wrote:

Look at the docs for any bash function, PHP, Perl, Java, and Python: they all include coding examples which demonstrate functionality, provide clarifications, and reinforce coding standards and provide without exception exactly what I am asking for, so it's hardly unreasonable, it is downright common. 

One thing to keep in mind is that these languages are >20 years old and have had quite some time to refine the examples in their documentation. Of course we would love to have an example for each method in the "encoding/hex" package. This will get infinitely better with time.

[Ian]

The *entire* standard library has that issue, Volker. It's not a specific issue; it's endemic to the language! 

While I wouldn't wish Objective-C naming upon Go, encouraging single letters to stand in for complex ideas is one of the language's conventions. While that works in some cases, in other cases, having one or two letters leaves you wondering what that letter actually stands for. And then if you're not the one who's maintaining that code, in a year or two, the maintainer will be sitting there wondering what "a" and "na" (for example) stood for! Sometimes it's obvious - i for index is a well-known convention; sometimes it's really not. 

I think of it this way - C has, by convention, cryptic naming. I can't think of a single interpreted language that encourages such practices; that's for a reason. That would be that the folk writing that particular language (whatever it is) reacted against cryptic naming, preferring to encourage names that communicate intent, etc. Arguably, that dreadful and complex naming system Microsoft used in the 1990's (I forget what it was called) was a reaction; ObjC naming certainly is! And while it might seem a little off-topic to this conversation about documentation, I felt it was worth pointing out simply because it is an issue if "read the source!" is part of the standard practice for understanding Go! After all, Java and PHP, at the very least, encourage the documenting of a function's parameters. 

While I realize that concrete examples are useful, I feel that would be, or lead to, bikeshedding, so hopefully I've managed to explain my point a bit better!

On Wednesday, June 17, 2015 at 4:54:12 PM UTC-4, Volker Dobler wrote:

Am Mittwoch, 17. Juni 2015 20:05:39 UTC+2 schrieb Ian:

[...]

[Andrew Gerrand]

On 18 June 2015 at 11:25, Ian <ian.gr...@gmail.com> wrote:

While I realize that concrete examples are useful, I feel that would be, or lead to, bikeshedding, so hopefully I've managed to explain my point a bit better!

As the primary maintainer of golang.org I will state clearly: it is very difficult to act on unfocused criticism.

Concrete, constructive, actionable feedback, however, is of great value. 

The specific examples in this thread have led to issues filed on the bug tracker, which have already led to new examples being written. Please provide more specific examples as issue reports. (If not now, then the next time you are confounded by something.)

Thanks,

Andrew

[Matt Silverlock]

> While I wouldn't wish Objective-C naming upon Go, encouraging single letters to stand in for complex ideas is one of the language's conventions. While that works in some cases, in other cases, having one or two letters leaves you wondering what that letter actually stands for. And then if you're not the one who's maintaining that code, in a year or two, the maintainer will be sitting there wondering what "a" and "na" (for example) stood for! Sometimes it's obvious - i for index is a well-known convention; sometimes it's really not

The "short variable naming" convention does not (to my knowledge) extend to public APIs in the standard library, besides perhaps function params.

Typically the short variable names are used in small functions as to keep them close to where they've been declared. If you've run into cases where you think that's not the case, or where naming could be improved for the sake of clarity, definitely raise it as an issue. 

[Dan Kortschak]

From what I can see the naming tends to follow the ideas put forward
here http://research.swtch.com/names.

On Wed, 2015-06-17 at 18:25 -0700, Ian wrote:
> While that works in some cases, in other cases, having one or two
> letters leaves you wondering what that letter actually stands for. And
> then if you're not the one who's maintaining that code, in a year or
> two, the maintainer will be sitting there wondering what "a" and
> "na" (for example) stood for! Sometimes it's obvious - i for index is
> a well-known convention; sometimes it's really not.

The name often needs to be read in the context, so the scope of the
context defines how descriptive the name needs to be. The context here
includes the type and the function that returned it. Both of these
things are readily at your fingertips.

> I think of it this way - C has, by convention, cryptic naming. I can't
> think of a single interpreted language that encourages such practices;
> that's for a reason. That would be that the folk writing that
> particular language (whatever it is) reacted against cryptic naming,
> preferring to encourage names that communicate intent, etc. Arguably,
> that dreadful and complex naming system Microsoft used in the 1990's
> (I forget what it was called) was a reaction; ObjC naming certainly
> is! And while it might seem a little off-topic to this conversation
> about documentation, I felt it was worth pointing out simply because
> it is an issue if "read the source!" is part of the standard practice
> for understanding Go! After all, Java and PHP, at the very least,
> encourage the documenting of a function's parameters.

Only in some places is the naming cryptic when you read the name in it's
context.

[Ian]

As I said - I'm avoiding bikeshedding. There are numerous instances throughout the source code where it's baffling what a variable name means, if it means anything. It comes out of the policy and convention for "short, not explanatory" variable and function names. From what I've read - quite a bit of the source - I could be here from now until Domesday listing examples; that's a waste of my time and yours.

My intention, Andrew, is not produce some "actionable" trouble ticket, but, in keeping with Everett's original point, to raise problems with *how* Go is documented. The standard response to many "I don't get how this works..." questions is "Read the source!" Well, that's not particularly helpful if the source code is rife with apparently meaningless variable names! It's an overall problem - if reading the source is how you gain understanding, but the source is confusing or obtuse because the variables make no sense... How is the reader to understand the source? This is particularly of concern because a stated goal of Go is to be easy to read. But if the variable names aren't easy to figure out, or lack any context, then... 

Here's an example, and it's merely an example; it's not a very good example, but it's the only one I could think of; it's late where I am: 

In strings.go:

// Split slices s into all substrings separated by sep and returns a slice of
   322	// the substrings between those separators.
   323	// If sep is empty, Split splits after each UTF-8 sequence.
   324	// It is equivalent to SplitN with a count of -1.
   325	func Split(s, sep string) []string { return genSplit(s, sep, 0, -1) }

Okay... What's genSplit?

// Generic split: splits after each instance of sep,
   274	// including sepSave bytes of sep in the subarrays.
   275	func genSplit(s, sep string, sepSave, n int) []string {
   276		if n == 0 {
   277			return nil
   278		}
   279		if sep == "" {
   280			return explode(s, n)
   281		}
   282		if n < 0 {
   283			n = Count(s, sep) + 1
   284		}
   285		c := sep[0]
   286		start := 0
   287		a := make([]string, n)
   288		na := 0
   289		for i := 0; i+len(sep) <= len(s) && na+1 < n; i++ {
   290			if s[i] == c && (len(sep) == 1 || s[i:i+len(sep)] == sep) {
   291				a[na] = s[start : i+sepSave]
   292				na++
   293				start = i + len(sep)
   294				i += len(sep) - 1
   295			}
   296		}
   297		a[na] = s[start:]
   298		return a[0 : na+1]
   299	}

Ah... What's sepSave? It's an int, but what does it do? sep is probably the separator; it's a string. c... (line 285) is... Char? Count? Oh, the first character of the separator. a... No idea. na... I'm not even going to guess that one. It's... How exactly do I describe "na" as I work my way through this code? Oh look - s. Where did that come from? Oh. It's in the parameter list. That makes it the string we're working on, right? What's n? Oh... It's listed in explode(...) 

    13	// explode splits s into an array of UTF-8 sequences, one per Unicode character (still strings) up to a maximum of n (n < 0 means no limit).
    14	// Invalid UTF-8 sequences become correct encodings of U+FFF8.
    15	func explode(s string, n int) []string {

That's not right, surely? Nope, it isn't. It's defined SplitN. Which isn't shown when you look at Split.

// SplitN slices s into substrings separated by sep and returns a slice of
   302	// the substrings between those separators.
   303	// If sep is empty, SplitN splits after each UTF-8 sequence.
   304	// The count determines the number of substrings to return:
   305	//   n > 0: at most n substrings; the last substring will be the unsplit remainder.
   306	//   n == 0: the result is nil (zero substrings)
   307	//   n < 0: all substrings
   308	func SplitN(s, sep string, n int) []string { return genSplit(s, sep, 0, n) }

--

That's an easy one (although I still have no idea what a and na are...); there are others, in reflect, in strings, in other packages, where reading the code is an exercise in hieroglyphic decoding. *If* code is to be a primary method of documentation, it really does need to be clearer. Especially as the people who are being urged to read it to understand how to use it (always a strange approach, but whatever) might not have English as a first language, might be trying to solve a problem and really don't want to be bothered with deciphering code in order to use it or simply might look at such an example and wonder why Go is considered such a modern language if it's using 1980's naming conventions! 

Overall, it's not a problem with GoDocs as a system. It's a problem with the Go source code. It's a peripheral issue that I thought worth mentioning, especially considering the general context of "read the code!", with the code being part of the general language documentation. Go is an interesting language; it has a lot going for it. Its documentation, especially when reading often obscure source code as a necessary part of that documentation, isn't one of those things. 

My apologies for the length and any obscurity in this; it is late and I did kind of rush through it!

On Wednesday, June 17, 2015 at 10:05:21 PM UTC-4, Andrew Gerrand wrote:

On 18 June 2015 at 11:25, Ian <ian.gr...@gmail.com> wrote:

While I realize that concrete examples are useful, I feel that would be, or lead to, bikeshedding, so hopefully I've managed to explain my point a bit better!

... 

Concrete, constructive, actionable feedback, however, is of great value.

... 

Thanks,

Andrew

[Ian Lance Taylor]

On Wed, Jun 17, 2015 at 10:27 PM, Ian <ian.gr...@gmail.com> wrote:
>
> The standard response to many "I don't get how this works..."
> questions is "Read the source!"

It seems pretty clear that at that point something has already gone
wrong. The source code is never going to be the documentation.

Ian

[Dan Kortschak]

On Wed, 2015-06-17 at 22:27 -0700, Ian wrote:
> That's an easy one (although I still have no idea what a and na
> are...);

a are the sub arrays (a slice though) and na is the index into a that is
being worked on; you can get this from the fact that a is the returned
value and that na (number of a) is indexing into a.

The rest of the parameters in genSplit are explained in the doc comment
for genSplit and Split.

[Andrew Gerrand]

I admit I'm a bit baffled by this response, because there's no reason any user should need to inspect the source of strings.Split in order to understand it. The documentation comment is very explicit about what the function does. Do you find it lacking somehow?

The source code is certainly not supposed to be a primary form of documentation. Doc comments and examples *are*, and I would appreciate pointers as to where they are lacking. For those of us who have worked with Go every day for years and years, it is hard for us to see this stuff with fresh eyes. I appreciate all the feedback.

Cheers,

Andrew

--

[Volker Dobler]

Am Donnerstag, 18. Juni 2015 07:27:05 UTC+2 schrieb Ian:

As I said - I'm avoiding bikeshedding. There are numerous instances throughout the source code where it's baffling what a variable name means, if it means anything. It comes out of the policy and convention for "short, not explanatory" variable and function names. From what I've read - quite a bit of the source - I could be here from now until Domesday listing examples; that's a waste of my time and yours. 

My intention, Andrew, is not produce some "actionable" trouble ticket, but, in keeping with Everett's original point, to raise problems with *how* Go is documented. The standard response to many "I don't get how this works..." questions is "Read the source!" Well, that's not particularly helpful if the source code is rife with apparently meaningless variable names! It's an overall problem - if reading the source is how you gain understanding, but the source is confusing or obtuse because the variables make no sense... How is the reader to understand the source? This is particularly of concern because a stated goal of Go is to be easy to read. But if the variable names aren't easy to figure out, or lack any context, then... 

Here's an example, and it's merely an example; it's not a very good example, but it's the only one I could think of; it's late where I am: 

Thanks for the example. Examples make abstract ideas visible and

concrete.

 

In strings.go:

// Split slices s into all substrings separated by sep and returns a slice of
   322	// the substrings between those separators.
   323	// If sep is empty, Split splits after each UTF-8 sequence.
   324	// It is equivalent to SplitN with a count of -1.
   325	func Split(s, sep string) []string { return genSplit(s, sep, 0, -1) }

Okay... What's genSplit?

genSplit is an _un_exported (private) helper function. genSplit is _not_ part of

of the documentation of strings.Split as you may check running go doc or visiting

golang.org/pkg/strings.

Are you okay with the actual documentation of Split itself? Is the description

verbose enough and are the parameters explained well enough? What remains

unclear? 

So your criticism is not so much about the documentation of the 

exported (public) functions, methods and types but of that the source

of the standard library's non-exported helper functions is too terse?

 

// SplitN slices s into substrings separated by sep and returns a slice of
   302	// the substrings between those separators.
   303	// If sep is empty, SplitN splits after each UTF-8 sequence.
   304	// The count determines the number of substrings to return:
   305	//   n > 0: at most n substrings; the last substring will be the unsplit remainder.
   306	//   n == 0: the result is nil (zero substrings)
   307	//   n < 0: all substrings
   308	func SplitN(s, sep string, n int) []string { return genSplit(s, sep, 0, n) }

What's wrong with the documentation of SplitN?

You mentioned missing cross references from Split to SplitN and

vice versa: Is your point here that some text like "With n < 0 SplitN

behave like Split (see above)." and in the documentation to Split

a reference like "If you want to limit how many parts are generated

use SplitN (see below)." would be helpful?

While I understand this wish, for me Split and SplitN are close

enough in the index as well as in the output. I look up SplitN

because I am unsure if sep or n comes first.

Maybe that's the difference between a tutorial ("See we've got these

function to chop strings apart.") and a library reference.

 

--

That's an easy one (although I still have no idea what a and na are...); there are others, in reflect, in strings, in other packages, where reading the code is an exercise in hieroglyphic decoding. *If* code is to be a primary method of documentation, it really does need to be clearer. Especially as the people who are being urged to read it to understand how to use it (always a strange approach, but whatever) might not have English as a first language, might be trying to solve a problem and really don't want to be bothered with deciphering code in order to use it or simply might look at such an example and wonder why Go is considered such a modern language if it's using 1980's naming conventions! 

Overall, it's not a problem with GoDocs as a system. It's a problem with the Go source code. It's a peripheral issue that I thought worth mentioning, especially considering the general context of "read the code!", with the code being part of the general language documentation. Go is an interesting language; it has a lot going for it. Its documentation, especially when reading often obscure source code as a necessary part of that documentation, isn't one of those things. 

My apologies for the length and any obscurity in this; it is late and I did kind of rush through it!

Let me make that clear first: I do _not_ think that someone should be

_requiered_ to read the whole source to understand what a (exported)

function does and I doubt this is what the community thinks when writing

documentation.

To me the unexported helper functions are by no means the "primary

method of documentation".  But for sure: If someone really doesn't 

understand what SplitN does after reading its docstring, then he _may_

dive into the guts of the implementation.

For me the documentation of the exported stuff in package strings is

fine and I doubt anyone will have to dig into the details of explode and

genSplit to understand how to use SplitN.

To my taste package encoding/hex is not documented enough.

(But I think the source of encoding/hex is pretty readable).

This thread was about the exported documentation and that it is lacking.

If it is lacking (as a reference), it should be improved and I believe this 

is undisputed consensus here.

If it is lacking one might consult the source and I think this should not be

the primary way of discovering how exported stuff works but it might be

okay for details or the true nerd who wants to understand all the gut details.

I hope this view is consensus here too.

If the exported documentation should be usable (also) as a tutorial

is at least controversial (I am against it).

But a discussion of how readable the Go standard library source is

should be a completely different thread.   

Regards

Volker

[Pranav Raja]

To my taste package encoding/hex is not documented enough.

(But I think the source of encoding/hex is pretty readable).

Although encoding/hex could certainly benefit from an example or two, I found the simple doc style quite approachable, even as a newcomer to the language.

On the other hand I struggled quite a lot with encoding/asn1, especially with the quite cryptic error messages. I think given its greater complexity, just a few examples of a successful and failed struct decodes would make this much easier to understand.

[Manlio Perillo]

Il giorno mercoledì 17 giugno 2015 21:47:33 UTC+2, Ian ha scritto:

There's nothing to say that Go's doc *has* to be generated from the source code. Or that the doc that is included in the source can't be improved. Or even that separate documentation can't be written. It's a convention that developed into a (highly restrictive) rule for some reason; it *should* be changed! 

I do have some suggestions for doc improvements:

1) Create a new golang-doc repository.

2) Decide about a documentation formatting tool (sphinx, asciidoc, ...).

3) Move documentation from golang/go/doc to the new repository, and reformat it using 2).

4) Select the best blog posts and move to new repository, in a tutorial section.

    With time add more tutorials (see Django docs).

5) Keep the tutorial up to date.

    This is important, some interesting blog posts are not up to date with recent go changes.

    An example is the blog post about the new text/template lexer.

6) Add a section about performance tips.

    This is important.  When looking for performance tips for high performance code, google

    returns several resources but I do not trust them since they are not up to date.

    In some slides the author says:

      - do not use builtin map, but use a custom hashmap

      - do not use channels in high performance sections of code

      - optimize the parser/lexer using a state machine

Thanks  Manlio

[Ian]

The thing is, Ian, the source *is* the documentation when the standard mantra is to tell someone "if you don't get it, go read the source!"

[Ian]

I get what it does, kortschak; what's the relationship between "a" and "sub arrays" and, in particular, what's the relationship between "na" and "index"? There isn't any! Not the mathematical or algebraical relationship, but *linguistic* one - the one many folk use to build mental models of how the parts fit together, as it were, in order to understand how some complex operation is constructed. (The split function isn't a complex operation; it literally just happened to be the only example I could think of.)

As I said, it wasn't the best example, but it was the only one I could think of at the time!

[Manlio Perillo]

Il giorno giovedì 18 giugno 2015 14:45:29 UTC+2, Ian ha scritto:

The thing is, Ian, the source *is* the documentation when the standard mantra is to tell someone "if you don't get it, go read the source!"

Try to understand how the go tools work just by reading the sources...

Regards   Manlio 

[DV]

On Wednesday, June 17, 2015 at 11:27:05 PM UTC-6, Ian wrote:

As I said - I'm avoiding bikeshedding. There are numerous instances throughout the source code where it's baffling what a variable name means, if it means anything. It comes out of the policy and convention for "short, not explanatory" variable and function names. From what I've read - quite a bit of the source - I could be here from now until Domesday listing examples; that's a waste of my time and yours.

My intention, Andrew, is not produce some "actionable" trouble ticket, but, in keeping with Everett's original point, to raise problems with *how* Go is documented. The standard response to many "I don't get how this works..." questions is "Read the source!" Well, that's not particularly helpful if the source code is rife with apparently meaningless variable names! It's an overall problem - if reading the source is how you gain understanding, but the source is confusing or obtuse because the variables make no sense... How is the reader to understand the source? This is particularly of concern because a stated goal of Go is to be easy to read. But if the variable names aren't easy to figure out, or lack any context, then... 

Here's an example, and it's merely an example; it's not a very good example, but it's the only one I could think of; it's late where I am: 

In strings.go:

// Split slices s into all substrings separated by sep and returns a slice of
   322	// the substrings between those separators.
   323	// If sep is empty, Split splits after each UTF-8 sequence.
   324	// It is equivalent to SplitN with a count of -1.
   325	func Split(s, sep string) []string { return genSplit(s, sep, 0, -1) }

Okay... What's genSplit?

"genSplit", having a lower-case letter, is an unexported function. An internal implementation detail of this package. Why do you want to know what it is or how it's implemented *without* reading the source? The entire point of an internal implementation detail is that it could change at any time. You seriously expect someone to update the docs any time an internal/private/unexported implementation detail changes? 

I'd be really curious to see examples of this, anywhere, in the wild. Java, C#, anything you can point to. 

The documentation for the public function, Split, I think is excellent. 

[Ian Lance Taylor]

On Thu, Jun 18, 2015 at 5:45 AM, Ian <ian.gr...@gmail.com> wrote:
> The thing is, Ian, the source *is* the documentation when the standard
> mantra is to tell someone "if you don't get it, go read the source!"

As far as I know, that is not the standard mantra. To the extent that
that is the standard mantra, then clearly we need to fix it--by fixing
the documentation. Fixing the source code in order to fix the
documentation is working at the wrong level.

Ian T

[Micky]

Looks like, you clearly must not have seen Python and Ruby docs. Or even C/C++ ones.

The only thing that I find sometimes irritating is implicit implementations to interfaces. But for that, we have lady "oracle" to enlighten us.

On Wed, Jun 17, 2015 at 1:04 AM, <eve...@fireproofsocks.com> wrote:

Sorry to rant in my first post here, but my god, who killed the Golang docs?  Clearly this "documentation" style is intentionally idiomatic, but why?!?  The official docs @ https://golang.org/doc/ are categorically terrible. Full-blooded examples are nearly extinct there.  Why is it so hard for the Go cult to write docs that TEACH instead of TELL?    

I want to like Go, but it's really hard to forgive the needless amounts of time it wastes when we have to tinker around for hours trying to get any single task to work. There is simply no valid excuse for it.

It would do wonders for the language if the Go contributors could find it in their hearts to clean up the minefields of the cratered landscape and plant some real examples there instead.  The failures outlined in this article http://stevelosh.com/blog/2013/09/teach-dont-tell/ were frighteningly on the nose, and I can only guess how much productivity has suffered.

</rant>

--

[Michael Jones]

Realize that I should have said… I love the documentation. I love looking at the source. (The design and implementation of the fmt and various io packages are high art.) Wanting more is fine, and I think worked examples are fantastic, but there is something important to understand about the meta issue here—who is the documentation designed for? Newcomers to programming? Newcomers to the UNIX and Linux traditions? Newcomers to concurrency and parallelism? Newcomers to compiled languages? Newcomers to the Algol/B/C/C++ line of expression? The “who” here is the critical question. 

For me, an old hand at each of the above, a glance is all I have needed to know. I’ve looked for “what is the Go name for the function that creates a pipe” and the like, but never wondered what a pipe is or failed to start with the knowledge of possible errors and conditions. Optimal documentation for people like me is about precision, concision, and searchability. I want a dictionary rather than an encyclopedia.

If I were a beginner in some dimension then I’d want more. I would welcome examples above all. This is how people learn new. We build a bridge from the known to the unknown by playing with the pieces. Working code that can be modified is is acme of this path. The docs are not rich in this and more would be better, as is the by-example website mentioned before, and as would be the cookbook I proposed. Very many users would gain from this kind of effort. I’m eager to help. But I should have said that I really like what is already there. Would regret losing its virtues as we’re developing examples.

Michael

— 

Michael Jones, CEO  •  mic...@wearality.com  •  +1 650 656-6989 

Wearality Corporation  •  289 S. San Antonio Road  •  Los Altos, CA 94022

[Dan Kortschak]

I'm not the author of that code, so I can't speak authoritatively to that, but a is the first letter of array and n is the first letter of number. However, the context matters here and tells you how to read names. Would you expect to be able to understand any given text's lexicon purely by reading the words out of context? The answer to that might surprise you.

[Ian]

"Would you expect to be able to understand any given text's lexicon purely by reading the words out of context? The answer to that might surprise you."

I don't understand your point, kortschak. Or, more to the point, I don't understand what a dictionary has to do with my general point about labeling within Go? Are you using "lexicon" in a colloquial way that I've perhaps missed? 

"a" is, as you note, the first letter of array, but an array of what? How is it intuitive to grasp that that particular "a" is an array of some sort?  And "na" is not especially informative, considering it's an index... However those names are defined, they are most assuredly not intuitively labeled. 

[Andrew Gerrand]

Ian, Dan,

The style in the standard library varies from package to package as it was written by many authors. This particular piece of code was written and reviewed by people who take code quality and readability seriously, and it passed their muster. They clearly have different aesthetics to Ian, but that is fine. Many equally valid viewpoints on internal naming conventions exist.

As has been said a couple of times already, "read the source" is most certainly not our answer to lacking documentation. We know our docs could be better and we want to improve them wherever we can. So far, as a result of this thread, examples have been added to three packages, with more on the way.

I don't see how a continued discussion about Go's internal naming conventions could be productive, and it's certainly off topic here.

Thanks,

Andrew

[Dan Kortschak]

No worries. I'll answer Ian's last off list and close.

[Ian]

[...]

As far as I know, that is not the standard mantra.  To the extent that
that is the standard mantra, then clearly we need to fix it--by fixing
the documentation.  Fixing the source code in order to fix the
documentation is working at the wrong level.

Ian T

It's common enough that I've come to think of it as the default reproach whenever someone asks about a function, etc, Ian. I'm so used to seeing it, I'm mildly bemused when I don't see it in response to some code-related question. It's become, I think, something of the "Go way", as it were - "if in doubt, read the code!" It is a perfectly acceptable response, by the way! Oftentimes, all that is needed is the urging to read the code. But if the code is difficult to read..?

In and of itself, using the source code as part of the documentation is not really an issue. But if the source is part of the documentation, then that must be a consideration in the design of the code. Would it be be useful to ask if the source should or shouldn't be the definitive documentation for the users of the language? (It clearly and obviously should be for the designers and writers of the language!) 

[Ian]

   325 func Split(s, sep string) []string { return genSplit(s, sep, 0, -1) }

Okay... What's genSplit?

"genSplit", having a lower-case letter, is an unexported function. An internal implementation detail of this package. Why do you want to know what it is or how it's implemented *without* reading the source? The entire point of an internal implementation detail is that it could change at any time. You seriously expect someone to update the docs any time an internal/private/unexported implementation detail changes? 

I'd be really curious to see examples of this, anywhere, in the wild. Java, C#, anything you can point to. 

The documentation for the public function, Split, I think is excellent. 

I think you missed a couple of things, DV. I've stated, even when I used it, that this was a poor example. And if the implementation is to be definitive documentation, then, yes, I really would need to know the details of an unexpected function such as genSplit. Split(...) relies on it! The comment for Split is adequate.

[Ian]

Just saw this.

I'll agree to disagree on whether this conversation is off-topic or not. (It is a side issue, but it is relevant to main topic.) I will refrain from any more conversation on this topic, on and off list.

[Peter Kleiweg]

I think with a few modifications in godoc, package documentation could be the perfect place for both reference and tutorial:

 1. Support for markdown, both in general package documentation (Overview) as well as for items in the index.

 2. Listing of sections from the Overview in the top menu, or at the top of the Overview section itself.

You can have both a general introduction and a tutorial with examples in the "Overview" section of a package. But you need more mark-up for this, so you can make sections, link to function documentation, define a language on a code block so you can have proper syntax coloring, use lists, tables, inline images that are part of the package as well as external. Basicly, all that is needed to write a longer, good tutorial.

[Peter Kleiweg]

Op vrijdag 19 juni 2015 15:57:36 UTC+2 schreef Peter Kleiweg:

I think with a few modifications in godoc, package documentation could be the perfect place for both reference and tutorial:

 1. Support for markdown, both in general package documentation (Overview) as well as for items in the index.

 2. Listing of sections from the Overview in the top menu, or at the top of the Overview section itself.

And the godoc package index should be easier to navigate. Like this:  http://pkleiweg.home.xs4all.nl/go/godoc-demo.html

[Manlio Perillo]

Il giorno venerdì 19 giugno 2015 15:57:36 UTC+2, Peter Kleiweg ha scritto:

I think with a few modifications in godoc, package documentation could be the perfect place for both reference and tutorial:

 1. Support for markdown,

No thanks.

I have seen too many source code with heavy formatting docstrings, where the formatting code visually "dominates"  the actual documentation.

Personally I find this very unpleasant.

If you need formatting, please write a separate documentation,  where the user will see the generated document and not the source code.

Regards  Manlio

[Peter Kleiweg]

Op vrijdag 19 juni 2015 16:45:58 UTC+2 schreef Manlio Perillo:

Il giorno venerdì 19 giugno 2015 15:57:36 UTC+2, Peter Kleiweg ha scritto:

I think with a few modifications in godoc, package documentation could be the perfect place for both reference and tutorial:

 1. Support for markdown,

No thanks.

I have seen too many source code with heavy formatting docstrings, where the formatting code visually "dominates"  the actual documentation.

Personally I find this very unpleasant.

That's why you should markdown. It doesn't dominate like other mark-up. A document woth markdown mostly reads like plain text.

 

If you need formatting, please write a separate documentation,  where the user will see the generated document and not the source code.

 

The point of godoc is that the user sees the formatted documentation, not the source code. (And for large package documentation, you should use a separate sourcefile anyway.)  It is very valuable to have documentation generated from the source code, because you have all the documentation of all extra packages you install in a single place. 

[Matt Harden]

I would like to say that I too find the Go docs to be, as a whole, refreshingly clear and concise compared to documentation I've seen other languages, including Python. I would not make any massive changes. A few more examples here and there, sure.

--

[Dmitri Shuralyov]

I am not familiar with the context, but I just can't resist closing this tab without leaving this quick note:

I am extremely happy with the quality of Go docs. I'd really like to echo Matt Harden's comment above. After all the languages/docs I've seen in the past, I really really appreciate how concise yet complete docs are. When people ask a question, I often feel like I could make a bet that there's going to be a sentence that answers their exact question in the docs. I'm rarely disappointed.

Go Docs are one of my favorite aspects of Go.

[Serge Hulne]

I agree entirely with Dmitri, besides

1. A doc is not manual.

2. Just compare the Python doc and the Go doc on a given (non-trivial) subject:

Go:
https://golang.org/ref/mem

Python:
https://docs.python.org/2/library/threading.html
https://docs.python.org/2/library/multiprocessing.html

(The same is true a fortiori for Java and C++)

The Go documentation is far more clear, concise and to the point,

Serge.

[Milan P. Stanic]

On Fri, 2015-06-19 at 20:52, Matt Harden wrote:
> I would like to say that I too find the Go docs to be, as a whole,
> refreshingly clear and concise compared to documentation I've seen other
> languages, including Python. I would not make any massive changes. A few
> more examples here and there, sure.

I agree with you except about examples. Go doc should describe functions
and data types in shortest and clear form as possible. Examples and
teaching/learning material belongs to books, articles, blogs ...

IMHO (in short) even examples doesn't belong to the godoc (source files)
but I don't want to propose any change to current state of it, although
I would be pleased by some simple markup in it.

> On Fri, Jun 19, 2015 at 11:57 AM Peter Kleiweg <pkle...@xs4all.nl> wrote:
> > Op vrijdag 19 juni 2015 16:45:58 UTC+2 schreef Manlio Perillo:

[Manlio Perillo]

Il giorno sabato 20 giugno 2015 11:08:59 UTC+2, mps ha scritto:

> [...] 

IMHO (in short) even examples doesn't belong to the godoc (source files)
but I don't want to propose any change to current state of it, although
I would be pleased by some simple markup in it.

I agree.

The godoc should be as short as possible, and, above all, should not distract the programmer who is reading the code.

The only exception is the package level godoc, but if possible it should be moved to a separate doc.go file.

Regards  Manlio 

[Nate Finch]

I think more examples would definitely be welcome.    Honestly, if your team was wondering what to do with an io.WriteCloser,. it means they were exceedingly new to Go, and some confusion just comes with the territory.  As a new gopher, my biggest problem was not that I couldn't understand the docs, it was that I didn't know where the common functions were that I wanted.  Reading the standard library's documentation will familiarize you with where those handy functions like strings.Join or ioutil.ReadAll or io.Copy, or handy types like bytes.Buffer or bufio.Scanner.

Although there are some tricky bits to the standard library, I think that for the most part, it only takes a few minutes per package to understand how it works.  Sure, the first time you use one of the encoding/* packages, you wouldn't know the patterns they expect you to follow.

For the record, I love the formatting of Godoc, I even wrote an article in godoc about godoc here: http://godoc.org/github.com/natefinch/godocgo ... I think it is exactly the right amount of formatting... mostly because it's just plaintext, which is awesome.  I don't want sigils all through my code. And I don't really want people getting all fancy with their documentation.  You can do enough with the formatting we have.

[Manlio Perillo]

l giorno sabato 20 giugno 2015 13:17:49 UTC+2, Nate Finch ha scritto:

I think more examples would definitely be welcome.    Honestly, if your team was wondering what to do with an io.WriteCloser,. it means they were exceedingly new to Go, and some confusion just comes with the territory.

What we need here are well organized tutorials (see Django docs).

Someone new to go should:

1) Read the tutorial

2) Read documentation about specific topics.

    Currently this is scattered between http://golang.org/doc/ and

    http://blog.golang.org/

3) Browse the godoc documentation for the packages he need

Regards  Manlio

[Benjamin Measures]

Thanks for really taking the time to teach.

On Wednesday, 17 June 2015 06:22:59 UTC+1, Everett Griffiths wrote:

So in addition to #learning and #references being the core roles of the docs, I would include #usage as well (i.e. the phrasebook).  In practice, the phrasebook functionality usually gets merged in with the reference material.  To clarify my frustration, it is the features of the phrasebook that I feel are largely missing from the current Go docs, and that's what I feel is a huge impediment to learning the language.

I'd actually heartily agree with you here. Go has IMO some great introductory material and excellent reference material but is a little short on phrase books or worked exercises that build vocabulary.

I think, perhaps, the the reason for this is that, given a language and dictionary, the Go community is itself still learning and evolving phrasing, or idiomatic Go. The articles at http://blog.golang.org/ (in part) document this evolution as practice yields idioms that "feel" right, with http://blog.golang.org/context serving as a case-in-point.

Where I think we may differ is that I really like the way that learning, phrasing and references are distinct document sets, each volume serving their use-cases with precision rather than some mega-document (eg. the PHP documentation) containing a confluence that serves to cause readers to thumb through and gain superficial knowledge with speed.

Seeing as there's a need, I might start something for worked exercises in Go in the style of collaboration.

[Joel Jensen]

One thing that seems to not be mentioned often is how great the Effective Go section of the documentation is. I used it every five minutes while learning Go.

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/eV3uNlEFPrc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.

[Sam Boyer]

I've arrived late to this party, but I think I can make some concrete suggestions, and since Andrew seems to be looking for some, I'll give it a shot.

First, I think Michael has identified one crux of the issue here:

On Thursday, June 18, 2015 at 12:44:14 PM UTC-4, Michael Jones wrote:

Realize that I should have said… I love the documentation. I love looking at the source. (The design and implementation of the fmt and various io packages are high art.) Wanting more is fine, and I think worked examples are fantastic, but there is something important to understand about the meta issue here—who is the documentation designed for? Newcomers to programming? Newcomers to the UNIX and Linux traditions? Newcomers to concurrency and parallelism? Newcomers to compiled languages? Newcomers to the Algol/B/C/C++ line of expression? The “who” here is the critical question. 

Total agreement. I also love the docs, and looking at the source. And, you can't write docs without a "who" (the OP's referenced article also makes this point). And defining that for a language isn't easy; Michael's examples only scratch the surface of the perspectives we could imagine. But I think there is a definition by which the stdlib docs are written, albeit an implicit one. It's drawn from some things that experienced gophers take for granted:

1. Go is simple, and orthogonal. Low-level TIMTOWTDI is not how we roll.
2. Simple and orthogonal means there's a relatively small number of highly generative basics. Knowing them pays big dividends.
3. Non-first-time programmers can usually grok most of Go's fundamentals pretty quickly - a couple days to a week, or so. The tour can get you there. (And this assumption is why so many have pointed to the tour in this thread)
   
4. The rabbit holes are usually pretty shallow.
   
5. Source code is transparent - no invisible bizarro type transformations, no possibility of unused imports or variables, no complex visibility rules, no error/exception bubbling (...ish). In short, no magic.
6. Most (stdlib) packages aren't merely bundles of related functions, but a coherent API around a concept.

Many of these ideas are stated in newcomer documentation, but they are deep concepts that require time and experience to root in peoples' minds. Plus, what language doesn't like singing its own praises? On my first read, I was 50/50 that these grand assertions were bullshit.

In any case, these assumptions translate, I think, into the following beliefs about readers of godoc:

1. They know the language basics, as defined by the tour.
2. They know that those basics are sufficient for reasoning about almost anything in the language. Similarly, they know that the rules they've learned will apply consistently, because no magic. This isn't Scala.
3. They know that rabbit holes are safe to chase, because a) they probably won't go terribly deep and b) most conceptual dependencies quickly get back to one of Go's fundamental building blocks. And, as I mentioned earlier, knowing the basics pays big dividends.
   
4. They know that the language spec is not arcane, and referring back to it is normal, and not terrifying.
5. They know that packages tell a story as a whole, and should be interpreted as such.

I think these are reasonable "who" assumptions for godoc. Not because they're always true, but because they're generally attainable, and they fit so uniquely well with the Go ecosystem/language. I suspect that adding enough to them such that every function becomes a reasonable entry point for newcomers would drastically undermine the simplicity that Henrik and many others find so resoundingly positive, as well as the singular cohesion to which Go's ecosystem can lay claim.

But that leaves open a gap, into which those who haven't yet ticked our "who" boxes fall. Everett is in this boat. He demonstrated as much with his encoding/hex example: he was concerned about having to chase down io.Writer. In another language, I might share his dread, but in Go it's more OK, because I know I can trust that rabbit hole is probably shallow. Plus, io.Writer is one of those "basics" (the tour covers-ish it) that it pays to grok. Examples could help - and of his requests, examples make by far the most sense to me. It's clear they also make sense to literally everyone else, so...duh, let's do that. Maybe even a rule - no new exported identifiers (or just funcs) in stdlib without a decent example? 

But I don't think his other recommendations fit, as they're based on the belief that Go is as discoordinated an ecosystem as, say, PHP - and that it therefore makes sense to make a little island out of each bit of docs. Even user comments - I'd rather see them be turned into PRs to create examples, which get reviewed and then committed. Rather, we should focus on what can be done to impart frustrated godoc readers with the necessary knowledge to use what's there docs effectively. And again, telling doesn't work - we have to show them. An ideal solution should be present on godoc pages themselves, so that readers can organically discover why godoc is organized as it is, and why the 

There are a lot of possibilities here, but to KISS: package docs. Perhaps we could aim to include an overview of the 'prerequisite knowledge' for any stdlib package in its package docs, including links as appropriate - perhaps to other packages, or perhaps to something from the blog, or maybe an apropos wikipedia page or two. Whatever. So long as it provides the frustrated reader with a broader, and reliably complete, sense of the context in which a given package exists, and from that, the steps they'll need to take to understand it. If done well, such intros could demonstrate all five of the "who" beliefs I defined above.

cheers

s

[Sam Boyer]

I've arrived late to this party, but I think I can make some concrete suggestions, and since Andrew seems to be looking for some, I'll give it a shot.

First, I think Michael has identified one crux of the issue here: 

On Thursday, June 18, 2015 at 12:44:14 PM UTC-4, Michael Jones wrote:

Realize that I should have said… I love the documentation. I love looking at the source. (The design and implementation of the fmt and various io packages are high art.) Wanting more is fine, and I think worked examples are fantastic, but there is something important to understand about the meta issue here—who is the documentation designed for? Newcomers to programming? Newcomers to the UNIX and Linux traditions? Newcomers to concurrency and parallelism? Newcomers to compiled languages? Newcomers to the Algol/B/C/C++ line of expression? The “who” here is the critical question. 

Total agreement. I also love the docs, and looking at the source. And, you can't write docs without a "who" (the OP's referenced article also makes this point). And defining that for a language isn't easy; Michael's examples only scratch the surface of the perspectives we could imagine. But I think there is a definition by which the stdlib docs are written, albeit an implicit one. It's drawn from some things that experienced gophers take for granted:

1. Go is simple, and orthogonal. Low-level TIMTOWTDI is not how we roll.
2. Simple and orthogonal means there's a relatively small number of highly generative basics. Knowing them pays big dividends.
3. Non-first-time programmers can usually grok most of Go's fundamentals pretty quickly - a couple days to a week, or so. The tour can get you there. (And this assumption is why so many have pointed to the tour in this thread)
   
4. The rabbit holes are usually pretty shallow.
   
5. Source code is transparent - no invisible bizarro type transformations, no possibility of unused imports or variables, no complex visibility rules, no error/exception bubbling (...ish). In short, no magic.
6. Most (stdlib) packages aren't merely bundles of related functions, but a coherent API around a concept.

Many of these ideas are stated in newcomer documentation, but they are deep concepts that require time and experience to root in peoples' minds. Plus, what language doesn't like singing its own praises? On my first read, I was 50/50 that these grand assertions were bullshit.

In any case, these assumptions translate, I think, into the following beliefs about readers of godoc:

1. They know the language basics, as defined by the tour.
2. They know that those basics are sufficient for reasoning about almost anything in the language. Similarly, they know that the rules they've learned will apply consistently, because no magic. This isn't Scala.
3. They know that rabbit holes are safe to chase, because a) they probably won't go terribly deep and b) most conceptual dependencies quickly get back to one of Go's fundamental building blocks. And, as I mentioned earlier, knowing the basics pays big dividends.
   
4. They know that the language spec is not arcane, and referring back to it is normal, and not terrifying.
5. They know that packages tell a story as a whole, and should be interpreted as such.

I think these are reasonable "who" assumptions for godoc. Not because they're always true, but because they're generally attainable, and they fit so uniquely well with the Go ecosystem/language. I suspect that adding enough to them such that every function becomes a reasonable entry point for newcomers would drastically undermine the simplicity that Henrik and many others find so resoundingly positive, as well as the singular cohesion to which Go's ecosystem can lay claim.

But that leaves open a gap, into which those who haven't yet ticked our "who" boxes fall. Everett is in this boat. He demonstrated as much with his encoding/hex example: he was concerned about having to chase down io.Writer. In another language, I might share his dread, but in Go it's more OK, because I know I can trust that rabbit hole is probably shallow. Plus, io.Writer is one of those "basics" (the tour covers-ish it) that it pays to grok. Examples could help - and of his requests, examples make by far the most sense to me. It's clear they also make sense to literally everyone else, so...duh, let's do that. Maybe even a rule - no new exported identifiers (or just funcs) in stdlib without a decent example? 

But I don't think his other recommendations fit, as they're based on the belief that Go is as discoordinated an ecosystem as, say, PHP - and that it therefore makes sense to make a little island out of each bit of docs. Even user comments - I'd rather see them be turned into PRs to create examples, which get reviewed and then committed. Rather, we should focus on what can be done to impart frustrated godoc readers with the necessary knowledge to use what's there docs effectively. And again, telling doesn't work - we have to show them. An ideal solution should be present on godoc pages themselves, so that readers can organically discover these connections.

There are a lot of possibilities here, but to KISS: package docs. Perhaps we could aim to include an overview of the 'prerequisite knowledge' for any stdlib package in its package docs, including links as appropriate - perhaps to other packages, or perhaps to something from the blog, or maybe an apropos wikipedia page or two. Whatever. So long as it provides the frustrated reader with a broader, and reliably complete, sense of the context in which a given package exists, and from that, the steps they'll need to take to understand it. If done well, such intros could demonstrate all five of the "who" beliefs I defined above.

cheers

s 

[Everett Griffiths]

Well, after a vacation from the thread, I penned a more thorough response on the topic: https://medium.com/@craftsmancoding/wtfm-dbbec02eb172

In short, I continue to find the Golang docs consistently lacking, and the thought of them as "high art" or other flattery comes across as absurd to me.  I repeat my assertion that this is not a series of isolated problems here or there that some poor contributor can patch with a long list of issue reports, but it is a failure in the overall structure and approach of the documentation.

I invite the group here to organize a test that will measure the doc's efficacy in terms of time it requires an unfamiliar user to complete a given task.  (I have done this with my developers, and the existing Golang docs failed significantly, but it's worth repeating that test with variations and larger test groups).

To summarize my points:

- Examples are the most important part of the docs.  Everything else is secondary.

- Docs are written primarily for newcomers, either to the language or to the package.

- Good formatting will streamline the "zoom level" so that experienced users can skim them for reference and less experienced users can drill down into more information. 

- A clear feedback mechanism is required where users can submit changes/fixes.

- Cross-referencing is critical (e.g. a "See Also" section)

- Automatically generated docs are inefficient for 2 reasons: examples culled from source code are not optimized for the same audience as examples optimized for education, and storing docs inside the source code makes the feedback loop less efficient.

The next step is to write an alternate version of the docs for one or more of the Golang packages and then measure the time it takes a group of unfamiliar developers to complete a task.  Although some people in this thread may disagree with my assertions, I am confident that more such tests will completely vindicate my comments.  Let the tests prove it.

[Jan Mercl]

On Wed, Jul 15, 2015 at 4:52 PM Everett Griffiths <eve...@fireproofsocks.com> wrote:

> - Examples are the most important part of the docs. Everything else is secondary.

Correctness and completeness is primary. Everything else is secondary.

> - Docs are written primarily for newcomers, either to the language or to the package.

Docs are written primarily for non-beginners. Beginners should start with some tutorials, how-tos, FAQ lists etc. Digging the real information hidden between explanations of everything a beginner may not understand destroys the practical value of the docs for anyone using it as a daily reference - _not_ as a learning material.

> - Good formatting will streamline the "zoom level" so that experienced users can skim them for reference and less experienced users can drill down into more information. 

Trying to please everyone often pleases no one. Dictionaries probably don't blend well with literature history texts.

> - A clear feedback mechanism is required where users can submit changes/fixes.

See: https://github.com/golang/go/issues/new

> - Cross-referencing is critical (e.g. a "See Also" section)

Web view from godoc is type cross-linked, even across package boundaries and is also hyperlinking text that looks like URL. What else do you have in mind?

> - Automatically generated docs are inefficient for 2 reasons: examples culled from source code are not optimized for the same audience as examples optimized for education, and storing docs inside the source code makes the feedback loop less efficient.

Once again, IMO package docs as well as examples are like 90% reference and, sometimes, 10% education. They are written with some level of experience in mind.

> The next step is to write an alternate version of the docs for one or more of the Golang packages and then measure the time it takes a group of unfamiliar developers to complete a task. Although some people in this thread may disagree with my assertions, I am confident that more such tests will completely vindicate my comments. Let the tests prove it. 

Godocs for the Go stdlib is quite probably the best documentation of any PL stdlib I have ever worked with in my lifetime. It could be made still better, sure. But changing the documentation into a beginners material would make it, IMHO, worse. (Was tempted to say it would ruin it, but changed my mind ;-)

--

-j

[roger peppe]

On 15 July 2015 at 10:52, Everett Griffiths <eve...@fireproofsocks.com> wrote:
> Well, after a vacation from the thread, I penned a more thorough response on
> the topic: https://medium.com/@craftsmancoding/wtfm-dbbec02eb172
>
> In short, I continue to find the Golang docs consistently lacking, and the
> thought of them as "high art" or other flattery comes across as absurd to
> me. I repeat my assertion that this is not a series of isolated problems
> here or there that some poor contributor can patch with a long list of issue
> reports, but it is a failure in the overall structure and approach of the
> documentation.
>
> I invite the group here to organize a test that will measure the doc's
> efficacy in terms of time it requires an unfamiliar user to complete a given
> task. (I have done this with my developers, and the existing Golang docs
> failed significantly, but it's worth repeating that test with variations and
> larger test groups).
>
> To summarize my points:
>
> - Examples are the most important part of the docs. Everything else is
> secondary.

A careful reader could generate examples from the reference docs,
but the other way around is generally not possible.
That should be true of the code too - just as with the Go language
itself, the reference documentation should come first and imply
the code. That's the reason I agree with Jan that the reference docs should
be considered primary.

Whenever I'm implementing a Go package, I'll start by trying
to write the reference documentation - it shines a useful light
on whether what I'm trying to do is coherent or not.

That said, I think we should have many more examples, particularly
in some of the darker corners of the Go stdlib. Examples are a great
way to elucidate possible uses of the available primitives.

Note that it is not possible for examples to demonstrate all possible
uses of the provided APIs - we provide nice general mechanisms
that can be put to many uses, but you'll need a little thought to do so.
I don't think that's a bad thing - it's called programming.

cheers,
rog.

[Everett Griffiths]

Per my article, correctness and completeness needs to come in the context of examples, and I believe you misunderstand what I mean here by documentation -- "beginner's material" is not the same as writing docs with newcomers in mind.  Examples are the best way to establish common ground here between users of all levels, per Pimsleur and Suzuki.  Lots of hard data to back that up to the point that I'm honestly surprised that this is a point of debate at all.

Re feedback, thank you for demonstrating my point:  a link to https://github.com/golang/go/issues/new is not listed on any of the Golang documentation pages (unless I'm missing something).  Feedback for the Go docs is doubly problematic when it has gatekeepers who do not see some of the omissions as problems but "features".  

Likewise, re cross-referencing, this is practically non-existent in the Go docs.  E.g. it might be good to know while reading up on the hex package that strconv might be useful to accomplish similar tasks.  There is no "See Also" or comparable section in the current format.  That costs valuable time and I can see no justification for the omission.

Opinions like "Godocs for the Go stdlib is quite probably the best documentation of any PL" is not actual data.  I remain confident that further tests will continue to show how inefficient they are for accomplishing real coding tasks. 

[Robert Melton]

On Wed, Jul 15, 2015 at 12:27 PM, Everett Griffiths <eve...@fireproofsocks.com> wrote:

Per my article, correctness and completeness needs to come in the context of examples, and I believe you misunderstand what I mean here by documentation -- "beginner's material" is not the same as writing docs with newcomers in mind.

I suspect I am misunderstanding, are you legitimately arguing that writing examples BEFORE reference documentation makes sense?  You expect someone to infer the meaning of parameters based on "enough examples", to infer things it can and can't do by trial and error via examples?  There is a reason reference material has to go first, it is a single truth on what the function does, from that an infinite numbers of examples can be spawned.

--

Robert Melton | http://robertmeta.com

[Bakul Shah]

If I may recast what Everett is saying... Reference material as well as examples are needed. Beginners learn faster from examples and even advanced users can read examples to quickly refresh a concept (if I am reading a man page I always look for examples first). Often examples can indicate how to use an API or a language feature effectively or show common idioms. Completeness of reference material by necessity needs to also deal with corner cases and errors however rare. Beginners can get lost in the details. I think there is value in having examples in addition to reference stuff right in the same place as they can clarify things. Before you can get something right, you first have to "get it"!

Now even if you agree with this, the main problem is that writing clear & complete documentation is hard. Adding examples is actually relatively easy but people have to see the value in doing so.

[roger peppe]

On 15 July 2015 at 13:54, Bakul Shah <ba...@bitblocks.com> wrote:
> If I may recast what Everett is saying... Reference material as well as
> examples are needed. Beginners learn faster from examples and even advanced
> users can read examples to quickly refresh a concept (if I am reading a man
> page I always look for examples first). Often examples can indicate how to
> use an API or a language feature effectively or show common idioms.
> Completeness of reference material by necessity needs to also deal with
> corner cases and errors however rare. Beginners can get lost in the details.
> I think there is value in having examples in addition to reference stuff
> right in the same place as they can clarify things. Before you can get
> something right, you first have to "get it"!
>
> Now even if you agree with this, the main problem is that writing clear &
> complete documentation is hard. Adding examples is actually relatively easy
> but people have to see the value in doing so.

I agree entirely. We need lots more examples! But they should not
be considered "primary" IMHO.

[Matt Harden]

OK, I'll take the fall. I killed the docs! I admit it!

[Joel Jensen]

On Wed, Jul 15, 2015 at 10:27 AM, Everett Griffiths <eve...@fireproofsocks.com> wrote:

Re feedback, thank you for demonstrating my point:  a link to https://github.com/golang/go/issues/new is not listed on any of the Golang documentation pages (unless I'm missing something).  Feedback for the Go docs is doubly problematic when it has gatekeepers who do not see some of the omissions as problems but "features".  

It looks to me like the current system of user feedback on documentation and the golang maintainers responding to those issues is pretty healthy. (see https://github.com/golang/go/issues?q=label%3ADocumentation )

Are you having a problem with your issues in the issue tracker being closed by "gatekeepers" in a non-constructive way?

[Micky]

Instead of criticizing, why not you submit a pull request with a few examples you miss?

There's something called Open Source.

--

[Florin Patan]

On Wednesday, July 15, 2015 at 4:52:06 PM UTC+2, Everett Griffiths wrote:
> Well, after a vacation from the thread, I penned a more thorough response on the topic: https://medium.com/@craftsmancoding/wtfm-dbbec02eb172
>
>

And here I was hoping for a: I came back from holiday and instead of pointless blogs about things I already said I did my first PR on Go docs PR. Oh well, I guess we'll have to wait for your next holiday.

Now, on a serious note: if you do want to help the ecosystem, sometimes it's better to lead by example. Show how some existing issues can be fixed and others will follow. Or at least your contributions will help out a new person and that would be perfect. You do the talk but can you do the walk?

[Christoph Seufert]

My 5 cents in the discussion:

I somehow agree to the OP. 

The go documentation is good for everyday reference if you are already quite familiar with a package and the standard library in general.

That it lacks totally: Helping to understand the concepts of a package.

For example: How URLs are matched against patterns in a hierarchy of ServeMuxes: From the documentation i actually don't get a clue. I even think that's the reason why people jump so quickly to libraries like Gorilla - for me that at least was true.

What i would wish for:

Apple does that quite well: You have the API Reference documentation and you have guide-documents. 

So i would like a (for example) HTTP-Programming guide (mainly the general package description quite much expanded). But i would like it to be separated from the API reference to not pollute that for everyday lookups.

[Carlos]

Some CLs expanding examples:

https://go-review.googlesource.com/#/c/11249/

https://go-review.googlesource.com/#/c/11371/

https://go-review.googlesource.com/#/c/11992/

Some CLs expanding examples already merged:

https://go-review.googlesource.com/#/c/11246/

https://go-review.googlesource.com/#/c/11240/

https://go-review.googlesource.com/#/c/11201/

[Pranav Raja]

Per my article, correctness and completeness needs to come in the context of examples

Examples are the best way to establish common ground here between users of all levels, per Pimsleur and Suzuki. 

When learning a new language, my primary goal is to imitate a native speaker (or close enough), even if I don't understand the roots of the language yet. This is why a phrasebook is so handy.

But when I'm writing code to use a library, my primary goal is to develop a mental model of how it works, and the assumptions it makes, so that I know the correct and simplest way to apply it to my specific use case.

A phrasebook, or set of examples, would be of little use here.

To make this more concrete, imagine if the python datetime docs, instead of opening with an explanation of the data structures and scope of the module, opened with a bunch of examples illustrating the edge cases. It would be terribly difficult to get anything done - I'd have to go read the code to figure out what was going on.

I remain confident that further tests will continue to show how inefficient they are for accomplishing real coding tasks. 

I agree with the need for objective testing, however, the trouble with productivity tests, like benchmarks, is that the results are only valid in the context of a set of assumptions, which in general don't apply to all or even most situations.

[Francisco Dalla Rosa Soares]

To be very honest, I've learned, first with Android then with Go, that sometimes Google made documentation might be lacking but that the source code normally is delightful to read, so it's just easier to go there. This is both a compliment and criticism. I promise to get back here with examples.

On Thu, Jun 18, 2015 at 3:45 PM, Andrew Gerrand <a...@golang.org> wrote:

I admit I'm a bit baffled by this response, because there's no reason any user should need to inspect the source of strings.Split in order to understand it. The documentation comment is very explicit about what the function does. Do you find it lacking somehow?

The source code is certainly not supposed to be a primary form of documentation. Doc comments and examples *are*, and I would appreciate pointers as to where they are lacking. For those of us who have worked with Go every day for years and years, it is hard for us to see this stuff with fresh eyes. I appreciate all the feedback.

Cheers,

Andrew

On 18 June 2015 at 15:27, Ian <ian.gr...@gmail.com> wrote:

As I said - I'm avoiding bikeshedding. There are numerous instances throughout the source code where it's baffling what a variable name means, if it means anything. It comes out of the policy and convention for "short, not explanatory" variable and function names. From what I've read - quite a bit of the source - I could be here from now until Domesday listing examples; that's a waste of my time and yours.

My intention, Andrew, is not produce some "actionable" trouble ticket, but, in keeping with Everett's original point, to raise problems with *how* Go is documented. The standard response to many "I don't get how this works..." questions is "Read the source!" Well, that's not particularly helpful if the source code is rife with apparently meaningless variable names! It's an overall problem - if reading the source is how you gain understanding, but the source is confusing or obtuse because the variables make no sense... How is the reader to understand the source? This is particularly of concern because a stated goal of Go is to be easy to read. But if the variable names aren't easy to figure out, or lack any context, then... 

Here's an example, and it's merely an example; it's not a very good example, but it's the only one I could think of; it's late where I am: 

In strings.go:

// Split slices s into all substrings separated by sep and returns a slice of
   322	// the substrings between those separators.
   323	// If sep is empty, Split splits after each UTF-8 sequence.
   324	// It is equivalent to SplitN with a count of -1.
   325	func Split(s, sep string) []string { return genSplit(s, sep, 0, -1) }

Okay... What's genSplit?

// Generic split: splits after each instance of sep,
   274	// including sepSave bytes of sep in the subarrays.
   275	func genSplit(s, sep string, sepSave, n int) []string {
   276		if n == 0 {
   277			return nil
   278		}
   279		if sep == "" {
   280			return explode(s, n)
   281		}
   282		if n < 0 {
   283			n = Count(s, sep) + 1
   284		}
   285		c := sep[0]
   286		start := 0
   287		a := make([]string, n)
   288		na := 0
   289		for i := 0; i+len(sep) <= len(s) && na+1 < n; i++ {
   290			if s[i] == c && (len(sep) == 1 || s[i:i+len(sep)] == sep) {
   291				a[na] = s[start : i+sepSave]
   292				na++
   293				start = i + len(sep)
   294				i += len(sep) - 1
   295			}
   296		}
   297		a[na] = s[start:]
   298		return a[0 : na+1]
   299	}

Ah... What's sepSave? It's an int, but what does it do? sep is probably the separator; it's a string. c... (line 285) is... Char? Count? Oh, the first character of the separator. a... No idea. na... I'm not even going to guess that one. It's... How exactly do I describe "na" as I work my way through this code? Oh look - s. Where did that come from? Oh. It's in the parameter list. That makes it the string we're working on, right? What's n? Oh... It's listed in explode(...) 

    13	// explode splits s into an array of UTF-8 sequences, one per Unicode character (still strings) up to a maximum of n (n < 0 means no limit).
    14	// Invalid UTF-8 sequences become correct encodings of U+FFF8.
    15	func explode(s string, n int) []string {

That's not right, surely? Nope, it isn't. It's defined SplitN. Which isn't shown when you look at Split.

// SplitN slices s into substrings separated by sep and returns a slice of
   302	// the substrings between those separators.
   303	// If sep is empty, SplitN splits after each UTF-8 sequence.
   304	// The count determines the number of substrings to return:
   305	//   n > 0: at most n substrings; the last substring will be the unsplit remainder.
   306	//   n == 0: the result is nil (zero substrings)
   307	//   n < 0: all substrings
   308	func SplitN(s, sep string, n int) []string { return genSplit(s, sep, 0, n) }

--

That's an easy one (although I still have no idea what a and na are...); there are others, in reflect, in strings, in other packages, where reading the code is an exercise in hieroglyphic decoding. *If* code is to be a primary method of documentation, it really does need to be clearer. Especially as the people who are being urged to read it to understand how to use it (always a strange approach, but whatever) might not have English as a first language, might be trying to solve a problem and really don't want to be bothered with deciphering code in order to use it or simply might look at such an example and wonder why Go is considered such a modern language if it's using 1980's naming conventions! 

Overall, it's not a problem with GoDocs as a system. It's a problem with the Go source code. It's a peripheral issue that I thought worth mentioning, especially considering the general context of "read the code!", with the code being part of the general language documentation. Go is an interesting language; it has a lot going for it. Its documentation, especially when reading often obscure source code as a necessary part of that documentation, isn't one of those things. 

My apologies for the length and any obscurity in this; it is late and I did kind of rush through it!

On Wednesday, June 17, 2015 at 10:05:21 PM UTC-4, Andrew Gerrand wrote:

On 18 June 2015 at 11:25, Ian <ian.gr...@gmail.com> wrote:

While I realize that concrete examples are useful, I feel that would be, or lead to, bikeshedding, so hopefully I've managed to explain my point a bit better!

... 

Concrete, constructive, actionable feedback, however, is of great value.

... 

Thanks,

Andrew

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[RickyS]

One style of documentation that you probably like is the PHP documentation.  See https://secure.php.net/docs.php.

It is huge at 57 megabytes.  The Go docs are 4 megabytes.  Plus 7 megs of blog.

The PHP docs are full of app notes on fine points and how-tos.  The very best part of the PHP language/system is the documentation.   It is very well organized and presented.  It clearly is curated by a professional team.  This group of doc writers must be larger than 'The Go Authors'. There could be 100 lines of documentation for each line of code.

On the other hand, PHP is such a mess it simply would not have survived without that enormous documentation effort. 

If someone would put together an effort of that quality for Go and its libraries, it would be a great boon.  But it wouldn't be nearly as fat.