Julie's blog post was first and foremost a request for feedback, so
thanks to Dan and everyone else for sharing your thoughts on this
thread. As I hope the post made clear, the discovery site is not done
yet, we know some ways it needs to be made better, and more
importantly we know we are probably blind to others, hence the call
for feedback. Please bear with us while we digest that feedback and
work on fixes. Please also keep in mind that, as I've noted before, we
are all pulling in the same direction. We all want what's best for Go.
I also want to say thanks for keeping us honest about all of this. The
last year has been hectic and we haven't been completely clear about
some of the reasons for recent work. (I take full responsibility for
this and am definitely not faulting anyone else on the Go team!)
It's very helpful when people call us out on that. The high standards
you all demand truly are one of my favorite things about open source.
This mail is a bit long but I hope it addresses most of the discussion
over the weekend. If you feel I'm missing anything, please reply and
I'm happy to continue the discussion.# Background about go.dev
One thing we've come to understand over the past year or so is that
many in the next wave of Go adopters want a “one-stop” web site full
of resources about Go, including how to get started, who is using it
already, links to learning resources, package docs, and so on. The newgo.dev
is meant to be that one-stop site. In the blog announcing it,
we called it “a new hub for Go developers
.”# Why is go.dev different from golang.org?
Personally, I thought that separating the two would allow go.dev
more inclusive of community content. Historically, golang.org
place where the Go project speaks authoritatively about Go: it has the
language spec, standard library docs, official downloads, and so on.
It has always been important to us not to dilute that with all the
other Go content in the world. The new site seemed to be an
opportunity to create a more inclusive place for other resources,
hence a second site.# Background about godoc.org
Gary Burd created and ran godoc.org
starting in late 2012. It was and
remains both a brilliant idea and clearly a valuable service to the Go
community. After a year or so, Gary decided he didn't want to run and
pay for the servers anymore, so he gave it to Go project to adopt. We
were happy to do that, to make sure this resource remained available
to all Go users. We said it back in 2014
when we adopted godoc.org
I will say it again: I am incredibly grateful to Gary for what he
created.# Why is there a new package docs site at all? Why not update
godoc.org in place?
With the introduction of modules and the notion of multiple versions
of a package, we knew we had to update the godoc.org
a hard look, it seemed worth starting anew, especially since thegodoc.org
server design, with its single-VM database, had been
starting to show its age. In addition to the modules work, there are
other things we're addressing, such as accessibility and overall
scalability of the service.
As a side note, there's almost nothing in the Go distribution that has
survived eight years without being redone. The compiler, the
assembler, the linker, the go command itself, most of the standard
library: all of them have been massively overhauled one or more times
since the start of Go. That's how we take what we learn and make
This kind of rewrite always involves a transition period in which the
old version is still the workhorse that most people use and the new
version has a new name for early adopters to test and find bugs in.# Why is the new package docs site on pkg.go.dev?
Docs for all the packages in the entire ecosystem are exactly the kind
of community-generated content that go.dev
is meant to help find, sopkg.go.dev
seemed like a good name. Especially since go.dev
has a much
broader scope than godoc.org
, it makes sense to take the opportunity
to fold it in and reduce the number of sites a typical user has to be
aware of (once the transition is complete).# Why are you talking about redirecting godoc.org to pkg.go.dev
when pkg.go.dev is still so broken?
Bluntly, so that you will help us understand what's broken, so we can
fix it before
the redirect happens. We know about some things but
would not be surprised to find there are things we're completely
unaware of. Better to find out earlier rather than later. Again, the
blog post was first and foremost a request for feedback about what
needs to happen before we actually do the redirect.# Why does pkg.go.dev require a detected license to show docs?
Why doesn't godoc.org?
The teams working on the proxy and on pkg.go.dev
have spent a lot of
time talking to Google's lawyers about what we can and can't do with
Go source code downloaded from the internet. The rule we've been given
to follow is that serving a pretty HTML version of the docs is
displaying a modified version of the original, and we can only do that
if there's a recognized known-good license that gives us that
When we adopted godoc.org
from Gary Burd back in 2014, it did not
occur to any of us to put it through that kind of review. If we had,
maybe the community would have gone through this licensing pain
earlier. For now we are focusing on making changes to pkg.go.dev
rather than correcting past mistakes on godoc.org
. (At this point,
more scrutiny of what godoc.org
does is not likely to have an outcome
that anyone likes.)# What fraction of popular packages don't display on pkg.go.dev?
Right now it looks like pkg.go.dev
sees 1,200 modules imported by at
least 100 other modules. Of those, it looks like 82 are flagged as not
redistributable, so that we can't show their docs. That's under 7%,
and we're working to understand that better. If any of those are
mistakes on our end, we'll fix them.
Another thing that was suggested that I think is a great idea is to
change the “no docs available” page to have a command-line to bring up
the docs in your own local godoc command.
[Aside: Speaking generally (not just about Go), you would be surprised
at the number of software packages that have a declared "license type"
in some kind of metadata (GitHub meta, package.json, SPDX codes, etc)
but no actual license text. This makes the license impossible to comply
with! For example, the MIT license requires that "The above copyright
notice and this permission notice shall be included in all copies or
substantial portions of the Software." But if there's no such notice
to include, only an "// SPDX-License-Identifier: MIT" comment, there's
literally no way to comply. It can be a real mess. If you have never before
encountered the differences between how programmers see the world
and how lawyers do, let me recommend "What Colour are your bits?
"]# Why hasn't pkg.go.dev been open sourced?
There's no conspiracy here. The original plan was to open source it,
but open sourcing puts pressure on the code base to be reusable in
other contexts. Right now the code is only written for one context:
the global pkg.go.dev
I helped edit the blog post and was responsible for the "Willpkg.go.dev
be open-sourced so I can run it at work for my private
code?" heading. I apologize for the offense I caused so many of you.
I did not mean to imply that there were no other reasons to run a doc server,
only to highlight as an example what I thought was the most common
reason Go developers would want the code.
In much the same way that we published an open source reference
implementation of the proxy that was not the global proxy site, I
still hope that we will publish an open source reference
implementation of a module-aware package docs site that need not be
exactly the global site. An implementation that doesn't have to hit
the same global scale can be much simpler to run—whether it's at work
with private code or on your laptop with completely open-source code.
I also hope the same server can serve index queries that make tools
like goimports and gopls faster. In contrast pkg.go.dev
serve such queries, at least not the same way, for both scaling
concerns and privacy concerns.
So the reason in the blog post is the real reason: the current code is
not what you probably should run, whether at work or on your laptop in
offline mode, and I thought we could deliver a much better answer for
But I very much hear all of you who want to see the code that is
running on the site and possibly contribute to it, whether it makes sense
to run in other contexts or not. I am going to look into that.
Again, thanks for calling us out when we're not communicating clearly
with all of you. It really helps. I hope this mail helps too, and if
not, let me know.