Upgrading ThinkUp's documentation

[Gina Trapani]

Last weekend at CodeConf I attended a fantastic talk by one of the
Django founders, Jacob Kaplan-Moss, called "Documentation is not
optional." In it, he railed against open source projects which have
incomplete and out-of-date documentation, and said that Django's
voluminous documentation was a big part of its success. Newsrooms were
simply more likely to use Django because it was so thoroughly and
clearly documented.

We practice test-driven development here at ThinkUp, but frankly,
documentation has been an afterthought. Jacob wants to more OSS
communities install a culture of document-driven development. He
thinks too many open source projects set up a wiki and just expect
documentation to magically appear there from newcomers and users when
the truth is, developers should be writing the documentation for the
features they add as they add them--first. He said that at Django, a
commit is only accepted into the project if the documentation is
included with it.

The talk rang several bells with me, and clarified so many things that
bother me about ThinkUp's documentation (or lack thereof, and the
subsequent effect on users and tech support burdens). I looked at
WordPress and said, "hey, they're using a wiki for documentation, so
why can't we." But our current documentation system has many
disadvantages, like:

* We're bound by the limitations of GitHub's wiki. We can't link to a
section of the page, there's no logical hierarchy or categories, we're
bound to GitHub's look and feel, and by hosting it there we're forcing
users to visit GitHub to find out more about ThinkUp. (A big no-no.)
* If GitHub's wiki's down, the documentation isn't available. If the
project changes names, or is moved to another username, the links
break.
* Reorganizing the wiki can create all sorts of dead ends. I love the
recent reorg, but right now whenever I browse the wiki, I hit a broken
link because a page was moved.
* There's no easy way to version our documentation and attach sets of
instructions to specific ThinkUp releases.

So, following Django's example, I've begun creating standalone ThinkUp
documentation using a tool called Sphinx (http://sphinx.pocoo.org/ --
not to be confused with the search tool). Here's how Sphinx works. We
write documentation files in plain text using a format called
reStructured text. We store those source files in the ThinkUp main
repository (in the docs folder). Sphinx is a tool which takes in those
source files and can generate HTML, PDF, ePub, and plain text versions
of it. A site called http://readthedocs.org can host Sphinx-based
documentation, and using GitHub post-commit hooks, can re-generate
that documentation when new versions are committed.

Here's the output of my first, incomplete take on this new set of docs:
http://readthedocs.org/docs/thinkup/en/latest/

Here are the source files:
https://github.com/ginatrapani/ThinkUp/commit/97a01846fb94d486e6e5d648cbfeb7e434a40f0c

You'll notice that Sphinx supports hierarchies (folders), custom
themes, release-specific docs, multiple output formats, and using it
we can distribute the HTML version of a specific release's
documentation with the ThinkUp package itself, so the docs will be
available on every individual ThinkUp installation. I'm pretty excited
about all the advantages of this tool.

Of course, the big advantage of GitHub's wiki is that it's very easy
to update and anyone can do it without having to submit a pull
request. With Sphinx-based docs, a contributor would have to submit a
pull request for approval just like they do with code. This stinks,
but the flipside is that using this system, documentation will be a
required part of every new patch, so the need for non-developers to
contribute docs will be reduced.

That all said, I don't believe we have to choose between Sphinx and
the wiki. I think the wiki is very well-suited for pages like
ThinkUp's inspirations, and maybe collaboratively drafting
documentation before it gets submitted for acceptance into the
repository.

Thoughts? Feedback? Let me know what you think.

Gina

--
http://ginatrapani.org
http://twitter.com/ginatrapani

[Anil Dash]

First of all, thanks for bringing this up and articulating a vision forward: We should have great docs, and it will be critical to ThinkUp's continued success.

On Fri, Apr 15, 2011 at 1:39 PM, Gina Trapani <gi...@ginatrapani.org> wrote:

Of course, the big advantage of GitHub's wiki is that it's very easy
to update and anyone can do it without having to submit a pull
request. With Sphinx-based docs, a contributor would have to submit a
pull request for approval just like they do with code. This stinks,
but the flipside is that using this system, documentation will be a
required part of every new patch, so the need for non-developers to
contribute docs will be reduced.

This is my only major concern. I think there's a huge audience of people who know how to write well, but know *nothing* of version control or even, for that matter, of wikis. I'd like to see us moving the bar closer to "if you can type, you can help fix the docs" rather than going the other way.

However, that might not be incompatible with using Sphinx. I just am not very well-versed in this area -- does anybody know if there's a way to bridge ease of creation/editing with the ecosystem benefits of Sphinx?

Anil

________________________________________
about: http://www.anildash.com/
im/twitter: anildash phone: +1 646 833-8659

[Philip Durbin]

On 4/15/11 1:39 PM, Gina Trapani wrote:
> Here's the output of my first, incomplete take on this new set of docs:
> http://readthedocs.org/docs/thinkup/en/latest/

This output looks fantastic, especially the hierarchy.

Does Sphinx have a way to create a single HTML page of all the docs?
We'd still want the individual pages too, of course, but sometimes I
like getting a "dump" of everything, for easy searching. Compare these,
for example:

http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Release_Notes/index.html

http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html-single/Release_Notes/index.html

(As a side note, I've always been impressed with Red Hat's
documentation. I think they use DocBook. Not to take anything away
from Sphinx!)

Hmm, I think I've answered my own question. It is possible, it seems:

http://sphinx.pocoo.org/invocation.html?highlight=singlehtml

> That all said, I don't believe we have to choose between Sphinx and
> the wiki. I think the wiki is very well-suited for pages like
> ThinkUp's inspirations, and maybe collaboratively drafting
> documentation before it gets submitted for acceptance into the
> repository.

I do wonder about documentation for the ThinkUp the code base versus
documentation for ThinkUp the community. I added a page to the wiki
recently that has nothing to do with the code base (it was about how to
participate in the podcast), and for community-oriented documentation
like this maybe it does make sense to keep the bar very low for the
community to contribute. So maybe a wiki for the community and Sphinx
for the code base? I don't know. . . there are a lot of advantages to
having all the documentation in one place and in the same system. . .

Finally, I wanted to point out that GitHub's wiki is open source (
https://github.com/github/gollum ), so effort *could* be put into
improving it, but it probably makes more sense for ThinkUp to grab
something off the shelf that works today, like Sphinx.

Phil

[Su]

On Fri, Apr 15, 2011 at 12:39 PM, Gina Trapani <gi...@ginatrapani.org> wrote:
> not to be confused with the search tool). Here's how Sphinx works. We
> write documentation files in plain text using a format called
> reStructured text. We store those source files in the ThinkUp main
> repository (in the docs folder).

Just throwing a quick note here about an announcement from Github today:
https://github.com/blog/844-forking-with-the-edit-button

If needing to learn how to get around in Git along with all the other
stuff has kept you from contributing to the docs in the past, this
might be of interest.