Documentation

[Daniel Yoder]

Team -

Just a quick note to let everyone know what I am shooting for
regarding documentation for Waves.

To begin with, I've never been a big fan of RDoc. Well, that isn't
quite it. RDoc itself is pretty awesome. It's more that I've never
been a big fan of the way most people use it. Namely, I really don't
like seeing huge blocks of comments in the code, ala Rails. I think
the RDoc is useful to quickly browse an API, and RDoc comments are
fine for brief clarifications. But it really clutters things up to try
and exhaustively document the API that way.

To this end, Matthew and I had been cooking up an alternative RDoc
that would allow us to integrate external documentation sources
(including a wiki) into an RDoc-like interface. However, I don't think
either of us have had the time to do this idea justice. (Matthew, feel
free to chime in here if you've made progress on this I am unaware
of ...)

As a result, I still plan to use RDoc for the foreseeable future. I
still expect to keep the RDoc-based documentation relatively sparse
(at least, for some people's tastes), but I want to compensate for
this by having really good docs outside of the RDoc. I envision 3
types of documentation.

1. Tutorials. Obviously. For most users, this will be the easiest way
to get started. Lots of examples, lots of code samples. I love what
Matthew started doing by having the tutorial code in GitHub (which
allows you to link to specific commits so that user's can actually run
the code for the tutorial at any given step). I'd like to do this with
every tutorial we do.

In fact, I'd like to take it one step further by having the steps in
the tutorial as branches so you can just checkout "step-4" or
whatever. This might seem like more work, but I think it is actually
less in the sense that it can make it much easier to modify and test
your tutorial code.

I'd also like to have screencasts of at least the major tutorials.
This is much easier to do these days because of iMovie's improved
editing capabilities. (I can't tell you how many takes I had to do on
the first Waves screencast ...)

2. Features. I think one of the troubles with the tutorials is that
they are so focused on helping people get started that it is not at
all obvious why Waves is better than other frameworks out there. I
think having some feature-focused docs would really help here. I am
thinking of things like "Better Security With Waves" or "Template-Free
Views Using Hoshi" and so forth. I think these could actually be
branches of the tutorials, so that we can direct noobs to the
tutorials if they aren't familiar with the assumptions made by the
article about using Waves (and, again, you can link to actual runnable
code, merge in changes from tutorials, if necessary, etc.)

3. Design. This would be the primary supplement to the RDoc, diving
into more specifics about each module and showing how they relate.
This is also our opportunity to explain atypical design decisions
we've made, which I think cause some people to freak out a bit when
using Waves. (I am thinking here mostly about Autocode and Functor.) I
think if people understood some of these decisions a bit better (or
how they benefit them as developers), we'd see more excitement around
what is arguably Waves greatest strength.

In the past, we've dabbled in all three areas, but it's been a bit
tough because Waves has kept evolving, so the docs are always a bit
dated, almost as soon as we finish them. While I would love to keep
adding features to Waves, at some point, we have to build a community
to really move forward. That requires really good documentation, so I
propose we actually freeze changes to the core of the framework and
focus on docs for awhile.

(Which is not to say we can't *add* things ... especially Foundations
or helpers or whatever. Just not things that require dramatic changes
to the existing docs.)

My current hope is that we can get out the next release in a week or
two and then I am going to focus entirely on documentation. Feedback
encouraged!

Regards,
Dan

[Eero Saynatkari]

Excerpts from Daniel Yoder's message of Sat Sep 05 07:50:55 +0300 2009:

> As a result, I still plan to use RDoc for the foreseeable future.

YARD addresses some, but not all, problems. I am of the
opinion that inline documentation is better than separate,
but it is a moot point as there are no complete solutions
for the latter anyway.

> 1. Tutorials.

Having the steps in a repository is fine. Additionally,
they *must* have tests, and those tests *must* be run
as a part of CI by developers. Some testwriting should
be included as a part of the tutorial, but the entire
test suite only needs to exist for the HEAD.

The repos could be submoduled to give easy access to them.
Same applies to the samples provided, unless they are just
removed in favour of the extended tutorials (with results
being, effectively, the samples.)

> 2. Features. I think one of the troubles with the tutorials is that
> they are so focused on helping people get started that it is not at
> all obvious why Waves is better than other frameworks out there. I
> think having some feature-focused docs would really help here. I am
> thinking of things like "Better Security With Waves" or "Template-Free
> Views Using Hoshi" and so forth. I think these could actually be
> branches of the tutorials, so that we can direct noobs to the
> tutorials if they aren't familiar with the assumptions made by the
> article about using Waves (and, again, you can link to actual runnable
> code, merge in changes from tutorials, if necessary, etc.)

The caveat of flexibility is that, I think, the strata
must be kept separate enough to avoid confusion of what
can and should be done in a given style of application.

> 3. Design. This would be the primary supplement to the RDoc, diving
> into more specifics about each module and showing how they relate.
> This is also our opportunity to explain atypical design decisions
> we've made, which I think cause some people to freak out a bit when
> using Waves. (I am thinking here mostly about Autocode and Functor.) I
> think if people understood some of these decisions a bit better (or
> how they benefit them as developers), we'd see more excitement around
> what is arguably Waves greatest strength.

Some people, like me, freak out over "black boxes" and
spend usually unnecessary effort to figure out what is
being done. The solutions to this seem to be either to
indeed document those better (Functor, in particular),
OR to completely de-emphasize their existence. A user
will not particularly need to care how the dispatch
logic is implemented. This is, perhaps, also related
to the separation of developer- and user documentation.

As post scriptum, I might have some time - and mental
resources - to work on Waves again shortly.

Eero
--
Magic is insufficiently advanced technology.

[Daniel Yoder]

YARD addresses some, but not all, problems. I am of the
opinion that inline documentation is better than separate,

Still, I'm curious. I realize I'm in the minority on this one, but I've always half-wondered if it was just because there really is no better way. I think my biggest beef with doing this is when the equivalent of an entire developer guide is spread across the source files. On the flip side, you end up seeing a lot of comments like this:

# Finds the given value

def find( val )

 ....

end

which are doubly useless in that they give the appearance of good documentation, when, in fact, they are just noise in the source.

Another issue I have is verbose summaries for the method signature, when you can just view source to look at it, and a one or two line comment could clarify it further quite concisely. 

BAD

#

# Method: find

# Arguments:

#   - value: the value to find

# Returns: the value that was found or nil

# Description: The #find method finds a given value an returns

#              the corresponding object. The value can be a 

#              string key or a hash of constraints.

#

GOOD

# value can be a string key or a hash of constraints

# returns nil if not found

More generally, I think it makes the most sense to have different "views" of a system, ranging from wiki pages to just the source code. Since I can't easily create these views with today's tools, I prefer to have documentation that don't concisely clarify a specific piece of code separate from the code itself.

I realize I'm preaching to the choir - it's not like we've been bloating the code with unnecessary comments. But, again, I am open to compromising on this for sure. What I really want most is for it to be easy for people to use Waves, and if we all think more comments in the code is the best approach, then we can do that.

Having the steps in a repository is fine. Additionally,
they *must* have tests, and those tests *must* be run
as a part of CI by developers. Some testwriting should
be included as a part of the tutorial, but the entire
test suite only needs to exist for the HEAD.

Excellent suggestion. This relates to some thoughts I have about writing tests, actually ... but I will post those separately.

The repos could be submoduled to give easy access to them.
Same applies to the samples provided, unless they are just
removed in favour of the extended tutorials (with results
being, effectively, the samples.)

This is a bit tricky, isn't it? I really like the idea of the tutorials being their own sub-repos. But then how to easily include them in the Waves distribution? My fear is that it is too easy to accidentally package samples that are out of date. But we could add some rake tasks to install them or something. Thoughts?

The caveat of flexibility is that, I think, the strata
must be kept separate enough to avoid confusion of what
can and should be done in a given style of application.

Another great point. For now, I think I'd like to focus on just two styles, compact and classic, with the bulk of the tutorials and features docs devoted to classic. One thing I've learned giving talks on Waves is that people are still really locked into MVC and have a hard time making the leap conceptually. Plus, Waves doesn't help them as much as it needs to, since the only alternative it Compact. Sinatra has demonstrated that people are ready for "I just need a quick-and-dirty Web server" and it is very easy to document, since there aren't that many features. But the REST stuff is just too ambitious for where we're at right now.

Some people, like me, freak out over "black boxes" and
spend usually unnecessary effort to figure out what is
being done. The solutions to this seem to be either to
indeed document those better (Functor, in particular),
OR to completely de-emphasize their existence. A user
will not particularly need to care how the dispatch
logic is implemented. This is, perhaps, also related
to the separation of developer- and user documentation.

Yes, I definitely wandered into some strange territory with Functor ... Waves was supposed to be white box all the way, the lack of undocumented magic being part of the whole point. And both Autocode and Functor are essentially attempts to codify "magical patterns" in a way that stands on its own merits. So my goal would be simply to provide better documentation for both and more examples of how they are used in Waves to implement specific requirements.

As post scriptum, I might have some time - and mental
resources - to work on Waves again shortly.

Obviously, that would be awesome.

And thanks for the feedback.

Regards,

Dan