A conceptual model of documentation and some ideas

[Peter Minten]

Hi all,

Maybe it's a good idea at this point in the development of ex_doc to
take a step back and think about the nature of documentation what we
want to support.

My current line of thought is that documentation can be divided into
three categories:

1. Reference: That's just the stuff we have already. Very structured.
2. Guide: A set of chapters intended to be read in order like a manual.
Somewhat structured (chapter, section, etc).
3. Miscellaneous: All kinds of files with all kinds of information that
the user may want to know. For example: README, LICENCE, CONTRIBUTING.

Guide and Misc documentation are similar in that they're both likely to
be markdown files. Each project only has one guide however and probably
multiple Misc documentation files.

Reference documentation has a lot of structure so there's a lot to talk
about. ;) Let me just write down a few associations (note that functions
= functions + macros + callbacks):

* A module contains zero or more types.
* A module contains zero or more functions.
* A record contains two types (t() and option()) which are of not much
use to the user (hard to parse mentally).
* A record contains zero (one?) or more fields.
* A record doesn't contain any functions (though some do through hacks).
* A protocol contains one type (which is apparently all the supported
types), in any case it's not of much use to the user.
* A protocol contains one or more functions.

If we disregard types you can classify our entities into two categories:
modules (including record modules and protocol modules) and
fields/functions.

I've recently looked at yard (http://www.yardoc.org, example:
http://rubydoc.info/gems/dm-core/1.1.0/frames). Yard is the inspiration
for ex_doc so it's not odd that yard and ex_doc look pretty much the
same. But there's a significant difference in the sidebar. yard has
"Classes | Fields | Files", ex_doc has "Modules | Records | Protocols".
Yard's fields list is simply a list of all methods and members from all
classes. So in yard the first two items are different views on the same
information whereas in ex_doc they are different pieces of information.

Lets think for a moment about the conceptual model of Elixir that exists
in a user's head. For a user the record ExDoc.ModuleNode lives in ExDoc.
The fact that it lives in ExDoc carries some meaning. The namespace
ExDoc means something independent of what functions are in the module ExDoc.

To the user what ex_doc calls modules, records and protocols all live in
one big tree, regardless of their representation. The current conceptual
model of ex_doc is that they are very different things. I believe this
mismatch causes decreased usability.

Let's take a look at different ways in which we could potentially
present information:

* Modules, records, protocols as a tree.
* Modules, records, protocols as a flattened list.
* Both but with only some of the types shown (filtering on module type).
* Both but with only things that match a certain name pattern.
* Both filtering on module type and name.

* Functions, macros and callbacks as a list (with their module in small
letters below them, like when you search for something).
* The list, but with only some of the types shown (filtering on member
type).
* The list but with only things that match a certain name pattern.
* Both filtering on member type and name.

The way I think of it we could incorporate all of these in ex_doc. We
already require Javascript, it should be possible to have both the tree
view and the flat view (have the tree data available to JS, flattening a
tree is easy).

In my mind the ideal ex_doc will have four main categories: Modules |
Functions/Fields (need a good name for that) | Guide | Files (Misc docs).

Within modules we'd have some toggle button like things: Modules (only)
| Records (only) | Protocols (only). These would choose what to filter.
The "(only)" would be a different button that would turn on the
corresponding button and turn off the others.

There would be an elegantly placed button somewhere to switch between
tree and flat form.

Types would not be included in this system (as they are not now), they
would be in the documentation and possibly some search system could find
them, but it's pretty rare that you search for types (especially
considering the autolinking that's in place).

I believe this would make ex_doc easier to use while not fundamentally
altering it.

Comments are very welcome. :)

Greetings,

Peter

[Oren Ben-Kiki]

I see how ex_doc does 1 (Reference) but I don't see how it can do a good job doing 2 (Guide) and 2 (Misc) unless it is extended.

One could say that these need to be done completely separately, but it would be nice to be able to integrate things - at minimum using cross references between them.

I still believe the simplest thing would be to allow ex_doc to generate additional views of the data, specifically a flat list in an order under programmer control to satisfy 2. That seems to be just me though...

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[José Valim]

 

If we disregard types you can classify our entities into two categories: modules (including record modules and protocol modules) and fields/functions.

The reason I moved away from the fields/functions classification is because I have never used them in YARD. If you are looking for a function, looking at a couple of modules or using a search will likely be more effective than a full, complete list.

The original idea in splitting in Modules and Records and Protocols is because they are the three different ways you integrate with a library code. Modules are behaviour dependency, Records are data dependency and Protocols are contracts you may want to implement. Not saying it is the best one, just how I first thought about it.

 

Within modules we'd have some toggle button like things: Modules (only) | Records (only) | Protocols (only). These would choose what to filter. The "(only)" would be a different button that would turn on the corresponding button and turn off the others.

There would be an elegantly placed button somewhere to switch between tree and flat form.

Having toggles are a nice idea, so we don't impose any particular view on the user. I am wondering though if we really want to go back to the tree view. It is definitely not worthy for small projects and we are splitting Elixir into 5 different ones (elixir, iex, mix, eex, and ex_unit).

[Peter Minten]

José Valim <jose....@plataformatec.com.br> wrote:
> > If we disregard types you can classify our entities into two categories:
> > modules (including record modules and protocol modules) and
> > fields/functions.
> >
>
> The reason I moved away from the fields/functions classification is because
> I have never used them in YARD. If you are looking for a function, looking
> at a couple of modules or using a search will likely be more effective than
> a full, complete list.

Regarding search: one idea to make it more powerful is to have a separate
search page (i.e. not in the side bar) that returns a table of item name/link
and a short description of it AND that can search within those descriptions.
We already have the data available and I suspect it would make search much
more effective. Question is of course if you're not going to send too much data
to the browser (all modules, functions, etc + descriptions) / if JS can search
through the data fast enough.

> The original idea in splitting in Modules and Records and Protocols is
> because they are the three different ways you integrate with a library
> code. Modules are behaviour dependency, Records are data dependency and
> Protocols are contracts you may want to implement. Not saying it is the
> best one, just how I first thought about it.
>
>
> > Within modules we'd have some toggle button like things: Modules (only) |
> > Records (only) | Protocols (only). These would choose what to filter. The
> > "(only)" would be a different button that would turn on the corresponding
> > button and turn off the others.
> >
> > There would be an elegantly placed button somewhere to switch between tree
> > and flat form.
> >
>
> Having toggles are a nice idea, so we don't impose any particular view on
> the user. I am wondering though if we really want to go back to the tree
> view. It is definitely not worthy for small projects and we are splitting
> Elixir into 5 different ones (elixir, iex, mix, eex, and ex_unit).

Well I was thinking of the whole lot of Error records that would pop up in the
list if you turn errors on. But against those it wouldn't help much anyway
(they're all in the root namespace).

I guess you're right, it wouldn't help much. Two levels is quite good enough.

[Chris Keele]

I have to admin, I never use the full function lists in rdoc, except in one scenario: when the function API is small and the library is poorly documented with few examples at the root level. I know it has a capability, but I have no idea where it sits in the library or what it's called.

That situation is easier remedied with better top-level documentation and tutorials/guides than a full function listing of a potentially huge library. The idea here I like the most is support for auxiliary pages, and top-level documents.

I think Rdoc uses READMEs to build indexes, right? Currently the root level for Elixir's docs defaults to Kernel. What about a fourth view, something like Structure? Exdoc could accept a list of directory names, with some flags (recursion defaults to false, display modules alongside the markdown defaults to true). It could then go into each directory and build all markdown files, with READMEs as indices. This would be the only tree-view we would really need, especially if modules were displayed inline.

Modules would get new URLs corresponding to the directory they're in, but that would be transparent to the current Modules view. This is flexible enough to add many of the discussed documentation features without modifying any of the current views :

1. If Structure exists, it becomes the first view in the documentation. The project README becomes the main index. If the ex_doc configuration for Structure is left blank, the documentation won't build that tab, so it's opt-in only and doesn't introduce new behaviour to existing projects.
2. Directories with specific intents get their own READMEs. If you have a Matrix protocol similar that provides an API for many different implementations, you probably just discuss the API in its docs. You also probably have a matrices folder with your different implementations, each of which documents the features and limitations of the implementation. A matrices/README.md could provide a better place to compare and contrast all implementations and situations to use each one. In the tree view, you'd see the links to the corresponding implementations as you were reading about them.
3. Miscellaneous pages like CONTRIBUTING.md get their own place, at any level in the project. I can see a test/CONTRIBUTING.md file going into more depth about how to contribute tests, another in lib for code, etc. Another perk: your LICENSE can be a part of your documentation, making the awesome :) or limited :( open-sourceiness of your project apparent.
4. Suites of documentations can live in first-class folders off of root. For example, a guides folder the way Rails does it, but it gets baked in to the documentation.
5. Searching for matri in the tree view could bring up the protocol, all implementations, and the documentation for the suite of implementations, if it's keyed to work on directory names, which seems handy to me, but may be white noise to others or complicate the search features. Not sure.

I know I for one would use the crap out of such a feature, because I love well-structured file systems and robust documentation. But it's opt-in, keeps the existing useful Modules|Records|Protocols divisions, and still offers a lot to even fairly flat or small projects.

[Peter Minten]

I don't think we need more than one guide per project. It would be a
really insanely big project (Rails size) that needs multiple guides and
we won't have them for a very long time I suspect. And if we do get such
a big project it shouldn't be too hard to add at that point.

With regard to the rest of your idea, what you describe as Structure I
call Files, but the name doesn't really matter. It's a bit of a debate
whether you want to structure that as a tree or as a flat list. Example
of tree:

* README
* matrices
* README

Example of list:

* README
* matrices/README

Personally I see more benefits to a flat tree, but it's worth discussing.

What you describe as how search works (find everything in all
categories) is not how it currently works, currently it just finds stuff
in the selected category, and I suspect this is what you want in the
majority of cases. A separate search page could be added for this though.

What should be the first page the user sees is something that should
probably be settable when documentation is created. Currently we have
the --main switch to set which module is the first thing the user sees
(this is why the main elixir docs always open on Kernel), that could be
extended so that you can set it to a README or the top level Guide page
or something.

On 09/01/2013 10:54 PM, Chris Keele wrote:
> I have to admin, I never use the full function lists in rdoc, except in one
> scenario: when the function API is small and the library is poorly

> documented with few examples at the root level. I *know* it has a

> capability, but I have no idea where it sits in the library or what it's
> called.
>
> That situation is easier remedied with better top-level documentation and
> tutorials/guides than a full function listing of a potentially huge
> library. The idea here I like the most is support for auxiliary pages, and
> top-level documents.
>
> I think Rdoc uses READMEs to build indexes, right? Currently the root level
> for Elixir's docs defaults to Kernel. What about a fourth view, something

> like *Structure*? Exdoc could accept a list of directory names, with some

> flags (recursion defaults to false, display modules alongside the markdown
> defaults to true). It could then go into each directory and build all
> markdown files, with READMEs as indices. This would be the only tree-view
> we would really need, especially if modules were displayed inline.
>
> Modules would get new URLs corresponding to the directory they're in, but

> that would be transparent to the current *Modules* view. This is flexible

> enough to add many of the discussed documentation features without
> modifying any of the current views :
>
>

> 1. If *Structure* exists, it becomes the first view in the

> documentation. The project README becomes the main index. If the ex_doc

> configuration for *Structure* is left blank, the documentation won't

> build that tab, so it's opt-in only and doesn't introduce new behaviour to
> existing projects.

> 2. Directories with specific intents get their own READMEs. If you have
> a *Matrix* protocol similar that provides an API for many different

> implementations, you probably just discuss the API in its docs. You also

> probably have a *matrices* folder with your different implementations,

> each of which documents the features and limitations of the implementation.

> A *matrices/README.md* could provide a better place to compare and

> contrast all implementations and situations to use each one. In the tree
> view, you'd see the links to the corresponding implementations as you were
> reading about them.

> 3. Miscellaneous pages like *CONTRIBUTING.md* get their own place, at
> any level in the project. I can see a *test/CONTRIBUTING.md* file going
> into more depth about how to contribute tests, another in *lib* for

> code, etc. Another perk: your LICENSE can be a part of your documentation,
> making the awesome :) or limited :( open-sourceiness of your project
> apparent.

> 4. Suites of documentations can live in first-class folders off of root.
> For example, a *guides* folder the way Rails does it, but it gets baked
> in to the documentation.
> 5. Searching for *matri* in the tree view could bring up the protocol,

> all implementations, and the documentation for the suite of
> implementations, if it's keyed to work on directory names, which seems
> handy to me, but may be white noise to others or complicate the search
> features. Not sure.
>
> I know I for one would use the crap out of such a feature, because I love well-structured

> file systems <http://xkcd.com/981/> and robust documentation. But it's
> opt-in, keeps the existing useful *Modules|Records|Protocols* divisions, and