ATTN contrib authors: Automatic documentation for clojure.contib
Tom Faulhaber
interested.]
There was some discussion (or maybe the wailing and gnashing of teeth)
about the fact that there's lots of cool functionally coming into
clojure.contrib but no easily accessible documentation. I had been
thinking about this already and figured maybe I'd set up the framework
for fixing it.
My idea is to use the namespace and var metadata to auto-generate a
set of documentation that has:
- an overview page that lists the namespaces in contrib and presents a
summary of each derived from the namespace metadata.
- each entry on the namespace overview page will have a link to an API
page for that namespace. That page will be basically like the API page
on clojure.org.
- probably should have an index page as well, but I haven't thought
about it much.
- an ability to link from a namespace to a custom created piece of
documentation (like the nice DatalogOverview that Jeffery wrote).
I've written the code that builds (a first cut of) this stuff from the
namespaces. It uses the following metadata:
On the namespace:
- :author to supply a string with the author(s) name(s)
- :wiki-doc to supply a wiki specific doc string (that can include
wiki markup) or, if that doesn't exist, we fall back to
- :doc the default doc string
On the vars, I use:
:wiki-doc and :doc, as above
:macro - to indicate if it's a macro
:arglists to grab the arglist
The last two are set automatically by defn and defmacro.
AUTHORS PLEASE NOTE: In order for this to be useful these fields need
to be filled in for your contributions. The system will work without
it, just not as nicely.
The next stage of my plan is to build a robot that watches the
subversion repository and updates the doc on every checkin. Therefore
the doc will typically correspond to the tip of the tree.
To avoid thinking too hard, I'm planning to use the wiki on
clojure.contrib's googlecode page because (a) there's almost nothing
on it now and (b) it's easy to use subversion to update stuff there.
Whether this is the right place long-term is another question that we
can worry about once it's running.
Does all of this seem like the right direction? Comments? Suggestions?
Tom
Rich Hickey
place for the contrib docs, contrib members should please feel free to
use it. That's one of the reasons I moved to googlecode.
Thanks,
Rich
Konrad Hinsen
> My idea is to use the namespace and var metadata to auto-generate a
> set of documentation that has:
> - an overview page that lists the namespaces in contrib and presents a
> summary of each derived from the namespace metadata.
> - each entry on the namespace overview page will have a link to an API
> page for that namespace. That page will be basically like the API page
> on clojure.org.
> - probably should have an index page as well, but I haven't thought
> about it much.
> - an ability to link from a namespace to a custom created piece of
> documentation (like the nice DatalogOverview that Jeffery wrote).
That sounds like an excellent idea! I'd be happy to add metadata to
my code to support such a system.
The only aspect of your proposal that I am not happy about is this:
> On the namespace:
> - :author to supply a string with the author(s) name(s)
> - :wiki-doc to supply a wiki specific doc string (that can include
> wiki markup) or, if that doesn't exist, we fall back to
> - :doc the default doc string
Since the standard docstring in :doc is required for Clojure's built-
in doc and find-doc, your proposal amounts to having a copy of the
docstring with wiki formatting added in :wiki-doc. I don't like the
idea of duplicating the docstring for several reasons:
- sooner or later :doc and :wiki-doc will differ because an update is
applied to only one of them
- it implies extra work for every single documented function
- it makes the code harder to read, all the more because there is no
syntax support for :wiki-doc.
I would prefer to have a single docstring containing optional markup.
There are (at least) two possibilities:
1) Use a very lightweight markup that preserves readability even
without any processing.
2) Add a markup stripper/transformer to Clojure's doc and find-doc
functions.
> The next stage of my plan is to build a robot that watches the
> subversion repository and updates the doc on every checkin. Therefore
> the doc will typically correspond to the tip of the tree.
That's a nice feature.
Konrad.
Jason Sankey
Tom Faulhaber wrote:
<snip>
> The next stage of my plan is to build a robot that watches the
> subversion repository and updates the doc on every checkin. Therefore
> the doc will typically correspond to the tip of the tree.
<snip>
I'd be happy to add generation/upload of documentation as a build step
or even separate project on the server at http://pulse.zutubi.com/. It
already has the ability to trigger on every change, so there would be no
need to create a separate robot to check for changes. Just let me know
if you think this would be helpful.
Cheers,
Jason
Sean
re-implementing stuff that exists in contrib. However, contrib isn't
the only library that needs to be documented. Besides the doc-string
metadata, is there any type of Clojure-doc, similar to Java-doc?
Is there utility to generate HTML docs that is part of Clojure contrib/
core?
Laurent PETIT
Some thoughts I had when reading your proposal:
Could this be made a little bit more generic ? (If you find it useful) :
Have a global var *rich-doc-renderers* which holds a map of type-of-rich-documentation to a map of renderer functions indexed by type of output (mime/type such as plain/html , plain/text for the :doc ...). This var could be enriched by libraries willing to contribute new standards : they would provide in the same time a new type of doc, and also ways to transform these docs into output formats:
Example of *rich-doc-renderers* "out-of-the-box" could be :
*rich-doc-renderers* { :default { :plain/html some.namespace/some-fn :plain/text some.namespace/some-other-fn ... } } ; default is what would be used for clojure / clojure-contrib ;
And we could have a :rich-doc instead of :wiki-doc , with a simple version (if the default type of rich doc is to be used) and an extended version:
simple version :rich-doc "my rich doc" => the type of formatting content to be used is :default , one can use the provided renderers for :default
extended version :rich-doc { :type :default :content "my rich doc" } => this version explicits the type of doc in :content. Of course for the general case it's not needed, but if a lib implementor wants to enrich *rich-doc-renderers* with a new set of renderers, it could be interesting ?
Do you thing it's interesting, or overkill ?
Laurent PETIT
Of course, some rules could be used to smoothly downgrade the documentation if the renderers throw exceptions, or are impossible to load ... :
the doc function could use this simple algorithm :
find the :doc tag.
If found, use it
If not found, try to use :rich-doc with a :plain/text renderer
the function for getting :rich-doc in a particular format could use this simple algorithm :
find the :rich-doc tag
if not found: find the :doc tag. if found, use it. if not found, return nil or "".
if found :
if it's a string, find and load the renderer for the :default type and the expected output format (if exception in the chain, try to return the :doc tag content)
if it's a map, get the value for :type, try to resolve it in *rich-doc-renderers*, and then search and load the renderer function for the expected output format (again, if exception in the chain, try to return the :doc tag content)
if neither a string or map, ? (throw an exception, or defaults to the doc with a warning message on *err* ?)
Well,
something like that ...
Tom Faulhaber
I too would rather have everything in a single :doc tag and it most
cases it will be (just as it is with the clojure.org API page). But I
wanted to leave open the option for people to create custom marked-up
version for the wiki if they wanted something fancier. Any solution
that involves including the markup in the :doc string seemed to me to
be not immediately practical either cause it mucks up the output of
(doc ...) or requires changes to core to do something with markup. I
didn't want to break what was there or have to wait for a new feature.
But we can change pretty easily later, I think, once it seems like we
know the right answer.
Tom
Tom Faulhaber
Thanks for the offer. I may take you up on it as I get this into
"production" such as it is.
Sean,
Nothing that I found in contrib (but who knows, there's no
documentation :-)). Rich posted some code (which I'm using in a
modified version in the center of things) for doing the API paage for
clojure.org here: http://paste.lisp.org/display/77339
When I get my code looking anything other than nasty (and anyone who's
looked at pprint knows that's a low bar!), I'll at least put it up on
github. Eventually I may add it to contrib, too, if the community
feels it should go (but let's solve the problem ahead of us).
Laurent,
You seem to be going in the opposite direction from Konrad and I would
worry about the duplication. I would be more inclined to think about a
single markup format for :doc strings that could be converted for the
various uses. But that's really a language change at some level, so
I'd rather defer that until we see how this stuff is used.
But to be honest, I haven't read your comments that closely because
today has been a long day. Maybe later tonight...
Thanks for all the feedback, guys!
Tom
Sean Devlin
for the comment by Brain Carper
http://groups.google.com/group/compojure/browse_thread/thread/67d92ceb4128a375?hl=en
Brian's code:
http://briancarper.net/clojure/compojure-doc.clj
This might be an alternate approach to the problem.
Tom Faulhaber
Thanks for the pointer here. There's a lot of commonality, but mostly
I think the products end up being different for two reasons:
1) For the moment, the contrib-autodoc stuff is targeting the
googlecode wiki, which is a weird and gross thing of its own. I've
been thinking of making some kind of builder syntax for it, but for
the moment I'm just using formats.
2) Most of the work is in dealing with subversion and the like to
check, get, build, etc.
But looking at what the compojure guys are doing is interesting and
educational. I think we have a ways to go in thinking about the amount
of stuff we can do to leverage doc-oriented metadata in Clojure in a
more flexible way.
Tom
On Apr 15, 7:07 am, Sean Devlin <francoisdev...@gmail.com> wrote:
> Check out the thread below in the Compojure group. Specifically, look
> for the comment by Brain Carper
>