Autogenerating Tiddlywiki API documentation...

[Jim Lehmer]

This topic was discussed in the hangout the other day. I've done a bit of research and it looks like YUIDoc may fit the bill. It looks like it is under active maintenance (multiple directories changed in Github in the past few weeks), even if YUI as a whole is no longer under active development.

Positive points:

1. npm module, could be worked into the build mechanism or used standalone.
2. Supports a set of JSDoc tags.
3. BSD license.
4. Supports Markdown in comment blocks for doc formatting.
5. Produces decent-looking output.

My suggestion would be to start with all the exported methods and properties, including all macros, in boot and core, and then perhaps circle back later for more.

Comments?

[Tobias Beer]

I am thinking this is actually an area where we definitely should "eat our own dogfood".

In other words: find a way to parse and transform modules / code / comments and output everything into a TiddlyWiki with proper fields / tags / presentation / plugins.

Imagine this as a node package other package distributors can use to generate the same type documentation by the click of a button, provided the same commenting rules are parseable.

This makes me think: Should code-comments actually be in the format of .tid files? This would make parsing enormously straight forward.

All we needed was some conventions on how to populate which (standard) fields, tags, text-templates with content. Perhaps some helper macros to cross-reference.

Doc-tiddlers would be without any $:/ prefix, simply showing documentation and the actual code being documented right below (from the shadow tiddler).

If we wanted to, we could develop a custom theme to show documentation and code side-by-side. After all, we know what line of a file some documentation block refers to.

As for non-js files and folders, there could be complementary .tid files as the documentation counterpart.

Best wishes, Tobias.

[Jim Lehmer]

Interesting...

On Thursday, January 29, 2015 at 9:50:14 AM UTC-6, Tobias Beer wrote:

Imagine this as a node package other package distributors can use to generate the same type documentation by the click of a button, provided the same commenting rules are parseable.

This makes me think: Should code-comments actually be in the format of .tid files? This would make parsing enormously straight forward.

All we needed was some conventions on how to populate which (standard) fields, tags, text-templates with content. Perhaps some helper macros to cross-reference.

If we used some sort of "code fencing" in the comments, we could easily discriminate between comments that had documentation in them and ones that didn't (i.e., we don't want to pick up every block comment, because some may be, well, comments! :).

Something like YAML block markers, but not as ugly. :) YUI simply uses an extra asterisk at the start of the block:

/**
<docs go here>
*/

...but that seems a bit ambiguous. I like Jeremy's "arms" block markers on modules:

/*\
<docs go here>
\*/

I agree, if we can come up with some field names we all agree on, they could then simply be tiddler-formatted. I worry a bit about collisions with field meanings already in use (around modules, for example). But maybe we simply use a set of prefixed field names, e.g., docs, dox, or twd, or something - or even take a page from YUI and similar Javadoc-based formats and simply use @ as a field name prefix. For example, given a Javascript function foo:

/*\
@method: foo
@params: bar, baz
@return: string
@format: text/x-markdown

This function gonculates the output using the [xyzzy](http://somewhere.com/xyzzy) algorithm on the `bar` and `baz` parameters.
\*/
function foo(bar, baz) {
...

If we don't try and get fancy to start by not auto-detecting function names and parameters and having them documented explicitly as fields, I agree it would seem like it could be a fairly straightforward approach. I think we need to make sure we support Markdown as well as tiddlytext to start because if we want others (outside the TW community) to use it to produce documentation TWs per your suggestion, I think MD would need to be supported simply because it is more ubiquitous for that purpose, thanks to Github, etc.

An interesting question to me would be how to document multiple parameters, each of which could get quite complex? YUI uses repeating fields:

/**
* My method description.  Like other pieces of your comment blocks, 
* this can span multiple lines.
*
* @method methodName
* @param {String} foo Argument 1
* @param {Object} config A config object
* @param {String} config.name The name on the config object
* @param {Function} config.callback A callback function on the config object
* @param {Boolean} [extra=false] Do extra, optional work
* @return {Boolean} Returns true on success
*/

I don't think that would work with TW fields, would it? And while a field that is a list (see my @params above) would work for parameters, it would be unwieldy for complex parameter documentation showing options, TW/MD markup, etc. Maybe we just come up with a convention that everything from the start of a @params field until the next field or blank line are parameters, and have some conventions around those:

/*\
@method: foo
@params:
* bar (string) - The starting string for `xyzzy`. An error is thrown if it is missing.
* baz (string, optional) - The ending string for `xyzzy`. If missing, `plugh` is assumed.
@return: string
@format: text/x-markdown

This function gonculates the output using the [xyzzy](http://somewhere.com/xyzzy) algorithm on the `bar` and `baz` parameters.
\*/
function foo(bar, baz) {
...

[Tobias Beer]

Hi Jim,

I think this is a very important topic ...it will get people up to speed with the TiddlyWiki core by magnitudes.

If we used some sort of "code fencing" in the comments, we could easily discriminate between comments that had documentation in them and ones that didn't (i.e., we don't want to pick up every block comment, because some may be, well, comments! :).

/*doc

tags: util

documenting the below function
*/

Actually, "doc" could be a flag defined as a global setting for the function that extracts the documentation. Theoretically there could even be different flags being used for different purposes using the same mechanism.

Here's my take on a "doc-tid" format...

/*doc

title: <should be automatic via rules, perhaps with an option to declare and thus overrule>
tags: <depending on well defined tagging rules>
namespace: $tw.utils <should be set on a file basis instead>
function: getAttributeValueFromParseTreeNode <should be automatic!>
arguments: node, name, defaultValue <should be automatic!>
node: (type of node) description of node
name: (type of name) description of name
value: (type of defaultValue) description of defaultValue
return: (type of return value) description of return value
summary: a short summary of what this function does

additional information, if there needs to be any, e.g. cross-references with other functions

! Example
a common example of how to use this function, if useful

*/

In addition, a type field could be specified so as to override the global default (rather than a "format" field).

The title may be derived from a namespace set for an entire js file whereas all functions in that file are then appended to the namespace via dot notation. No sure we even need that function field. Ideally, it will be identified by the parser. Same for the arguments field!

But maybe we simply use a set of prefixed field names, e.g., docs, dox, or twd, or something

Why? We know what fields we use in the docs, no prefix needed. We can't use @ as that is invalid in html attributes... an I don't see why we should.

I think we need to make sure we support Markdown as well as tiddlytext to start because if we want others (outside the TW community) to use it to produce documentation TWs per your suggestion, I think MD would need to be supported simply because it is more ubiquitous for that purpose, thanks to Github, etc.

That is possibly true, but then I would have a global setting for the repo to define what format to use, which you could overwrite for a single tiddler by setting the type field accordingly.

Best wishes, Tobias.

[Jim Lehmer]

On 2/1/2015 3:24 PM, Tobias Beer wrote:

Why? We know what fields we use in the docs, no prefix needed. We can't use @ as that is invalid in html attributes... an I don't see why we should.

I see that as simply a marker that would be stripped by the parser. I mean, we're not going to leave the trailing colons in the names either, true? :) To me, it is a way for a parser to quickly rip through a file looking for just the strings it cares about, and something like /@([a-z]+):/i (I haven't tested that regex :) ), where we're looking for strings starting with alpha between an @ and a : and capturing just the string itself and not the delimiters seems like a viable approach (and note the case insensitive flag, so people wouldn't have to worry about @title: vs. @TITLE: vs. @Title:).

My concern here was around the fields in comments already in use (and I believe relied upon by TW) in modules. I can see where modules would need some "documentation" fields as well as normal "TiddlyWiki" fields, and was trying to namespace the documentation fields basically, so that we could have a something like:

title: $:/boot/boot.js  // This is already in use by TW
@title: boot.js         // or "Boot module" or "Das Boot" (joke)
                        // or whatever

That is possibly true, but then I would have a global setting for the repo to define what format to use, which you could overwrite for a single tiddler by setting the type field accordingly.

Probably a good approach, although I was thinking default would be TW syntax and we make the Markdown people work harder. :) Just kidding!

[Tobias Beer]

Hi Jim,

 

I see that as simply a marker that would be stripped by the parser. 

I mean, we're not going to leave the trailing colons in the names either, true? :)

What trailing colons do you mean?

To me, it is a way for a parser to quickly rip through a file looking for just the strings it cares about, and something like /@([a-z]+):/i (I haven't tested that regex :) ),

To me, with a doc-tid, we're looking at a plain tiddler definition, no special parsing, except for the function head that follows it, to get both the title and the arguments and that's it. We already get those arguments in the right order; all we need is to remove those commas. No need for any regexp voodoo in the actual doc-tid. As for the function title, we may need to handle / replace some namespace prefix for a file, e.g. module.exports is hardly what we want, so perhaps a file could start with...

//doc _namespace=$tw.utils

...and then keep adding the bit behind the last dot in that function title to construct the function title. If ever necessary, the _namespace field could be manually set for each individual doc-tid.

I can see where modules would need some "documentation" fields as well as normal "TiddlyWiki" fields, and was trying to namespace the documentation fields basically, so that we could have a something like:

True, with my first proposal above, if we had an argument called title or type we had a bit of a problem: We sure can't set a title or type field without breaking TiddlyWiki's representation. The simplest workaround would indeed be to prefix all fields. How about underscore, e.g. _type?

So, to put it all together once more...

/*doc

_summary: a short summary of what this function does
_namespace: $tw.utils <should be set on a file basis instead of per doc-tid>
_function: <should be automatically parsed from the function head!>
_arguments: node, name, defaultValue <should be automatically parsed from the function head!>
_node: (type of node) description of node
_name: (type of name) description of name
_value: (type of defaultValue) description of defaultValue
_return: (type of return value) description of return value

title: <should be automatic via rules, perhaps with an option to declare and thus overrule>
tags: <depending on well defined tagging rules>

type: text/markdown

additional information, if there needs to be any, e.g. cross-references with other functions

## Example
A common example of how to use this function, if useful.

*/

Stripping the above down to the minimum, assuming the rest was parsed from the function head or otherwise defined via defaults, or not necessary...

/*doc

_summary: this function does yada
_node: (node) the [[node]] bla
_name: (string) blip name blup
_value: (number) the bleep value
_return: (boolean) true: foo / false: bar
tags: utils
*/

Best wishes, Tobias.

[Jim Lehmer]

On 2/1/2015 6:36 PM, Tobias Beer wrote:

What trailing colons do you mean?

The ones separating the field name from the field content, e.g.:

title: Foo
     ^ - that one.

My point was simply that prefixing it with 1-n characters to create a doc field namespace doesn't have to imply it has to be valid in HTML attributes or wherever, because they can just be stripped out by the parser.

To me, with a doc-tid, we're looking at a plain tiddler definition, no special parsing,

Well, except for the fact we're in a Javascript comment. :) And that means we're going to have to deal with this:

/*doc
_summary: blah
*/

...as well as this:

/*doc
 * _summary: blah
*/

...as well as this:

/*doc
** _summary: blah
*/

...unless we prohibit all but the first format.

Honestly, my original thoughts on this was to stay somewhat close to YUI/Javadoc "standards" (conventions), because those are known to a wider audience and I was hoping to get some tool reuse. That isn't saying I think you're wrong - I just wonder how much wheel reinvention we'll have to do, especially if we begin parsing the function/property name and parameters directly instead of having them explicitly defined in the doc fields? Which, again, I don't think is wrong, but increases the complexity of the solution. "The perfect is the enemy of the good" and all that.

True, with my first proposal above, if we had an argument called title or type we had a bit of a problem: We sure can't set a title or type field without breaking TiddlyWiki's representation. The simplest workaround would indeed be to prefix all fields. How about underscore, e.g. _type?

So, to putting it all together once more...

/*doc

_summary: a short summary of what this function does
_namespace: $tw.utils <should be set on a file basis instead of per doc-tid>
_function: <should be automatically parsed from the function head!>
_arguments: node, name, defaultValue <should be automatically parsed from the function head!>

_node: (type of node) description of node
_name: (type of name) description of name
_value: (type of defaultValue) description of defaultValue
_return: (type of return value) description of return value

title: <should be automatic via rules, perhaps with an option to declare and thus overrule>
tags: <depending on well defined tagging rules>

type: text/markdown

additional information, if there needs to be any, e.g. cross-references with other functions

## Example
A common example of how to use this function, if useful.

*/

I could live with that.

Question: How do you see module/"class"-level documentation working? Right now we have this:

/*\
title: $:/boot/boot.js
type: application/javascript

The main boot kernel for TiddlyWiki. This single file creates a barebones TW environment that is just sufficient to bootstrap the modules containing the main logic of the application.

On the server this file is executed directly to boot TiddlyWiki. In the browser, this file is packed into a single HTML file.
\*/

Is that enough? Just extract that text, stick it under a "boot.js" header/tiddler and be done?

Another thought. If we're going to parse out the functions/properties, it would be an interesting exercise (I think) to have an automatically generated "references/referenced by" set of links, too. But that increases the complexity even more! :)

[Tobias Beer]

Hi Jim,

 

My point was simply that prefixing it with 1-n characters to create a doc field namespace doesn't have to imply it has to be valid in HTML attributes or wherever, because they can just be stripped out by the parser.

I think it's a lot better to use precisely what we're going to find in TiddlyWiki, no special handling or logic... except within the TiddlyWiki output template itself.

that means we're going to have to deal with this:

/*doc
 _summary: blah
 */

...as well as this:

/*doc
* _summary: blah
*/

...as well as this:

 

/*doc
** _summary: blah
*/

The way I see it, we don't have to deal but with the first. Let's start by defining our pattern and that's it. No catering for all possible styles to begin with. If later on someone desperately wanted that, we can perhaps try and cater for it, and perhaps get a small paycheck for it, but not when we start out. To me, using a simple tid format is of utmost importance because that is precisely what guarantees compatibility with TiddlyWiki.

 

...unless we prohibit all but the first format.

So, yes, that is what I would recommend. Let's not try to make ammends for the whole world but start with something that works perfectly well with our not so small project. We have enough work ahead to produce our own documentation, rather than writing parsers for someone else's. Let's not imagine all possible things anybody could want but strip it down to a basic set to start out with, one that makes sense for doing this kind of thing with TiddlyWiki. 

Adapting an existing project to TiddlyWiki style documentation will need manual intervention anyway, so... I don't see why we would start out with requirements imagining all possible documentation styles people may have come up with, e.g. YUI. Perhaps later there could be custom parser modules to convert such custom comment formats to tid format, but that's surely not how we would start, at all. Let's NOT compete with YUI trying to get our thing working for their crew. If that's a challenge you want to live up to, do it later, after we're set ourselves.

Honestly, my original thoughts on this was to stay somewhat close to YUI/Javadoc "standards" (conventions), because those are known to a wider audience and I was hoping to get some tool reuse.

That is precisely where I believe you're making a big mistake. There is no point for us in trying to adapt to any more-or-less-standard. We can perfectly define one ourselves, based on tiddlers, imho. Why would you pick YUI? You could pick any arbitrary project standard. Let's not do that.

I just wonder how much wheel reinvention we'll have to do, especially if we begin parsing the function/property name and parameters directly instead of having them explicitly defined in the doc fields?

If you ask me, trying to adapt to a standard that doesn't suit our pattern actually is reinventing a wheel, their wheel, and trying to make it fit to our vehilce whereas we already have wheels lying around that do quite nicely for the thing we want and need to get going, namely the tid file format. Those fit perfectly to the vehicle called TiddlyWiki.

 

Which, again, I don't think is wrong, but increases the complexity of the solution. "The perfect is the enemy of the good" and all that.

So, to perhaps draw that bottom line, catering a number of possible commenting styles, is what I would call increasing complexity. Let's not do that.

Question: How do you see module/"class"-level documentation working? Right now we have this:
/*\
title: $:/boot/boot.js
type: application/javascript
The main boot kernel for TiddlyWiki. This single file creates a barebones TW environment that is just sufficient to bootstrap the modules containing the main logic of the application.
On the server this file is executed directly to boot TiddlyWiki. In the browser, this file is packed into a single HTML file.
\*/

That would need some adjusting. At least there would have to be a newline after type. Never took a look at the process, but I can imagine that this stuff is parsed to actually build the core component tiddlers. So, here we're not only in "documentation-land" but rather in build-territory.

I would sure not want to see any _namespace:foo.bar or _summary:this file does bla see in the distro, but rather only in the documentation wiki. We could possibly ignore any of those that start with underscore in the build-process, including the text.

I think it's perhaps not a good idea to intermingle the two. My tentative recommendation: have one block for the build instructions and another for the file documentation.

/*doc-module
_namespace: bla.blup
_summary: this file defines the foo class

details, details details...
*/

Is that enough? Just extract that text, stick it under a "boot.js" header/tiddler and be done?

So, nope, I think there would either have to be some parsing rules for that block or some form of separator if one wanted both in one. If you think of other projects, they will not have TiddlyWiki built instructions but their own. So, documentation and built / test comments should sure be separated.

As a sidenote, I also don't think that any of that documentation content, build comments, jslint comments, etc.. should end up in the distro. But that, too, is a thing for another day... hopefully not too far down the road.

 

Another thought. If we're going to parse out the functions/properties, it would be an interesting exercise (I think) to have an automatically generated "references/referenced by" set of links, too. But that increases the complexity even more! :)

Let's take it one step at a time. ;-)

Surely, if we set the _namespace field correctly and know some function name for which we create a doc-tid we could easily have a list in a conditional view-template within TiddlyWiki that queries the tiddler store exactly for that and lists occurences at the documentation tiddler in a slider.

This could also list results where _namespace:foo matches in a file head and then references to that function via this.someFunction or perhaps self.someFunction. there could also be fuzzy matches just listing occurences of the plain function name... all within TiddlyWiki itself, nothing that the "build-process" for the documentation would do. The big chunk of the logic will be within the target / template TiddlyWiki, not some node-comment-parser(!) That parser should be as simple and light-weight as possible, imho.

Best wishes, Tobias.

[Felix Küppers]

Hi,
 

I am thinking this is actually an area where we definitely should "eat our own dogfood".

When it comes to API docs, we should stick to standards like JSDOC. Code needs to be documented in a most comprehensible and standard compliant way to ensure other developers can pick it up and understand it fast when acutually reading the code.

Also, I am against reinventing the wheel it just binds way too many developer resources. Implementing a comment parser with all features is a huge project. This is valuable time we (especially Jeremy) could use to make tw better. I use JSDOC to document the plugin api (http://bit.ly/tiddlymap_api) and I can only recommend it.

-Felix

[Jeremy Ruston]

Thanks for bringing this up. After a bit of digging I found an old version of TW5 that renders JavaScript through a JavaScript parser; I've mentioned it a few times in the hangouts I think.

It's from August 2012. You can download the ZIP file and view the index.html here:

https://github.com/Jermolene/old.jermolene.github.com/tree/e7bbdc735963b7d9dc84798eec176193ad54831d

Navigate through Shadow Tiddlers to one of the .js shadows and you'll see how JS code is highlighted and comments are rendered as wikitext sticky notes (illustration attached).

I'm still quite keen on this approach because it's very native to TiddlyWiki, leveraging a lot of existing code. I think that the standards discussed in this thread could perhaps be implemented as macro definitions.

Best wishes

Jeremy.

--
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywikidev.
For more options, visit https://groups.google.com/d/optout.

--

Jeremy Ruston
mailto:jeremy...@gmail.com

[Tobias Beer]

Hi Felix,

 

When it comes to API docs, we should stick to standards like JSDOC. Code needs to be documented in a most comprehensible and standard compliant way to ensure other developers can pick it up and understand it fast when acutually reading the code.

I find it difficult to agree. Perhaps that is owed to my not being a hardcore developer but I would think that there factually is no such hardcore documentation standard. There are project guidelines, styles and implementations ...adopting this or that method, maybe jsdoc.

In some distant past, some text editor developer thought it pretty to generate...

/**
 * @param foo {string} does bar
 * @return {strong} gives blip
 *
 * bing bong
 */

..I don't agree at all that we need or want to see or write those superfluous asterisks. Why would I bloat my code and documentation with that? I'll never ever use emacs in my life. That boat has sailed.

To me, it's quite important that we input the documentation in precisely the way we know it will render as a tiddler. This seems rather important so as to make things right. Doing the mental yoga to anticipate what some parser will do with it seems a rather poor design choice, imho. Let's stick to .tid format! We don't need jsdoc for TiddlyWiki... and if others at some point want to render a TiddlyWiki based documentation for their (particular flavor of) jsdoc documented code, then there will (have to) be someone willing to write that jsdoc parser that turns that into tiddlers. Why would we waste time and energy on that? What we need rather sooner than later is documentation that works for us, not this "standards compliance" or that.

Keeping it simple and smart.

Also, I am against reinventing the wheel it just binds way too many developer resources. Implementing a comment parser with all features is a huge project. This is valuable time we (especially Jeremy) could use to make tw better. I use JSDOC to document the plugin api (http://bit.ly/tiddlymap_api) and I can only recommend it.

That's what I'm saying, let's not waste time on writing a jsdoc parser to crank a lot of wheels so as to turn that into tiddlers! Let's use tiddler format, because we sure know how to parse that and we know how those things end up in TiddlyWiki! Simple, straight forward.

I'll put up a draft on GitHub on how I would envision this and perhaps we can discuss the specifics there. I'd really appreciate if I never see those asterisks in that doc... or any characters like @ that simply yield invalid tiddler fields ...or are translated into some other string so as to be valid.

It's from August 2012. You can download the ZIP file and view the index.html here:
https://github.com/Jermolene/old.jermolene.github.com/tree/e7bbdc735963b7d9dc84798eec176193ad54831d

Other than in the code itself, I would perhaps not intermingle documentation and code like that when presenting... in the docmentation wiki that is. I'd rather have a parser that runs some build-doc command which outputs an edition with the documentation tiddlers, so they can be properly tagged, reused, etc... Those sure have a relation to the code they are documenting, but they are not actually shown within that code. The code would be documentation free, also in the actual build of empty.html as we'd otherwise ridiculously explode the core with documentation. Simple comments could perhaps stay...

//I stay in the build, maybe...

Best wishes, Tobias.

[Tobias Beer]

I mean, we can sure implement jsdoc, but I believe it will significantly slow down the process and make things more complicated for the TiddlyWiki project... and only maybe easier for everyone else.

On the other hand, if you want to become familiar with TiddlyWiki's code, maybe grab some basics about how simple it is first, e.g. the tid format, and then perhaps forget jsdoc ...when given a chance to, after all.

To me, the focus is not so much to provide a jsdoc compliant TiddlyWiki doc process... but rather for us to get that API documented and have easy means to give it some smarts... WITHIN that API TiddlyWiki ...not whichever output some YUIDOC or jsdoc node implementation generates.

So, the barrier for jsdoc to be a good candidate for this project to me truly is this: Who is willing to write the parser for it? ...and willing to listen to all the complaints about "Why do I need to put all those ridiculous asterisks? They just clutter my TiddlyWiki / Markdown format." Why do I need @ param when it's all clear "_foo" is a param?

Sure, using jsdoc it would possibly be easier for someone else to output the TiddlyWiki documentation in a different way. But then I feel like I want to ask: Why? TiddlyWiki is a near perfect environment to document / expose its own API... and possibly any other js API, maybe, later...

Best wishes, Tobias. 

[Jim Lehmer]

I think what you're missing is the jsdoc parser (and node integration) is already written, and in fact has been long written, and stable.

On 2/5/2015 9:18 AM, Tobias Beer wrote:

I mean, we can sure implement jsdoc, but I believe it will significantly slow down the process and make things more complicated for the TiddlyWiki project... and only maybe easier for everyone else.

On the other hand, if you want to become familiar with TiddlyWiki's code, maybe grab some basics about how simple it is first, e.g. the tid format, and then perhaps forget jsdoc ...when given a chance to, after all.

To me, the focus is not so much to provide a jsdoc compliant TiddlyWiki doc process... but rather for us to get that API documented and have easy means to give it some smarts... WITHIN that API TiddlyWiki ...not whichever output some YUIDOC or jsdoc node implementation generates.

So, the barrier for jsdoc to bea good candidate for this project to me truly is this: Who is willing to write the parser for it? ...and willing to listen to all the complaints about "Why do I need to put all those ridiculous asterisks? They just clutter my TiddlyWiki / Markdown format." Why do I need @ param when it's all clear "_foo" is a param?

Sure, using jsdoc it would possibly be easier for someone else to output the TiddlyWiki documentation in a different way. But then I feel like I want to ask: Why? TiddlyWiki is a near perfect environment to document / expose its own API.

Best wishes, Tobias. 

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywikidev/VE4eVzoaReQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywikide...@googlegroups.com.

[Tobias Beer]

I think what you're missing is the jsdoc parser (and node integration) is already written, and in fact has been long written, and stable.

Ok, so what does it return? Is that usable for us? We still want to turn it into tiddlers, at least I sure do. I don't want some pdf or HTML of which YUIDOC thinks its perfect. I believe, we also want that function head parsed, not just the comments. Does it do all that in a way that we can, say, handle some big JSON output and generate the tiddlers from it?

Even if all that was a given, it would be much harder to understand how things are eventually rendered as a tiddler, just by looking at that jsdoc compliant documentation.

How do we extend that jsdoc format with our own needs, e.g. tags?

Here's that draft I said I'd make...

https://github.com/tobibeer/TiddlyWikiDoc

Best wishes, Tobias.

[Jim Lehmer]

It DOES do all the function header and parameter parsing. It gives a series of HTML pages that are all nicely cross-linked, decently styled and perfect for the (already static) dev doc site.

Frankly I am with Felix on this one. You think it won't be that much effort to add it to TW, and maybe with what Jeremy was pointing at, that's true. But I was looking at the project as "Let's get this stuff documented to help people (like me) bleeding through trying to understand how all the code fits together NOW," and not another feature to add to TW (isn't there a moratorium? :)

To me, the thing worth investing time in was actually adding the documentation comments to the code (which doesn't affect the code in ways that needs extensive testing) to then get the documentation generated and PUBLISHED quickly. I could get behind volunteering to write some of that documentation (because doing so would help me solidify my understanding of the codebase as a whole). I wasn't looking for yet another coding project, which the TW approach would entail, so am not that interested in volunteering for that. Finite amount of time, different priorities and all that.

The TW approach may be more "TW-pure," and more power to whoever implements it.

On 2/5/2015 9:47 AM, Tobias Beer wrote:

I think what you're missing is the jsdoc parser (and node integration) is already written, and in fact has been long written, and stable.

Ok, so what does it return? Is that usable for us? We still want to turn it into tiddlers, at least I sure do. I don't want some pdf or HTML of which YUIDOC thinks its perfect. I believe, we also want that function head parsed, not just the comments. Does it do all that in a way that we can, say, handle some big JSON output and generate the tiddlers from it?

Here's that draft I said I'd make...

https://github.com/tobibeer/TiddlyWikiDoc

Best wishes, Tobias.

[Felix Küppers]

Hi Tobi,

I find it difficult to agree. Perhaps that is owed to my not being a hardcore developer but I would think that there factually is no such hardcore documentation standard. There are project guidelines, styles and implementations ...adopting this or that method, maybe jsdoc.

JSDOC is highly similar to PHPDOC (http://phpdoc.org/docs/latest/index.html) to JAVADOC (http://en.wikipedia.org/wiki/Javadoc)... also it the standard at Google. If developers look at the tiddlywiki code they should feel familiar with the doc comments and not be scared or confused.
 

In some distant past, some text editor developer thought it pretty to generate...

/**
 * @param foo {string} does bar
 * @return {strong} gives blip
 *
 * bing bong
 */

 

..I don't agree at all that we need or want to see or write those superfluous asterisks.

 It is a convention so your editor knows how to highlight the block-comments and people can see that each line belongs to a comment block dedicated to one topic. This has really nothing to do with emacs...

To me, it's quite important that we input the documentation in precisely the way we know it will render as a tiddler. This seems rather important so as to make things right. Doing the mental yoga to anticipate what some parser will do with it seems a rather poor design choice, imho. Let's stick to .tid format!

Do you mean the dictionary format? How can I document nested objects with that or go multiline?
 

Keeping it simple and smart.

Do you want to write a parser that automatically does all the rendering and automatic linking?

That's what I'm saying, let's not waste time on writing a jsdoc parser to crank a lot of wheels so as to turn that into tiddlers! Let's use tiddler format, because we sure know how to parse that and we know how those things end up in TiddlyWiki! Simple, straight forward.

Why do you want to do this with tiddlywiki? This is just code documentation directed at developers. Joomla and Drupal are Content Management systems. Do they say: "let's develop and write our code docs in a joomla parsable format and publish it as a Joomla or Drupal Website?"

So I am strongly in favour of using a standard here.

[Tobias Beer]

Hi Jim,

 

It DOES do all the function header and parameter parsing. It gives a series of HTML pages that are all nicely cross-linked, decently styled and perfect for the (already static) dev doc site.

That right there is the reason which makes it a no-go for me.

Frankly I am with Felix on this one. You think it won't be that much effort to add it to TW, and maybe with what Jeremy was pointing at, that's true.

Adding it may not be difficult, but it will tremendously constrain our workflow and means to present things the way we want or need, rather than they think is most suitable.

 

But I was looking at the project as "Let's get this stuff documented to help people (like me) bleeding through trying to understand how all the code fits together NOW," and not another feature to add to TW (isn't there a moratorium? :)

Yes! Perfectily fits the moratorium and would possibly be a prime candidate for the top of the "documentation taskforce todo list".

To me, the thing worth investing time in was actually adding the documentation comments to the code (which doesn't affect the code in ways that needs extensive testing) to then get the documentation generated and PUBLISHED quickly.

To me as well, and I will possibly keep on waving that flag not to adopt jsdoc or yuidoc or foodoc.

 

I could get behind volunteering to write some of that documentation (because doing so would help me solidify my understanding of the codebase as a whole).

Great! And I believe once we have stamped out the workflow, we can and should go ahead and start documenting functions, whether we have that parser up and running or not.

I wasn't looking for yet another coding project, which the TW approach would entail, so am not that interested in volunteering for that. Finite amount of time, different priorities and all that.

The thing is, not using TiddlyWiki will create more work and constraints... is what I think. We do want our stuff documented the way we need it, e.g. including other reference content for example. Using some predefined xdoc templates will be a pain in the b.

The TW approach may be more "TW-pure," and more power to whoever implements it.

I am quite certain it will actually boost the project as a lot of people will be quite interested in using it themselves seeing how simple it will be. Sure, it will also be simple with YUIDOC, simple in the way they think is best for you... not my cup. Not because I wnt to do it my way, but because I know we can do better and should. To me, this is what TiddlyWiki is made for and good for, not just some basic "nonlinear personal notebook". ;-)

Best wishes, Tobias.

[Tobias Beer]

Hi Felix,

 

Do you mean the dictionary format? 

How can I document nested objects with that or go multiline?

I mean a simple...

title:foo
bar:baz

text

...all neatly enclosed for "doc'ing"...

/**
title:foo
bar:baz

text
*/

If an editor needs superfluous asterisks to highlight comment blocks, then maybe it's a "suboptimal" editor.

/**
 * title:foo
 * bar:baz
 *
 * text
 * */

Do you see how much that impinges on readability? ... if we wanted it to be tid-style?

To me it's shooting our own foot, with little peas maybe. ^^

 

Do you want to write a parser that automatically does all the rendering and automatic linking?

The parser wouldn't do that. That's the entire point. The parser would only output straight forward tiddlers. So, the minimal logic that is actually required goes into components for a TiddlyWiki API edition that does all the cross-referencing with TiddlyWiki elements, e.g. macros, viewtemplate sections, etc. That's very straight forward stuff... and a pretty good case for leveraging and showcasing the power of this thing we all like.

 

Why do you want to do this with tiddlywiki?

Because, that's the entire point of TiddlyWiki... to be used for documentation... in this case, it's very own API.

This is just code documentation directed at developers. Joomla and Drupal are Content Management systems. Do they say: "let's develop and write our code docs in a joomla parsable format and publish it as a Joomla or Drupal Website?"

The difference is, those are not designed for that task, TiddlyWiki is. All it takes is a rather simple build script (within TiddlyWIki), one that isn't all too different from the one that creates TiddlyWiki itself... and we're set.

I encourage you to take a look at that draft on GitHub, maybe that gives a more concrete image.

[Jeremy Ruston]

>All it takes is a rather simple build script (within TiddlyWIki), one that isn't all too different from the one that creates TiddlyWiki itself... and we're set.

With the approach that I'm suggesting we'd only need a build process for generating static documentation snapshots. For interactive use, you'd just install the JS parser and view the raw tiddler modules to see the formatted documentation inline with the code. The sample I showed had CSS that displayed the comments as sticky notes, but something like this would be preferable:

http://underscorejs.org/docs/underscore.html

Best wishes

Jeremy.

 

On Thu, Feb 5, 2015 at 4:25 PM, Tobias Beer <beert...@gmail.com> wrote:

Hi Felix,

 

Do you mean the dictionary format? 

How can I document nested objects with that or go multiline?

I mean a simple...

title:foo
bar:baz

text

...all neatly enclosed for "doc'ing"...

/**
title:foo
bar:baz

text
*/

If an editor needs superfluous asterisks to highlight comment blocks, then maybe it's a poor editor.

/**
 * title:foo
 * bar:baz
 *
 * text
 * */

Do you see how much that impinges on readability? ... if we wanted it to be tid-style?

To me it's shooting our own foot, with little peas maybe. ^^

 

Do you want to write a parser that automatically does all the rendering and automatic linking?

The parser wouldn't do that. That's the entire point. the parser would only output straight forward tiddlers. So, the miniml logic that is actually required goes into components for a TiddlyWiki API edition that does all the cross-referencing with TiddlyWiki elements, e.g. macros viewtemplate sections, etc. That's very straight forward stuff... and a pretty good case for leveraging and showcasing the power of this thing we all like.

 

Why do you want to do this with tiddlywiki?

Because, that's the entire point of TiddlyWiki... to be used for documentation... in this case, it's very own API.

This is just code documentation directed at developers. Joomla and Drupal are Content Management systems. Do they say: "let's develop and write our code docs in a joomla parsable format and publish it as a Joomla or Drupal Website?"

The difference is, those are not designed for that task  TiddlyWiki is. All it takes is a rather simple build script (within TiddlyWIki), one that isn't all too different from the one that creates TiddlyWiki itself... and we're set.

I encourage you to take a look at that draft on GitHub, maybe that gives a more concrete image.

https://github.com/tobibeer/TiddlyWikiDoc

Best wishes, Tobias.

--

You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.

To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywikidev.
For more options, visit https://groups.google.com/d/optout.

[Tobias Beer]

Hi Jeremy,

 

For interactive use, you'd just install the JS parser and view the raw tiddler modules to see the formatted documentation inline with the code.

That might constrain documentation abilities quite a lot as it's difficult to address and manage individual functions other than render them somewhat distinct from the surrounding code.

~

Addendum:

Come to think of it, we actually don't need any dedicated doc-build process.

All of this can be done in an API TiddlyWiki, e.g. on startup... parsing all *.js tiddlers of $:/core for those doc-comments and creating shadow tiddlers from them which thus form the documentation.

So, what is needed is actually for the default build process to strip all /**documentation comments*/ and then some switch that gives the entire source including doc-comments for the API edition... which may also have a static presentation, but I would sure see a lot of benefit in it being a full-blown TiddlyWiki... even thouh extending static site capabilities with some indexing / cross-linking smarts is also a worthwhile endeavour.

Best wishes, Tobias.

[Jeremy Ruston]

Hi Tobias

That appears to quite constrain documentation abilities as there is no way to address and manage individual functions other than render them somewhat distinct from the surrounding code.

Do you mean that you want to browse the code by function/method rather than by file/tiddler?

Best wishes

Jeremy.

 

Best wishes, Tobias.

[Tobias Beer]

Do you mean that you want to browse the code by function/method rather than by file/tiddler?

Basically yes, or rather both. I see the power in a full-blown TiddlyWiki in its abilities to leverage individual tiddlers — in this case individual API functions and modules — rather than entire module files ...and to extend that with ui smarts, e.g. automatic crosslinks, external links, templates, etc... about parameters, return values, js objects, data types, etc...

Now, that could possibly all be done "one-by-one" by whichever renderer transforms those inline comments into a different presentation... however, for any wiki smarts to enhance the experience, all documentation comments would need to be preparsed into documentation tiddlers with whichever properties would help make that API a little more than just highlighted text.

At the moment I believe it might be a similar process to "unpacking the core"... e.g. "extract those documentation tiddlers and produce shadows from them"... albeit not system shadows, but "documentation shadows", i.e. not under any $:/foo namespace but rather just api:foo... pointing back to the actual code.

However, for the actual module tiddlers to not be "bloated" with documentation blocks when looked at, those comments would preferably be stripped from them... in view-mode ...as they now are "documentation shadows". There could be a special widget mode for the transclude widget that has some switches to remove those comments in view-mode while leaving them untouched in edit-mode. This is possibly preferable as the actual line numbers would remain intact, including the doc-comments.

Best wishes, Tobias.