[TW5] TiddlyWiki code structure and documentation, try 2. Any volunteers?

[Jed Carty]

Instead of working on what I was planning today I started putting together another version of a documentation wiki for the tiddlywiki code.

My main motivation is to have a searchable database so I can look up function definitions when developing. I would like to extend this to having more in-depth documentation for the code to make contributing and learning easier. It should also help give a better overall view of the code to help with optimisations and the like.

What I would like is help adding everything and input about how it should be organized. At the moment it is mostly empty placeholder tiddlers.

What I have now is here.

If you are interested in helping let me know. It needs people to do things like adding descriptions for each file (a lot of which will be things like 'This is the tiddler for the right arrow image'), adding descriptions of what functions do, giving examples of usage for a function, finding where the different functions are used and probably other things.

[Mat]

Jed - while admirable, I think it is the wrong approach to statically document anything for TW, and one we've used just so many times for other TW documentation projects - with failure after failure. The most successful example using this documentation approach is tiddlywiki.com - but even this exceptional case suffers problems that stem from a static approach; The docs on tiddlywiki.com ARE very good (of course) but it is problematic to revise the material and keep it up to date due to the process that all content there has to go through (which effectively burdens one single individual), and the documentation must keep a rigid structure that is good is some cases but unfortunate in other (...for example, a user wondering how to use "tags" who types this into the search field on tiddlywiki.com... will not be happy.)

The superior way for a project like this is what you've already laid the foundation for, namely TWederation. (Ah, of course ;-)

IMO the following is what is required for a successful TW documentation project:

1. get TWederation working
   
2. set up a documentation TW that fetches peoples documentation
   
3. ...which for a particular "doc-topic" filters the relevant tiddlers to make an aggregated topic-display
   
4. include "quality systems" such as some voting mechanism or versioning to have tiddlers qualify for that topic-display
   

The key is to not rely too much on any one individual. The high activity on the boards is a definite indication that there is no end to questions so documentation has to be flexible and allow for accumulation rather than the typical static documents we've had so far.

Just my thoughts.

<:-)

[Jed Carty]

That is the goal, but I want some structure for this part. There is less flexibility for the javascript documentation than for general user documentation. 

I will move back to working on twederation soon. I haven't figured out why it didn't work from dropbox and it is very frustrating. 

[Mat]

@Jed - might some stuff from gihub be directly transferrable to this? To take an arbitrary example, the browser-sniff plugin has a description.

...which makes me think that perhaps;

@Jeremy - do you perhaps have structured notes/descriptions on functions and/or tiddlers that potentially could be copied into the type of thing Jed is proposing?


<:-)

[Mark S.]

Wouldn't this be better posted in the developer's group? They're the ones that would know how to fill out the descriptions for the functions, etc.

The problem with the standard TiddlyWiki documentation is that it has to go through the same onerous vetting that the actual code does, and you don't get to see the results for weeks or months when the next release comes out. Pretty dispiriting actually.
 
Good luck!
Mark

[PMario]

Hi Jed,

Just to be sure. There has been a hangout, where Jeremy made a "code walk-through". see: http://hangouts.tiddlyspace.com/#HangOut_010

It's relatively detailed. So you can link directly to youtube. ...

It was one of the first hangouts, but the main code structure hasn't changed since then.

have fun!
mario

[PMario]

My general thoughts about TW "reference documentation"

IMO the only way, we can keep a reference docs up to date is, that we create the reference docs within the source code and extract it automagically.

Jeremy's viewpoint is, that there is enough "inline" documentation in the code already. ... That's probably true -- for him :)

So we can't expect him to add more. That's ok. It should be a community effort to add and extract what new contributors need. TW has the possibility built in, to extract javascript comments and transfer them into readable html pages or even TWs. ...

The problem here is, that it needs Jeremy to merge pull requests, which is very time consuming. ... So imo we need a mechanism, that allows "key contributors" to merge those "docs-only-PRs", without the need for Jeremy to "waste" his time.

I did introduce one possible way in a hangout lately. ... But this mechanism has to be discussed and refined. ... Which will be one major topic for me in the July meetup.

The same is true for translations. Translations are 2 fold

 a) core UI text translations will always be "core" PRs
 b) Language specific editions eg: german, french, ... versions can be separated

Language specific landing pages, should stick to the TiddlyWiki "Corporate Identity" but can be completely different, depending on the target audience. Different countries have a different "cultural" background and imo a landing page should reflect this. ....

just some thoughts
-mario

[Jed Carty]

This comment is strictly in the context of developing using the javascript portions of tiddlywiki. This would be a developers tool, not a users guide.

For the code it isn't that there isn't enough documentation for the core, but that it isn't easily searchable. At a minimum I would like a function reference. I keep finding myself having to search through a bunch of core files to find the definition for something like getTextReference, so even a list like I already have that lets you search for a function and see what file it is in will save me a lot of time. Documentation or examples on top of that will hopefully be useful for lowering the barriers to entry for new people.

I would like, even as just an experiment, to cut up the code so each function has its own tiddler and then use templates to build the source code. I think that there could be many benefits to usability by doing this, like making it easy to search, helping reduce redundant code and allowing much more complete documentation along with the code itself. Last time I tried starting this Jeremy had brought up a similar project he had attempted a while ago, so I know I am not the only one with the idea. I am sure there will be plenty of problems and we would need to build new tools to make it work.

[Richard Smith]

Hi Jed,

Do you use an integrated editor to do your coding? I use ATOM and find it quite easy to search for functions across a whole project folder (ie; the TW repo)

The idea of rebuilding the code so that each function //is// a tiddler is very interesting.

Regards,

Richard

[PMario]

On Sunday, June 5, 2016 at 11:46:32 AM UTC+2, Jed Carty wrote:

This comment is strictly in the context of developing using the javascript portions of tiddlywiki. This would be a developers tool, not a users guide.

I didn't write about a users guide.
 

For the code it isn't that there isn't enough documentation for the core, but that it isn't easily searchable. At a minimum I would like a function reference. I keep finding myself having to search through a bunch of core files to find the definition for something like getTextReference, so even a list like I already have that lets you search for a function and see what file it is in will save me a lot of time. Documentation or examples on top of that will hopefully be useful for lowering the barriers to entry for new people.

As Richard wrote, this is also a problem of the dev environment. I use brackets.io (without Extract) [1] editor, which allows me to highlight a function and type CTRL-E, which opens an inline editor, with the function code. The "global search" utility shows you all the lines where an eg: function is called. ... Opening the files is just a double click away.

 

I would like, even as just an experiment, to cut up the code so each function has its own tiddler and then use templates to build the source code.

I did some experiments with the "leo outliner" [2] as a TW IDE. ... The leo editor can be used as a literate programming tool. .. The problem is, that it uses <<...>> as the indicator for text macros. In the TW context this is similar to a "transclusion". The editor has  no possibility to re-define the sequence, so it's kind of useless, since TW syntax uses <<..>> for macros :/
 

I think that there could be many benefits to usability by doing this, like making it easy to search,

Yes. The workaround here is an editor that supports multi-file, multi-directory search and eventually "inline" function editing.

helping reduce redundant code and allowing much more complete documentation along with the code itself.

Right. This would be nice to have.
 

Last time I tried starting this Jeremy had brought up a similar project he had attempted a while ago, so I know I am not the only one with the idea. I am sure there will be plenty of problems and we would need to build new tools to make it work.

That's right. ... but from my point of view, it needs to be something, that is created automatically. If human intervention is needed, we simply can't support it, as a "spare time" project.

just my thoughts
mario


[1] http://brackets.io
[2] http://leoeditor.com/

[Mark S.]

Hi Jed,

Searching your TW, I can't find a blank for "getVariable()". I also can't find it looking at TW's raw code, mostly because I don't know how it's prefixed as a definition and also because it gets used so many places that searching for its definition is swamped. Maybe it's part of some other code library? Having documentation for this sort of thing would be great.

Had a thought it might be a new part of standard javascript, but a search through the definitive guide reveals no such function.

Thanks,

Mark

On Saturday, June 4, 2016 at 6:51:45 AM UTC-7, Jed Carty wrote:

[Richard Smith]

Hi Mark,

getVariable is defined in widget.js 

If you search the whole project using an editor such as Atom or Brackets, you get a list of all the lines containing the string and you can look through to find the definition or you can try searching for "getVariable =" and it should find the place where there is a definition.

Regards,

Richard

[Jed Carty]

Only functions in wiki.js are in it so far. That is why I am asking for help.

[Jeremy Ruston]

Sorry to be late jumping into this thread.

I have a couple of thoughts:

* having observed other projects, the most robust way of producing API documentation is to use tooling to extract the documentation from the source code. In our case, we'd extract it into a TiddlyWiki that can add a decent searchable UI over the top. That's why I've generally tried to use TW wiki text in the comments above functions in the core (or at least those that have a comment!). It looks like you're doing something like that, I recognise the main wiki.js docs

* to permit hyperlinking the docs better, I'd be interested in exploring splitting the core source files into individual tiddlers for each function. I'd want to explore the performance and usability implications of so many more shadow tiddlers, and so many separate files when using an ordinary text editor to browse the source

> Jeremy's viewpoint is, that there is enough "inline" documentation in the code already. ... That's probably true -- for him :)

That's not quite right, I absolutely acknowledge that the inline docs are inadequate at the moment.

> So we can't expect him to add more. That's ok. It should be a community effort to add and extract what new contributors need. TW has the possibility built in, to extract javascript comments and transfer them into readable html pages or even TWs. ...

I'd certainly appreciate help with filling out the code comments.

Best wishes

Jeremy

--

Jeremy Ruston

jer...@jermolene.com

http://jermolene.com

On 6 Jun 2016, at 09:34, Jed Carty <inmy...@gmail.com> wrote:

Only functions in wiki.js are in it so far. That is why I am asking for help.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/d521f241-f69c-49c5-8687-82971cc50b7b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[c pa]

I'm trying to do this same thing at tw-for-bunnies, but I'm approaching it from the other end. Here are my observations (some of which I'm thinking of on the fly)

# As a user I need to understand the structure of widgets and macros so I can use them myself
  #* The structure of widgets and macros documentation seems to be as follows
    #** Widget or macro name
    #** Widget or macro parameters
    #** Widget or macro description
    #** Widget or macro examples
    #** Widget or macro called_variables
    #** Widget or macro usage_by_other_widgets_macros_and_tiddlers
# As a developer I need to know if the requirement for documentation means that I need to change my coding style so I can adopt that standard once and then focus on development
  #* Jeremy has adopted the following standards
    #** <!-- html commenting his code -->
    #** Code comment includes
       #*** purpose
    #** Separate doc tiddler to document the macro or widget
    #** doc tiddler text field contains
       #*** Widget or macro name
       #*** Widget or macro parameters
       #*** Widget or macro description
       #*** [[Link to]] Widget or macro examples
    #** The code itself contains
       #*** Widget or macro called_variables
# As a documentation developer I need widgets and macros coded in a standard format so I can extract the documentation rather than documenting each widget or macro individually

It would be helpful if we could decide on a standard way to approach this and then Jeremy could publish that standard as part of the core

It seems we have much of a standard format defined for us. Once a standard is established, writing tiddlers and macros to build a documentation management interface would be fairly simple.

Next step would be figuring out a pipeline to feed community developed documentation and examples into the core. (This is the internet so the horde will skip and mix steps but . . .) Something like

# Anyone           develops things on their own
# Collaborators   post code to a "test it out and provide comments here" website such as twederation
# Moderators      browse the twederation to identify "good stuff" and provide professional polish
# Contributors     Post polished tiddlers to github
# Jeremy            says "Hey everyone needs to know this" and puts it in the core docs

I'm thinking of implementing the following in my documentation standard.

# Segregating each widget or macro into a separate tiddler
# Use the following fields in the macro or widget tiddler itself to outline the macro usage
#* name        = widget or macro name
#* parameters  = List of parameter names
#* description = Long description
#* caption     = Short description
#* tags        = If contains "$:/tags/macro" this is a macro
#* type        = if application/javascript this is a widget
# Use the following in each macro or widget example tiddler
#* example-for = List of macros or tiddlers that this example documents

Why I haven't adopted this yet
# During development of macros, I find that writing a whole bunch of macros together in one tiddler is the most efficient way to develop for a particular use-case since the macros tend to be dependent on each other

What I'm using instead
# Each use case is a separate tiddler with a whole bunch of macro definitions
# If there is only one macro in the tiddler, I add the following text and make sure the display is illustrative

! macro name
* `<macrocall $name="macro-name" parameter1="example" parameter2="example" />`
* <macrocall $name="macro-name" parameter1="example" parameter2="example" />

# If there are many macros in the one tiddler, I add text after the definitions to create a table, one row per macro with the illustration