About Documentation

[Nicolas Cannasse]

Hi,

While discussing on [haxelang] about the opportunity of integrating the
Flash API documentation into Haxe code, I started thinking about ways to
improve the current documentation.

Current documentation available on http://haxe.org/api works as the
following :

- it can be inserted in .hx files of haxe/std
- it can also be edited on the wiki (and translated in many languages)

When we make a new release, the API documentation is synchronized :

- we add new classes, methods and vars and remove the
no-longer-available ones
- we keep the previous wiki documentation unless if was empty and some
was added to .hx sources

It works well since it allows us to synchronize things without losing
changes.

But :

- not so many people are actually contributing documentation on the
wiki, so I guess we need more dedicated people to that (the foundation
could help making that happen by putting some money in the process)

- the API using the wiki syntax/mechanisms, its browsing is far from
being perfect, and would benefit a dedicated site (api.haxe.org ?)

- the API is just a comment, it might need some more complex semantics
in order to work well : parameters documentation, examples, link to
sources, etc.

What do you think of the fact that the API is not stored directly in .hx
files but instead edited on another website (currently the haxe.org wiki
or maybe later another site ?)

Pro :
- allows contribution from people not having commit access to Haxe/SVN
- allows translations
- more easy to edit/change/tweak (fixes can be applied in realtime
instead of having to run an doc generation + deploying the upgraded version)

Cons :
- is less likely to be up to date with the source (if the source change
we might forget to edit the documentation)
- cannot be displayed at completion (would require a separate export of
a documentation file that could be reused by the compiler)

I'm interested in hearing about your opinions on this topic, since I
think we have major progresses to make here, both in technical and
organizational terms.

Best,
Nicolas

[Franco Ponticelli]

I like the idea but I think that the disconnection from the code is too important to be disregarded. Having inline documentation in the IDE is extremely important in my opinion and I'm always perplexed when IDE do not do that by default. It is the most obvious place where the documentation should be.

Translations should be possible even if the documentation is stored in the code ... obviously their content must be kept somewhere else.

What if we keep the documentation external as you suggest and we use some kind of hook in the compiler to use that when available instead of the one in the code? That way we have a clear distinction or roles but we are still able to retrieve it when it is needed.

Franco

[Nicolas Cannasse]

Le 30/07/2012 18:27, Franco Ponticelli a �crit :

> I like the idea but I think that the disconnection from the code is too
> important to be disregarded. Having inline documentation in the IDE is
> extremely important in my opinion and I'm always perplexed when IDE do
> not do that by default. It is the most obvious place where the
> documentation should be.
> Translations should be possible even if the documentation is stored in
> the code ... obviously their content must be kept somewhere else.
> What if we keep the documentation external as you suggest and we use
> some kind of hook in the compiler to use that when available instead of
> the one in the code? That way we have a clear distinction or roles but
> we are still able to retrieve it when it is needed.

Yes, documentation at completion is clearly not a show-stopper. We will
just have to have an .hxdoc format that would store the documentation in
case it's not in the source code (would still be the default when you
have your own code).

Additional pros for the extern documentation :
- allows user comments
- allows examples (in a separate section, not directly inlined in the
class documentation)

Best,
Nicolas

[Lee Sylvester]

I've been thinking about this, too.  I think it's good to keep some documentation in the classes as needed ,but perhaps some kind of tagging can be applied in the wiki (package class function) whereby the info in the wiki can be presented alongside with ability to hide / show.  Sometimes it's better to see both generated and manual content.

Lee

Sent from my iPad

[Cauê Waneck]

I'm not sure this is a good idea.
IMHO, the haxe documentation is too open already, and opening it more is not a solution.
What I think we need is structure. We have a lot of great info lurking in the haxe wiki, but we hardly ever can find them, as they get lost in many old, irrelevant info or poorly organized structure.

My counter proposal would be to add support for tags in haxedoc (@param, @throws, @see, ...), and add a comments section (with maybe up/down voting), also add user-editable examples and related links.

Then we'd need to define an opionated standard for the haxe docs (that takes care of documenting target specific behavior, raised excwptions, etc). I like how apple does its documentation, it focuses on tasks (eg for haxe.io.Bytes it would have "Creating an Instance" ofData, alloc; "Accessing its contents"....)

Regardless of the format, we need to define one, and follow it. I volunteer myself in re-writing the core docs in the fornat decided. I still have a lot to document about the targets, and well, the std lib isn't so big after all ;))

--
Sent from Gmail Mobile

[Cauê Waneck]

Also, I've been thinking for a while about this, I think it would really be awesome if we could hire someone right now to do some work for Haxe that never gets priority ;)

For example, doing what Greg has told us, about being approved by Norton Antivirus, singing our executables, and after all of this, structuring the haxe website. For example, I really think we could use a tour.haxe.org, which would use the try haxe api to propose an introduction to the haxe syntax and powrful benefits.

I would be glad to help monthly with money for that. 

2012/7/30 Cauê Waneck <wan...@gmail.com>

[Lee Sylvester]

Well, I’ve already volunteered for the book/manual, but I don’t want anything for it :-)

 

Lee

[Juan Delgado]

If you are going to update haxedoc can we make it fully
template-based? I gave it a go myself, ended up with a lame-half-arsed
experiment called xdoc:

https://github.com/zarate/xdoc

The idea is that anyone without major knowledge (aka, me!) should be
able to heavily modify the output. Not only styles, but ideally also
from HTML to PDF to CHM.

J

--
Juan Delgado - Zárate
http://zarate.tv
http://blog.zarate.tv

[Cauê Waneck]

That's true, Lee!

You are awesome :)

Cheers!

Cauê

2012/7/30 Lee Sylvester <lee.sy...@googlemail.com>

[Russell Weir]

Since 2008, chxdoc has supported all the javadoc style tags we could possibly need in a documentation system, so I propose that I merge chxdoc as a replacement to haxedoc.

I agree completely with Franco here... documentation in the source is vital. I hardly ever refer to the online docs myself, using only code for my reference, as I find the online docs don't ever reflect the nuances (throws, errors, params etc) of the methods. However, the documentation in the std lib classes does need direct work and perhaps translations.

So here's my idea. On the haxe.org website, there's an interface for translating or updating existing documentation, which writes to an sqlite file. I add support to chxdoc to pull translations from sqlite, and a config switch --language [en,fr] etc to it. Assuming the documentation official language is English, any entry in the db that is an 'English translation' would be treated as a correction or addition to the code docs, so there should be a tool for maintainers to manually view corrections and update the std lib sources accordingly. (Which would be a manual process including svn updates to haxe)

For any changes to the std lib code or docs during development, there should be a command line tool for the developer to 'invalidate' translations or delete them entirely. Basically, if there is a minor api change, or a change in English docs for a member, one might run "doctranslate invalidate haxe.io.Output.prepare", which would then be reflected on the website as 'any translation of haxe.io.Output.prepare needs updating'. If the api function is removed or the documentation undergoes a major change run "doctranslate remove haxe.io.Output.prepare" (here, Output.prepare was removed or the docs changed drastically). If a method or class is moved, "doctranslate move haxe.io.Bytes Bytes", which would move all haxe.io.Bytes to the root package (which I would love to see, hint hint.. but I digress).

The database would be synced to svn daily on any changes, and would basically be

path : String
language : String
doc : String
Primary key: path+language

Where path is the full path to the class for class level docs, or direct to any member inside the class (haxe.io.Output.bigEndian)

This does create some extra work for anyone contributing to the std lib sources, but I think it is well worth the effort. We need those docs updated and more docs added, as many things in the current documentation are just assumed to be known by the reader.

Cheers,
Russell

[Russell Weir]

By the way...

haxedoc was updated yesterday for haxe 2.10, I added xml config file support today (not in svn yet), and a little known fact is that chxdoc runs natively in mod_neko (though I haven't tested it lately, since I have no mod_neko server).

[Nicolas Cannasse]

Le 31/07/2012 23:46, Russell Weir a écrit :
> Since 2008, chxdoc has supported all the javadoc style tags we could
> possibly need in a documentation system, so I propose that I merge
> chxdoc as a replacement to haxedoc.

Thanks Russell for joining the discussion.

I kind of agree with your proposal, but still trying to figure out how
exactly to put all these pieces together.

Currently we have :

- haxe -xml which extracts type signatures + "raw" documentation into a
simple XML file

- haxedoc, chxdoc, and hxWiki ApiSync that uses these XML to render
HTML/Wiki-Markup

What we would maybe need is new file format ".hxdoc" that would be maybe
a bit more structured (separate @param/@return/@throw documentation,
maybe examples in a separate section, etc.).

And from there :

- we could import/sync both .hxdoc and haxe -xml output into a database
- then from a web interface we could easily display + modify/enrich the
documentation (using wiki markup syntax ?)
- then we could export an .hxdoc file from the database that could be
read by the compiler to extract the documentation and display it at
completion time.

Does that sound like a good idea this way ?

Best,
Nicolas

[Philippe Elsass]

Nicolas, would this system be bi-directional?

As far as completion is concerned, having raw doc is fine because IDEs which are able to display doc are already parsing them. At least FlashDevelop only keeps raw doc in memory and parse it when displaying a tip.

[Nicolas Cannasse]

Le 01/08/2012 10:45, Philippe Elsass a �crit :

> Nicolas, would this system be bi-directional?

If you mean be able to inject back the doc into code, I guess that could
be done.

But since documentation is usually provided at completion time, and
since Haxe compiler handles the completion, we could also support .hxdoc
which would be more efficient than parsing/formatting the raw documentation.

Best,
Nicolas

[Cauê Waneck]

Imho, A bi-directional system would ve too complex. Maybe a "report" button, plus comments, plus user editable "related" and "examples" would be more than enough

Le mercredi 1 août 2012, Nicolas Cannasse <ncan...@motion-twin.com> a écrit :

> Le 01/08/2012 10:45, Philippe Elsass a écrit :
>>
>> Nicolas, would this system be bi-directional?
>
> If you mean be able to inject back the doc into code, I guess that could be done.
>
> But since documentation is usually provided at completion time, and since Haxe compiler handles the completion, we could also support .hxdoc which would be more efficient than parsing/formatting the raw documentation.
>
> Best,
> Nicolas
>

[Nicolas Cannasse]

Le 01/08/2012 11:08, Cau� Waneck a �crit :

> Imho, A bi-directional system would ve too complex. Maybe a "report"
> button, plus comments, plus user editable "related" and "examples" would
> be more than enough

Well having the following schema is quite easy :

code -> hxdoc <-> database

We could then think about having (code <- hxdoc) as well

Best,
Nicolas

[Russell Weir]

I think the simpler solution works. I find that using php's doc pages is very helpful, with all the user discussion as a message board for each function. Tips and usage examples right there at each function, which never has to be synced anywhere.

On that same page, a button allows the user to update the source code documentation, which would just stuff it in the database, email the documentation maintainer, who approves or denies it, and changes the source file accordingly.

Imho, there aren't enough methods in the standard library to make a complex system worthwhile. After an initial rush of doc changes and manual updating of source code, maintenance would be small.

[Cauê Waneck]

I agree!
Also I don't like the separation of doc and source code that this system would add

[Andy Li]

What about making a 2-way github mirror, and then let ppl send pull request on documentation improvements?

The haxe.org API page may be built automatically on a new commit and then provide a link to the edit page of the corresponding file on github.

Best regards,

Andy

[Andy Li]

To make it clearer, I mean letting ppl to modify the inline doc in the haxe source code.

[Lee Sylvester]

I'm in agreement. We run the risk of screwing up peoples commenting when the door swings both ways :-)

Lee

[Lee Sylvester]

That's potentially a solution, but this way we still run the risk of the docs being behind the code, which is what we're trying to avoid.

Lee

[Cauê Waneck]

2012/8/1 Lee Sylvester <lee.sy...@googlemail.com>

That's potentially a solution, but this way we still run the risk of the docs being behind the code, which is what we're trying to avoid.

 

Are we trying to avoid this? :) 

[Andy Li]

Is there a way to avoid the docs being behind the code?

Making it compulsory to commit code + docs to the svn in the same commit??

With github two-way sync:

• Both haxe developer and community can contribute to the inline docs.
• Docs can be reviewed and discussed before merging.
• I believe inline docs from compiler developers is already the most updated one => the auto-generated API page will always be the most updated.

What do you think?

Best regards,

Andy

[Lee Sylvester]

I'd certainly like to think so :-). My supposition is that developers are more inclined to update inline documentation as they update classes and method signatures.  Next would come the wiki.  A separate doc on GitHub would come last, I'm sure :-)

Lee

[Lee Sylvester]

There's no way to force a dev to maintain docs :-). Most lib changes occur as the dev requires them, so work comes before documenting.  Ergo, documenting will update when the developer has time.

Lee

[Cauê Waneck]

2012/8/1 Lee Sylvester <lee.sy...@googlemail.com>

I'd certainly like to think so :-). My supposition is that developers are more inclined to update inline documentation as they update classes and method signatures.  Next would come the wiki.  A separate doc on GitHub would come last, I'm sure :-)

Ah Sorry! I understood wrong what you meant! :)

[Andy Li]

On Wed, Aug 1, 2012 at 10:55 PM, Lee Sylvester <lee.sy...@googlemail.com> wrote:

There's no way to force a dev to maintain docs :-). Most lib changes occur as the dev requires them, so work comes before documenting.  Ergo, documenting will update when the developer has time.

That's what I thought :)

A separate doc on GitHub would come last, I'm sure :-)

Can't agree more. Just a note that I'm not proposing to use a separate doc, but merely using github for accepting contribution(pull request).

Thus the current API page will still be used, and will be generated from the haxe source like it has always been (but automated on each commit). It is just the edit button will redirect to the github source for ppl to fork and edit, instead of saving the changes to the haxe.org wiki database.

Best,

Andy

[Russell Weir]

Using existing tools is always a good option, however not all users
know Git well enough to take all that time to make a change to one
@param in one method, which will reduce the amount of submissions from
the community. However, allow the manual git submissions, but also
make the website submit git requests for user edits on the web side.
That way they don't have to 1) get git, 2) install git, 3) learn git
4) checkout haxe etc. etc.... that's a lot of work for a new git user.

If both submission paths (web and manual git) lead to a git pulls, it
makes it easy for the documentation maintainer to manage submissions.

[Andy Li]

No, ppl do not need to learn git to use github ;) But a github acc. is needed.

For example, visite https://github.com/jgranick/NME/blob/master/nme/display/Sprite.hx

There is a "Edit" button at the top-right corner of the source.

Having the git pull running behind, instead of using github directly sounds possible,

and it saves the need of registering on github.

But I'm not sure how easy to get it running and things might get more complex if there is a conflict.

Best,

Andy

[Joshua Granick]

I think Nicolas has a sensible approach.

To my knowledge, no one is suggesting we eliminate code completion driven from source code. This is an idea that would provide an alternative.

By providing completion using a separate document, it makes it much cleaner to update completion (driven from the wiki) and to even support multiple languages.

For NME, we could benefit because the exposed user API for code completion is (and should be) different than all of the properties which are accessible. Since NME has multiple backends, it is also impractical to include documentation directly in each source file.

[Plop]

Well having the following schema is quite easy : 
  code -> hxdoc <-> database 
We could then think about having (code <- hxdoc) as well 

+1. it's quite easy and 3 repo allows everything we want. 

+1 git repos  : A decentralized approach will benefit to lib, and local projets too.

to simplify the life cycle of the differents parts : 3 Repositories :  codeRepo, hxdoc, database. (eventually additional local repo via haxelib?)

• local code <-> codeRepo  : (svn, cvs... but decentralized git, mercurial seems to be the best)
• local code -> hxdoc   : local code could generate hxdoc localy. (code completion use hxdoc official buch or generated... hxdoc is a cache)
  
• codeRepo -> database   : doc in code repo is merged to database.
• web frontends <-> database  : (wiki, example.haxe.org...) get/set data from database.
• codeRepo <- database : updated doc from database is merged into source code. (light doc.. maybe english only)
  
• hxdoc <- database   : full hxdoc bunch (multilangual, howto, articles...) are generated from database too. (official in depth doc) like doc.jar in java world

The persistence need versionning. we could use hibernate/envers... (an abstract layer for persistence will be better...)

But, decentralized versionning system ( git, mercurial...) seem to be the most efficient. 

We only need a service layer between all the repositories to update the doc. 

The repositories can be hosted on gitub (benefit of pull request, fork...) or on custom local/inhouse places. ( like Maven allows ).

 i think a solution based on git and neko should be the best. git protocol allows bindings to community platforms, and it's decentralized. For companies, using neko is a great showcase. Locally, compiler and haxelib (more git integration) could be used in addition to manage the code (hxdoc, std), from local repo to the mainCodeRepo and hxdoc repo, in both way.

 

Like litterals it's simpler to write doc in code. for small and offline projects it's easier to drive the doc from the sources. 

Database allows any kind of web frontends. (example.haxe.org , wiki, ...)  wiki enable easy translations, contributions, examples, full documentation written by dedicated peoples. The system allows sync of doc when source code change, and warnings when the doc need an update.

Official bunch releases of hxdoc could be provided and used in priority by autocompletion. but by default , thanks to codeRepo <- database and then   local code -> hxdoc   we can have light doc (without howto articles) just from source code. Sync back to code allows a single document to rules them all (easy commit/merge of doc changes, local projet can use transparently .hx and have autocompletion).

The service of doc merging between web frontends, database and main code repo  should be hosted by the fundation : Free for opensource libs and accessible for companies as a service if we want this to take part of the business model.

Best