Generating documentation using only JavaScript
LitenDavid
bit before settling for JsDoc as my documentation generator, much
impressed by it's ease of use and solid implementation.
Lately I found the need to generate documentation directly in the
browser, using only JavaScript. Is that possible using JsDoc? I've
wrestled a bit with the js parserrelated files, but couldn't get it to
work. I have a nagging suspicion that I'm going about things
completely the wrong way, but I didn't get any wiser than that..
Happy new years! :)
Michael Mathews
Thanks for your kind words. Your question is reasonable, in fact an
older version of JsDoc Toolkit actually supported on-the-fly
documentation generation (called "fly mode," versus "rhino mode"). I
eventually dropped this for several reasons:
1. It is very slow and very memory-hungry. With big source files it is
essentially impossible -- as the browser starts alerting about an
"unresponsive script."
2. JavaScript in the browser is crippled by an inability to get a
directory listing. This is on purpose, the browser actively wants to
prevent you from doing that. If you cannot get a directory listing,
which you can't, then that means the user must provide the exact name
of every source file. In large projects this is unwieldy.
3. It is useless. I have not yet heard a use-case where the
documentation needed to be regenerated every time someone looked at
it. API docs are simply not that dynamic. More likely the docs change
only very rarely, it's a wasteful design to require the exact same
output be regenerated over and over again.
Due to these reasons I've written JsDoc Toolkit so that it must be run
in Rhino. It may be possible to pull out parts and get it to run in a
browser, but can you justify the reason you think you need doc
generation to run in the browser? There may be another way to
accomplish the same thing.
Michael
> --
>
> You received this message because you are subscribed to the Google Groups
> "JsDoc Toolkit Users" group.
> To post to this group, send email to jsd...@googlegroups.com.
> To unsubscribe from this group, send email to
> jsdoc-2+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/jsdoc-2?hl=en.
>
>
>
Donny Viszneki
> 1. It is very slow and very memory-hungry. With big source files it is
> essentially impossible -- as the browser starts alerting about an
> "unresponsive script."
If a documentation generator is resigned to doing *anything* during
the initial pass, it could very easily be paring down the source files
to their doc strings and rolling them up into a browser-viewable
documentation file.
> 2. JavaScript in the browser is crippled by an inability to get a
> directory listing. This is on purpose, the browser actively wants to
> prevent you from doing that. If you cannot get a directory listing,
> which you can't, then that means the user must provide the exact name
> of every source file. In large projects this is unwieldy.
I don't think this is true. For one thing, web browsers are not
designed with an intent to prevent you (user or developer) from
accessing a "directory listing." Web browsers deal with URLs, which
despite looking hierarchical, aren't quite. There is no web standard
concerning a notion of "directories" so it would be up to the
developer to implement a such an abstraction.
Trying to understand the problem: Does JSDoc-Toolkit support searching
multiple paths for Javascript files being processed? That is the only
thing I can think of that would make directory listing even slightly
relevant :\
> 3. It is useless. I have not yet heard a use-case where the
> documentation needed to be regenerated every time someone looked at
> it. API docs are simply not that dynamic. More likely the docs change
> only very rarely, it's a wasteful design to require the exact same
> output be regenerated over and over again.
When writing Jazzdocs, I considered generating a great deal at
run-time so that the user could tweak the processing options. I wanted
the formatting used in our comment-captive documentation strings to be
beautiful when viewed as fixed-width text or when translated into rich
text by Jazzdocs, therefore I had to give up some formatting methods
that would have been easier to parse, like inline HTML.
Given that I try to recognize a *lot* of different kinds of ad-hoc
text formatting (even tables!) it's not quite as uncommon as I'd like
that a doc string gets formatted poorly. I think it would serve a good
purpose for the viewer to be able to tweak the settings, or even get a
live doc string editor TEXTAREA, and rapidly view the ramifications of
tweaks to quickly arrive at a pleasing display.
Donny Viszneki
<donny.v...@gmail.com> wrote:
> When writing Jazzdocs, I considered generating a great deal at
> run-time so that the user could tweak the processing options. I wanted
"at run-time..."
*sigh*
By that I mean, of course, at view-in-your-browser time, not at
process-source-code-into-browser-viewable-file(s) time.
Michael Mathews
> Wed, Dec 30, 2009 at 7:19 AM, Michael Mathews <mic...@gmail.com> wrote:
>> 1. It is very slow and very memory-hungry. With big source files it is
>> essentially impossible -- as the browser starts alerting about an
>> "unresponsive script."
>
> If a documentation generator is resigned to doing *anything* during
> the initial pass, it could very easily be paring down the source files
> to their doc strings and rolling them up into a browser-viewable
> documentation file.
I don't understand how that statement is an answer to my observation
of a fact. It is simply true that if you are asking about a way to
generate API docs from source code using *only* the browser, then
performance will be a major issue. If you're not asking about that
then you can ignore my statement.
>
>> 2. JavaScript in the browser is crippled by an inability to get a
>> directory listing. This is on purpose, the browser actively wants to
>> prevent you from doing that. If you cannot get a directory listing,
>> which you can't, then that means the user must provide the exact name
>> of every source file. In large projects this is unwieldy.
>
> I don't think this is true. For one thing, web browsers are not
> designed with an intent to prevent you (user or developer) from
> accessing a "directory listing." Web browsers deal with URLs, which
> despite looking hierarchical, aren't quite. There is no web standard
> concerning a notion of "directories" so it would be up to the
> developer to implement a such an abstraction.
This is semantics. Whether it is the browser or the webserver, or, in
the case of a file:// URL (which is what I was thinking of), the
browser and/or the OS, preventing you from getting a list of filenames
in a directory, the effect is that in a client-side script you can not
rely on the ability to get a list of filenames from a directory.
Obviously you can configure something on the server to provide a list
of filenames and do so in a format that can be reliably parsed by your
client-side script, but that violates the premise of your original
question, as I understood it, of how to implement a client-side only
solution.
>
> Trying to understand the problem: Does JSDoc-Toolkit support searching
> multiple paths for Javascript files being processed? That is the only
> thing I can think of that would make directory listing even slightly
> relevant :\
Sorry, I was under the impression that you'd used JsDoc Toolkit enough
to know that yes it does support recursive directory searches for
source files. It would be very inconvenient if it didn't. Read the
wiki docs if you want to learn more about the basics of JsDoc Toolkit.
>
>> 3. It is useless. I have not yet heard a use-case where the
>> documentation needed to be regenerated every time someone looked at
>> it. API docs are simply not that dynamic. More likely the docs change
>> only very rarely, it's a wasteful design to require the exact same
>> output be regenerated over and over again.
>
> When writing Jazzdocs, I considered generating a great deal at
> run-time so that the user could tweak the processing options. I wanted
> the formatting used in our comment-captive documentation strings to be
> beautiful when viewed as fixed-width text or when translated into rich
> text by Jazzdocs, therefore I had to give up some formatting methods
> that would have been easier to parse, like inline HTML.
>
> Given that I try to recognize a *lot* of different kinds of ad-hoc
> text formatting (even tables!) it's not quite as uncommon as I'd like
> that a doc string gets formatted poorly. I think it would serve a good
> purpose for the viewer to be able to tweak the settings, or even get a
> live doc string editor TEXTAREA, and rapidly view the ramifications of
> tweaks to quickly arrive at a pleasing display.
I don't really understand much of that I'm afraid. Here's my advice...
If you want to parse the documentation into a client-side usable
format (like JSON) using rhino *and then* use a client-side script to
display it prettily, that makes some sense and can already be done in
JsDoc Toolkit (I can provide live examples).
If you want some sort of live documentation generator you will either
have to pare down the features available or suffer the performance
issues I've already described (based on my having actually tried to do
this in the past).
On the other hand, if you want to disagree with me on this then I
applaud your independent spirit and look forward to seeing your
working solution when it is finally implemented. I can't offer much
more advice on how to do it however.
> --
> http://codebad.com/
LitenDavid
Michael, Donny, thank you for your answers. I realize that in the
common use case for JSDoc, on-the-fly generation is impractical and
not really necessary. My use case is rather peripheral; I am to teach
an advanced JavaScript class where the students will create a small
singleton application, and for various silly reasons I needed to
create a "portable" developing environment for them. This meant making
a small JS library that gives you tabs showing unit tests, JSLint
results, data printouts, a console, etc, meaning that you can work on
IE6 and Notepad if forced.
I'm now trying to add a tab showing the documentation of the
application. I realize it is not the most relevant of ideas, but I
want my students to be forced to document as they go, and maybe even
use it as a resource. Having it under their noses when they debug
might help them get into the right frame of mind, I naively hope..
The practicality won't be a problem - the scripts to be parsed will be
rather small, and more often than not they are limited to a single
file. My confusion came from not understanding whether or not this was
in fact a feature of JSDoc. When I snooped through the JS source it
seemed to be capable of doing what I needed, and I think I've seen
some reference to this in the past (probably in the former version you
mentioned).
So, if I understand you correctly - what I need (a JS function that
parses the documentation of a given file into JSON or whatnot) is not
currently supported in JSDoc, but I might have some luck making it
happen through hotwiring the internals?
Michael Mathews
in-browser, documentation generator, despite all my discouragements,
you would really have to start from scratch. There may be parts of
JsDoc Toolkit you can borrow from, but there are faster (and less
feature-complete) ways to do something similar. For example you could
use simple regular expressions. Such an approach would fail on tricky
cases like the following:
var aString = "/**not the start of a doc comment";
var aRegex = /not the end of a doc comment*/;
JsDoc Toolkit takes a more difficult approach, by tokenizing the
entire source file, one character at a time, keeping track of string
starts and string ends, regex starts and regex ends, etc. This is
slower, but also less likely to be wrong.
This tokenizing process is actually browser independent (it does not
use any browser-provided objects) so that could borrowed, as long as
you were dealing with very small source files. I even us that same
process to generate syntax hilited source files in the generated docs.
Once you have a token stream you must then walk it to find the doc
comments and the code those comments describe. This is also quite a
heavy-duty job.
Finally the symbol tree (from the previous steps) needs to be
formatted into HTML, yet another lot of hard work.
Still another issue to deal with is the fact that Rhino supports many
(nice) JavaScript features which older, less capable (cough IE, cough)
browsers don't support. So you may find that some downgrading of code
will be required.
If I can have one more try at dissuading you: a much more useful
project might be to get JsDoc Toolkit to run from within an IDE like
Eclipse or Aptana. It would then be possible to press a keystroke to
have the doc generation initiated. I don't personally use (or even
like) IDEs so I wouldn't provide much enthusiasm myself, but I think
many people do use them and would appreciate your work.
Eric Berry
If I can have one more try at dissuading you: a much more useful
project might be to get JsDoc Toolkit to run from within an IDE like
Eclipse or Aptana. It would then be possible to press a keystroke to
have the doc generation initiated. I don't personally use (or even
like) IDEs so I wouldn't provide much enthusiasm myself, but I think
many people do use them and would appreciate your work.
Or a "smart" text editor even - like jEdit. If you are willing to introduce a small build process to your project (which I personally encourage), you could easily set it up to execute JsDoc Toolkit (or the JsDoc Toolkit Ant plugin) when a js file is saved. Then a simple refresh of the browser would show the updated documentation.
--
Learn from the past. Live in the present. Plan for the future.
11101000
http://www.townsfolkdesigns.com/blogs/elberry
Donny Viszneki
needs seem to me to be that a student on any Windows computer can work
without installing anything. Hence the emphasis on a web client and an
outdated browser.
> --
>
> You received this message because you are subscribed to the Google Groups
> "JsDoc Toolkit Users" group.
> To post to this group, send email to jsd...@googlegroups.com.
> To unsubscribe from this group, send email to
> jsdoc-2+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/jsdoc-2?hl=en.
>
Michael Mathews
it could be done) but probably of very limited use generally. Another
possibility is to run it as a remote web service.
Donny Viszneki
Ryan Zielke
Ryan
Osher El-Netanany
Michael Mathews
Brett Zamir
1) If you wanted to do the easy way (well, at least potentially the easy
way) and use Java, Firefox does support LiveConnect (newly reborn,
including for use with privileges by Firefox extensions, as of around
Java 1.6v12), now using solely cross-browser code internally, allowing
you to use JavaScript as used in basic Firefox interface interactions to
communicate with a Java plug-in to make Java calls. But Java and the
plug-in must both be installed for users to be able to use the extension.
2) If you do decide to write the parser in JavaScript, if you are
feeling brave, you might try ANTLR (which even has a JavaScript version)
and see about feeding it an EBNF for ECMAScript
(http://www.antlr.org/grammar/1153976512034/ecmascriptA3.g ). There is
also the application ANTLRWorks which might help, though you might need
to adapt the EBNF formatting--if it is even possible. I tried it briefly
and wasn't able to generate code. I think there is a real need for a
generic parser of JavaScript (or ideally other languages too) written in
JavaScript (there's http://jslint.com for JavaScript which might be
adapted for generic use on JavaScript only but after a little searching,
I couldn't find anything besides the author's copyright, despite claims
elsewhere that it was under BSD).
If you do decide to go this route, you might also find some interest by
the (Mozilla-affiliated) developers of Bespin, a purely
JavaScript(+canvas)-based IDE.
best wishes,
Brett
Louis Santillan
JSCC (works with SM, Rhino, V8, JS/WSH) http://jscc.jmksf.com/
Kouprey http://www.deadpixi.com/Deadpixi.COM/Kouprey.html
OMeta http://www.tinlizzie.org/ometa/
-Louis