docs / meta-docs / pythonic docs

[Irakli Gozalishvili]

Hi,

I was experimenting some ideas that have been inspired by the way of documenting python code and the trick that used by ecmascript 5 to enable "strict mode";

Before diving into the details I would like to make short intro:

While hacking with scripting languages I found myself heavily working with a shell in order to test stuff etc.. In many cases I would be so much happier if I could just run foo.doc() and get an output with the documentation of it :) Sound like a dream, would've safe a lot of time of finding right file and line in the source, which might be helpful. Actually while looking at the source in trying to get a picture it's quite easy to get a negative effect, like loosing focus of a tusk you was performing and trying to get a bigger picture. Of course it's good in the long term but sometimes you need things to be done and there is no long term scenario. Messing with a Narwhal source was kind of complex thing for me as well and many times I fond myself looking into completely different parts of the framework, without any idea what I was looking for. On the other hand it would've be so cool to have interactive shell not only for the live scripting, but also for live doc lookups. 

Well basically that's small overview of what I had in mind while starting experimenting with some ideas to get more or less what I was missing (hopefully you guys share this feelings with me :)  Anyway result is quite nice IMO, it better to show then explain:

gozala@ijarti:~/Projects/meta-doc  (master)$ narwhal

Rhino 1.7 release 3 PRERELEASE 2009 04 05

js> require("meta-doc/toolkit").doc()

toolkit module

provides utilities for getting a live docs

@type module

js> typeof require("/Users/gozala/Projects/meta-doc/examples/functions.js").example

function

js> require("/Users/gozala/Projects/meta-doc/examples/functions.js").example.doc()

This is a function foo which is supposed to do something.

But the best thing is that it has nice documentation in it.

@param {String} a                   first argument

@param {String} b                   second argument

@returns {String}                   Concatinated string literal

js> require.doc()

Not documented

js> 

Of course source is also available around: http://gist.github.com/219927

Basically idea is in creation of an array with a single multiline string element in it in the beginning of the function or module. Array is not saved in a variable so it's not supposed to take any memory. Technique is compatible with rhino, tracemonkey, squirrelfish and V8. Have not tried with ie but guess none cares :) 

Looking forward to get some feedback, of course gist is very simple and it can be improved a lot. I have bit more complicated implementations where the doc comments are merged with the data from function definition so that number and argument names are generated even if they are not specified. 

 

--
Irakli Gozalishvili
Web: http://rfobic.wordpress.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Nathan Stott]

I like it

[Kevin Dangoor]

On Tue, Oct 27, 2009 at 5:14 PM, Irakli Gozalishvili <rfo...@gmail.com> wrote:

gozala@ijarti:~/Projects/meta-doc  (master)$ narwhal

Rhino 1.7 release 3 PRERELEASE 2009 04 05

js> require("meta-doc/toolkit").doc()

toolkit module

provides utilities for getting a live docs

@type module

js> typeof require("/Users/gozala/Projects/meta-doc/examples/functions.js").example

function

js> require("/Users/gozala/Projects/meta-doc/examples/functions.js").example.doc()

This is a function foo which is supposed to do something.

But the best thing is that it has nice documentation in it.

@param {String} a                   first argument

@param {String} b                   second argument

@returns {String}                   Concatinated string literal

js> require.doc()

Not documented

js> 

Of course source is also available around: http://gist.github.com/219927

That's really cool, Irakli. I like how simple it is!

Kevin

--
Kevin Dangoor

work: http://labs.mozilla.com/
email: k...@blazingthings.com
blog: http://www.BlueSkyOnMars.com

[George Moschovitis]

Interesting,

too bad, JavaScript multi line strings suck :(

-g.

--
blog.gmosx.com

[Hannes Wallnoefer]

2009/10/28 George Moschovitis <george.mo...@gmail.com>:

> Interesting,
> too bad, JavaScript multi line strings suck :(

On the #helma IRC channel we've come up with a twist that fixes this,
using an E4X XML literal:

http://helma.pastebin.com/m7c2e0258
http://helma.pastebin.com/m58060200

This makes multiline comments easy. The only downside is you have to
be xml-wellformed and escape '<', '>' etc (and you need E4X, of
course).

Hannes

[Ash Berlin]

On 28 Oct 2009, at 07:50, Hannes Wallnoefer wrote:

2009/10/28 George Moschovitis <george.mo...@gmail.com>:

Interesting,

too bad, JavaScript multi line strings suck :(

On the #helma IRC channel we've come up with a twist that fixes this,
using an E4X XML literal:

http://helma.pastebin.com/m7c2e0258
http://helma.pastebin.com/m58060200

This makes multiline comments easy. The only downside is you have to
be xml-wellformed and escape '<', '>' etc (and you need E4X, of
course).

Hannes

And you need to escape { somehow to - since <docs>{a}</doc> would attempt to interpolate a variable 'a'.

Really cool idea tho.

[Irakli Gozalishvili]

And platform should support E4X :(

For me it's not an option as I would like to stick to the same way of documenting stuff both on server and on client (Chromium, Webkit). But I do agree E4X option would've be is great!! 

 
--
Irakli Gozalishvili
Web: http://rfobic.wordpress.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Kris Zyp]

As an alternative idea, the approach taken in Persevere and also
supported in Dojo is providing metadata through JSON schema data
structures. Granted this more focused on class structures, rather than
individual functions, but for class style structures, this provides a
very comprehensive and portable approach, IMO, with a well-defined
mechanism for discovering information about methods. There actually
isn't really any library or helper code needed, it is just a convention,
but it basically just implies that you create strucutures such that one
could do:
MyClass.methods.foo.description -> the description of method foo
MyClass.methods.foo.parameters[0].type -> the expected type of method
foo's first parameter
MyClass.properties.bar.type -> the expected type of property bar
And you can even do things like leveraging this information to add
optional type checking:
http://www.sitepen.com/blog/2009/06/23/unobtrusive-javascript-typing-via-json-schema-interfaces/
This is all pretty language agnostic, so the definitions are very
portable as well.

And another idea, in terms of actually being able to do
someFunction.doc()... I believe Rhino actually supports parsing JSDocs
into the AST. I have suggested before [1] that being able to optionally
access Rhino's great AST API from a function object would provide some
pretty impressive introspective capabilities (like implementing a nice
.doc() method), especially in combination with the JSDoc support.
Obviously using something like JSDoc is much more elegant than having an
array instantiation (or worse yet, E4X code execution) in all your
functions (probably not even feasible in hot functions)
[1]
http://markmail.org/message/qzieujftptnyxvj3#query:+page:1+mid:6xodxibbqfx6epmw+state:results
Kris

[Irakli Gozalishvili]

On Wed, Oct 28, 2009 at 16:07, Kris Zyp <kri...@gmail.com> wrote:

As an alternative idea, the approach taken in Persevere and also
supported in Dojo is providing metadata through JSON schema data
structures. Granted this more focused on class structures, rather than
individual functions, but for class style structures, this provides a
very comprehensive and portable approach, IMO, with a well-defined
mechanism for discovering information about methods. There actually
isn't really any library or helper code needed, it is just a convention,
but it basically just implies that you create strucutures such that one
could do:
MyClass.methods.foo.description -> the description of method foo
MyClass.methods.foo.parameters[0].type -> the expected type of method
foo's first parameter
MyClass.properties.bar.type -> the expected type of property bar
And you can even do things like leveraging this information to add
optional type checking:
http://www.sitepen.com/blog/2009/06/23/unobtrusive-javascript-typing-via-json-schema-interfaces/
This is all pretty language agnostic, so the definitions are very
portable as well.

And another idea, in terms of actually being able to do
someFunction.doc()... I believe Rhino actually supports parsing JSDocs
into the AST.

From I discovered when I was looking at rhino AST it it doesn't really parses jsdocs it just includes comments in a source AST. Besides show stopper for me again is a dependency on rhino, but what I had in mind is that doc function implementation can be platform specific and in case of rhino indeed it can incorporate with rhino AST.  

[George Moschovitis]

On Wed, Oct 28, 2009 at 9:50 AM, Hannes Wallnoefer <han...@gmail.com> wrote:

2009/10/28 George Moschovitis <george.mo...@gmail.com>:

> Interesting,
> too bad, JavaScript multi line strings suck :(

On the #helma IRC channel we've come up with a twist that fixes this,
using an E4X XML literal:

http://helma.pastebin.com/m7c2e0258
http://helma.pastebin.com/m58060200

This makes multiline comments easy. The only downside is you have to
be xml-wellformed and escape '<', '>' etc (and you need E4X, of
course).

Even more interesting ;-)

thanks,

-g.

--
blog.gmosx.com

[Hannes Wallnoefer]

2009/10/28 Kris Zyp <kri...@gmail.com>:

>
> And another idea, in terms of actually being able to do
> someFunction.doc()... I believe Rhino actually supports parsing JSDocs
> into the AST. I have suggested before [1] that being able to optionally
> access Rhino's great AST API from a function object  would provide some
> pretty impressive introspective capabilities (like implementing a nice
> .doc() method), especially in combination with the JSDoc support.
> Obviously using something like JSDoc is much more elegant than having an
> array instantiation (or worse yet, E4X code execution) in all your
> functions (probably not even feasible in hot functions)

I am pretty much done implementing JSDoc parsing based on Rhino's AST.

http://github.com/hns/helma-ng/blob/master/modules/helma/jsdoc.js
http://helma-ng.appspot.com/jsdoc/

The code recognizes various CommonJS exporting patterns. The problem
with AST-based JSDoc is that it is sometimes hard to associate
comments with functions. For example, if you do "exports.foo =
function..." the comment is associated with the assignment, and with
"var foo = exports.foo = function..." it is associated with the var
statement.

Also, comments are lost when the AST is transposed into bytecode, so
what I do is to parse each module specially for JSDocs generation, and
I'm not sure if it would be possible to make jsdoc available at
runtime. Besides, I don't think parsing JSDoc comments would be much
cheaper than parsing E4X XML literals, but I could easily be wrong
about that.

Hannes

[Irakli Gozalishvili]

Thanks to everyone for all the feedback!!

Kris, Tom I was hoping to get some feedback from your side as well cause actually I think it would be really nice to get this into narwhal, which can be disabled by default, but with possibility to enable it in debug mode for example. Several different options had been mentioned here as well, but unfortunately non of those options are cross platform / cross engine, even though they look much nicer :( I would like to also make couple of points here.

• Implementation of the doc method is an implementation detail which I think can be engine specific, so that a lot's of fancy staff can be added along with a simple comment dumping. But I do thing the most important is to have agree on a way of putting jsdoc comments.
• I do hate escaping line breaks :) I do share your emotions, but I would happily compromise that in order to have all the sugar of live docs. I started using this approach widely on my code and have to admit that it feels way more natural and nicer then I was expecting it to be. BTW if you can tweak your editor in a way that escaped chars are semi transparent the level of annoyance by it is reduced to a minimum. (I'm willing to implement this for bespin BTW.)
• JSON Schema options is interesting, but I thinks it lucks of simplicity.  

I'm thinking to propose this on commonjs mailing list as well cause having common way of documenting the code seems to be a good idea. Actually it was my initial idea.

P.S.: On squirrelfish & v8 to string of the function includes comments as well, it would've be perfect to use comments inside of function body instead but unfortunately nor rhino nor xulrunner doesn't seem to share opinions on that. It might be an idea to incorporate with Ecmascript working group on this, if the comment inclusion in toString() will a standard... 

   

--
Irakli Gozalishvili
Web: http://rfobic.wordpress.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Irakli Gozalishvili]

Hi I started this thread on narwhal mailing list but think it makes a lot of sense to move discussions here as the main idea is to have a common way of adding docs to our js code. Please find details of the discussion below!!

  

Regards!!

----

[Irakli Gozalishvili]

I would happily write proposal on wiki but I just like to get feedback before

--
Irakli Gozalishvili
Web: http://rfobic.wordpress.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

[Kris Kowal]

On Thu, Oct 29, 2009 at 1:48 PM, Irakli Gozalishvili <rfo...@gmail.com> wrote:
> Thanks to everyone for all the feedback!!
> Kris, Tom I was hoping to get some feedback from your side as well cause
> actually I think it would be really nice to get this into narwhal, which can
> be disabled by default, but with possibility to enable it in debug mode for
> example. Several different options had been mentioned here as well,
> but unfortunately non of those options are cross platform / cross engine,
> even though they look much nicer :( I would like to also make couple of
> points here.

I don't think that run-time introspection is critical; the key is that
making documentation *must* be easy or it's just not going to happen,
whether we bless it or not. I think the jsdoc comments are probably
the way to go for the time being. It's even possible that we could
add optional hooks to the module loader that would scrape jsdocs
dynamically and monkey-patch the module's exports on completion.

Kris Kowal

[Ryan Dahl]

It would definitely be nice to have runtime accessible javascript
docs. It's too bad we don't have access to the AST (in general) and
could just pull out that information. Something which is uglier but
doesn't require parsing the source would be preferable to me:

module.doc = "Some simple math functions.";

exports.add = function (a, b) {
return a + b;
};
exports.add.doc = "Adds two objects.";

exports.subtract = function (a, b) {
return a - b;
};
exports.subtract.doc = "Subtracts two objects.";

Or maybe:

module.addDoc("Some simple math functions.");

module.addDoc("add", "Adds two objects.");
exports.add = function (a, b) {
return a + b;
};

module.addDoc("subtract", "Subtracts two objects.");
exports.subtract = function (a, b) {
return a - b;
};

Ryan

[Ryan Dahl]

On Mon, Nov 2, 2009 at 12:07 PM, Ryan Dahl <coldre...@gmail.com> wrote:
> It would definitely be nice to have runtime accessible javascript
> docs. It's too bad we don't have access to the AST (in general) and
> could just pull out that information. Something which is uglier but
> doesn't require parsing the source would be preferable to me

Ack, Sorry, I thought this was the CommonJS forum!

[Christoph Dorn]

Kris Kowal wrote:
> I don't think that run-time introspection is critical; the key is that
> making documentation *must* be easy or it's just not going to happen,
> whether we bless it or not. I think the jsdoc comments are probably
> the way to go for the time being. It's even possible that we could
> add optional hooks to the module loader that would scrape jsdocs
> dynamically and monkey-patch the module's exports on completion.

+1 for jsdoc - people know how to write compliant docs

Christoph