Collaboration

[Tobie Langel]

Hi everybody,

There's quite a bunch of us with commit rights now collaborating on
this project.

It would be very helpful if we all liaised on this mailing list just
to make sure we don't step on each other's toes.

I've added a commit hook to pdoc's twitter account, so keeping up to
date on changes should be as easy as following @pdoc.

(I'm also trying to add a commit hook for this Google Group too, but
that has proven harder than expected.)

Thanks for your help!

Best,

Tobie

[Samuel Lebeau]

Hi everyone,

I just pushed a "speed" branch with an experimental (at least in
style) memoizing feature [1].

With this commit, building the whole Prototype documentation takes
only ~45s on my laptop, versus ~8 minutes with the master branch.
This is sort of a hack but it avoids rewriting all
`Documentation::Base` subclasses throwing `@ivar ||= expr` everywhere
and doesn't break a single test.

What do you guys think ?

Regards,
Samuel.

[1] http://github.com/tobie/pdoc/commits/speed

[Tobie Langel]

I say let's be pragmatic go for it.

Tobie

[Andrew Dupont]

Samuel,

FANTASTIC! For me the speed difference is even more dramatic — from ~20 minutes to ~20 seconds.

I had experimented with memoization, but never got results this good.

Cheers,

Andrew

[Samuel Lebeau]

Andrew,

Cool !! What OS/ruby are you using BTW ?
I'm willing to make everything work on all Ruby implementations.
Parsing is slower on MRI 1.9 (from 30s to 50s), probably because of
new encoding abstractions, and renderring simply blocks, apparently
doing nothing.
Replacing ERB by Erubis fixes it so I'm considering doing this change.

Has anyone an explanation for this ERB-related bug?

Best,
Samuel.

[Andrew Dupont]

On May 31, 2009, at 5:43 PM, Samuel Lebeau wrote:
> Cool !! What OS/ruby are you using BTW ?

Ruby 1.8.6 on Leopard. To be more precise, here are the figures from
my last doc build (74 seconds is the largest figure I've seen for the
generation phase since I've been running your optimizations):

Parsing completed in 11.807067 seconds.
Documentation generated in 74.335587 seconds.

> I'm willing to make everything work on all Ruby implementations.
> Parsing is slower on MRI 1.9 (from 30s to 50s), probably because of
> new encoding abstractions, and renderring simply blocks, apparently
> doing nothing.

Before memoization, I found Ruby 1.9 took _far_ longer to generate the
docs, but used only half as much memory. Would you care to look into
how much memory 1.9 uses during doc generation, as compared to 1.8.6?
I might try some other flavors (jRuby, macruby) for curiosity's sake.

> Replacing ERB by Erubis fixes it so I'm considering doing this change.

Lovely – this was another attempted optimization on my part, but I
didn't see much improvement, most likely because the bottleneck was
previously elsewhere. You might consider implementing it the way we've
done the Markdown library loading — we prefer RDiscount, but fall back
to BlueCloth. [1]

Cheers,
Andrew

[1] http://github.com/tobie/pdoc/blob/master/lib/pdoc/generators/html/bluecloth_extension.rb#L2

[Andrew Dupont]

In jRuby 1.2:

Parsing completed in 23.57 seconds.
Documentation generated in 29.974 seconds.

Brilliant.

[Tobie Langel]

You guys rock!