Last Weekends Adventure

[pedz]

I spent last weekend adding Spry to one of my web sites. I'm using
Ruby On Rails and prototype with perhaps a bit of script.aculo.us in
this web site.

What attracted me to Spry is the region and repeat features. Lets us
a table as an example. The basic idea is the page that is sent out
has only the header and a single row. You have to put a div around
the table to define the region. And the data row in the table has a
"repeat" attribute in it. Somewhere you hook a data souce up to the
region. In my case, I went back to my Rails server with a json
request which sprewed out the selected items in a javascript array,
each element being an item.

The initial row has macros like {name} that get expanded to fields in
the items. In my particular case, while a row only has 8 elements
across, there are a lot of things in each td like hidden drop down
lists, links, etc. The work to create a json row is small compared to
the work to create the equivalent html row. What I see attractive
with Spry is it moves the work of creating the html to the browser and
offloads the server. There is just no reason for the server to be
doing that work.

My belief is that ultimately, moving the creation of the html to the
browser is the way of the future. The server serves data. The view
processing will eventually be done in the browser. There is no need
for the server to do hard labor for the view processing.

But I found Spry to be less than stellar. The main biggest issue is
when I have a page with 1800 rows, it takes my Firefox about a half a
minute of dead time to create the page and render it -- during that
time, my whole laptop is dead pretty much with the dreaded spinning
beachball. This half minute is divided into two parts I'm sure. Part
of the time is Spry running creating the 1800 rows. I'll use the term
"fill out" to refer to that process. Then there is the time for
Firefox to actually render the page.

I also found the package to not be very flexible and configurable. It
has a sort feature but it has far fewer features than the tablesort.js
file found on the net.

So, it put me back to "can I do this myself" mode.

A small side trace: I have seen the Rails cast where Ryan creates an
"endless page". My issue with that approach is that instead of one
query, he makes N queries -- one for each page. That seems like it
would add load to the server while I'm trying to take load away from
the server.

It seems like I could leverage prototypes template feature for most of
the work. The main thing I would do different is make one request
(like Spry) but I would only render the first page or two of rows.
Then as the user scrolled, more data would be filled out hopefully
before the user actually got to it.

There are lots of problems to all this. How do you make the scroll
bar act "right"? What do you do when the user does a search within
the page? How do you handle a sort? As you add rows, the width of
the columns may grow which is not expected. etc.

But the essential concept is that all the data would be held as a
javascript object and then the page filled out as it was needed.

So, the question is, has anyone done something like this starting with
Prototype? I sure don't want to recreate the wheel here. Is there
any suggestion of other work that I can look at and build from? Is
there any interest by any of the readers to help out?

Thank you,
pedz

[emitch]

This may be an overly simplistic first step suggestion, but I've been
in a similar situation where I needed to "render" thousands of table
rows in the browser via a JSON response from the server. I found that
by calling a render row function/method (somewhat asynchronously on
the client) via a setTimeout per row when processing the data gave
relief to the browser while looping over the data, so that it wasn't
just hanging there waiting to complete the whole table/loop. That
looping and such can be split into groups for pagination etc. The page
was built row by row and the browser was responsive during the load.
I'm not familiar with Spry or your specific situation, but maybe this
could help if you haven't already tried it. Hopefully others will
have some suggestions as well. I'm curious about other solutions as
well.

[T.J. Crowder]

Hi,

I don't know the Spry framework at all, but if it's building the table
rows by creating elements through the DOM API (document.createElement,
HTMLElement#appendChild, etc.) or Prototype's wrappers for those [new
Element(...)], that may be part of the problem.

It may be somewhat counter-intuitive, but creating a TR element,
adding a bunch of TD elements to it, and then appending that to a
TBODY, all using the DOM API, is much, much slower on most browsers
than building a string of tags in memory and then adding them via the
non-standard (but common) innerHTML property. I figure this is
because parsing HTML strings and turning them into displayed elements
is one of the most speed-critical things that browsers do, and so
their native handling of it is extremely optimized, working directly
with their internals.

I did an exercise a while back (here[1]) comparing DOM methods vs.
Prototype's wrappers for DOM methods vs. concatenating a string and
setting innerHTML. The last method, innerHTML, wins on every browser
I tried, hands down, usually by at least one order of magnitude.
(Prototype's wrappers aren't very costly, but going through the DOM
methods is, and so they inherit that cost.) The smallest discrepancy,
IIRC, is on Chrome ("only" about four times faster rathe than 10x or
100x faster) -- but then, Chrome is freakishly fast at nearly
everything. :-)

So when I need to do this sort of thing, I build up a string (perhaps
using Prototype's template stuff) and then apply it via Prototype's
Element#update method[2] (which uses innerHTML under-the-covers but
also handles some edge cases for you).

If you're doing 1,800 rows, even with innerHTML, I really like
emitch's suggestion of breaking it into chunks and periodically giving
the browser a chance to display what's been done so far. You can use
setTimeout, or Prototype's convenient wrappers for it Function#defer
[3] and Function#delay[4].

[1] http://pastie.org/521342
[2] http://prototypejs.org/api/element/update
[3] http://prototypejs.org/api/function/defer
[4] http://prototypejs.org/api/function/delay

HTH,
--
T.J. Crowder
tj / crowder software / com
Independent Software Engineer, consulting services available

[pedz]

On Jun 23, 4:31 am, "T.J. Crowder" <t...@crowdersoftware.com> wrote:
> I did an exercise a while back (here[1]) comparing DOM methods vs.
> Prototype's wrappers for DOM methods vs. concatenating a string and
> setting innerHTML.  The last method, innerHTML, wins on every browser
> I tried, hands down, usually by at least one order of magnitude.

Holy @#$%!!! I played with your page in Firefox, set rows to 1800.
The DOM says it took 16 seconds but really it took longer because
there was another 15 seconds of spinning beach ball before I saw the
page. The html took 300ms and displayed instantly.

> If you're doing 1,800 rows, even with innerHTML, I really like
> emitch's suggestion of breaking it into chunks and periodically giving
> the browser a chance to display what's been done so far.  You can use
> setTimeout, or Prototype's convenient wrappers for it Function#defer
> [3] and Function#delay[4].

Yes, I had considered that before although I didn't know which
interfaces to use. Somehow I need to tell the browser to "relax"...
no one is going to view 1800 rows immediately. I didn't know if doing
it via a timer would achieve that effect or not. And, I don't know
if / how to do that with Spry.

But, given what I just learned, I think I want to ease away from
Spry. One problem is it has not changed in 18+ months. There is a
lot of talk asking "Is it dead?". Adobe says "No" but I just don't
see how they can keep up with projects like prototype and JQuery that
are open development. And, more importantly, it is hugh and
complicated and its not "mine". I feel like prototype is so small and
relatively simple, I can "own" the code (i.e. really understand it and
work with it instead of on top of or in spite of it).

I hope others reply too. But, thank you very much to T.J. and emitch.