Autorendering list and a scalar doesn't work

[Catch22]

I am trying out PURE and came across a simple example that does not
work. I am trying to autorender this:
{people: [{name: "John"}], day: "Sunday"}

where the first field is a list and the second field is a simple
scalar. I get this exception on Firefox 3:

fns[i] is not a function
concatenator()()pure.js (line 167)
compile()()pure.js (line 580)
autoRender()pure.js (line 602)
main()test.html (line 8)
onload()1 (line 2)
[Break on this error] fnVal = fns[i]( ctxt );\n

You can see this sample at http://proto.kudang.com/pure/test.html

I like the simplicity of PURE compared to other javacript/html
template engines.
I am trying to use PURE as the template engine for a node.js app and I
really hope some one can tell me what I am doing wrong.

[Mic (BeeBole)]

If you reverse the order of the <p>'s it works:

<div id="template">
<p>
Today is <span class="day"></span>
</p>
<p>
Here is a list of people:
<div class="people name"></div>
</p>
</div>

I'll check the autoRender code to see where the problem comes from.
Thanks for reporting the problem.
Cheers,

On Apr 3, 3:22 pm, Catch22 <kud...@gmail.com> wrote:
> I am trying out PURE and came across a simple example that does not
> work. I am trying to autorender this:
> {people: [{name: "John"}], day: "Sunday"}
>
> where the first field is a list and the second field is a simple
> scalar. I get this exception on Firefox 3:
>
> fns[i] is not a function
> concatenator()()pure.js (line 167)
> compile()()pure.js (line 580)
> autoRender()pure.js (line 602)
> main()test.html (line 8)
> onload()1 (line 2)
> [Break on this error] fnVal = fns[i]( ctxt );\n
>

> You can see this sample athttp://proto.kudang.com/pure/test.html

[Catch22]

Thanks Mic. If it helps, I realized that adding an explicit directive
also solves the problem.

[Catch22]

Interestingly, I tried adding the scalar div both before and after the
list element like this:

<p>
Today is <span class="day"></span>
</p>

<p>
Here is a list of people:
<div class="people name"></div>
</p>

<p>
Today is <span class="day"></span>
</p>

And something really crazy happened - somehow the list div got
embedded inside the scalar div!
(I have updated http://www.google.com/url?sa=D&q=http://proto.kudang.com/pure/test.html
with the new test)

Thanks,
/Chandan

On Apr 5, 5:34 pm, "Mic (BeeBole)" <tch...@gmail.com> wrote:

[Mic (BeeBole)]

Which revision are you using?
There was a bug few months ago in the outerHTML using an XML
serializer instead of DOM.
Causing the DIV's to nest.

I see in the test link, you are using the revision 2.25 while we are
at 2.70.
Could you try with the last version of pure.js:
https://github.com/pure/pure/raw/master/libs/pure.js ?
You can always take safely the last version as it runs our web app ans
is well tested.

[Catch22]

I switched over to the latest pure.js from https://github.com/pure/pure/raw/master/libs/pure.js
and the problem(s) still exist.

I have created 2 versions for easy testing:
http://proto.kudang.com/pure/test.html : The list gets nested into the
scalar element

http://proto.kudang.com/pure/test1.html : Auto render does not work
when scalar is *before* list

http://proto.kudang.com/pure/test2.html : Auto render works fine when
the scalar and list elements are swapped.

Thanks,
/CK

[Mic (BeeBole)]

Thanks for the test cases. It will be easier to debug.
I don't use autoRender in our app, that's why it's the poor method of
the lib.

Did you know you can complement autoRender with directives?
This is quite theoretical as I think it was never used in a production
app.

On Apr 8, 9:44 am, Catch22 <kud...@gmail.com> wrote:
> I switched over to the latest pure.js fromhttps://github.com/pure/pure/raw/master/libs/pure.js

[Catch22]

It would probably help if I explained what I am trying to do with
Pure. Basically I am trying to implement a very simple MVC framework
for NodeJS without using any 3rd party "middleware". This would be
very similar to how CakePHP or Symfony are structured in theory.

So when a 'Controller' finishes processing, it would set variables
that would be available for the view. The view file is a static HTML
which would be processed via Pure to embed the view variables in
appropriate places in in the HTML file.

I did realise that everything works if I add explicit directives, but
the problem is that in each controller I need to add these directives
that essentially tells Pure to do exactly what it would have done via
autorender.

While writing a large MVC application, it is cumbersome to have to add
directives to each controller and preferable if the controller
developer does not have to worry about how the variables get embedded
in the view. This is where I was hoping that the autoRender would help
me.

I did try using other JS templates, but they all use their own special
syntax instead of standard HTML, and Pure is the only template that I
came across that works with standard HTML with no other magic. I would
like to know if you plan on supporting autoRender functionality in
future ? I will be glad to help you debug the issues as I come across
if that helps.

[Mic (BeeBole)]

Sure, I want autoRender to work as expected.

It is just bad timing right now, I'm very busy building a new web site
for our web app.
I check your test cases sometimes but didn't find yet why it breaks.
Feel free to have a look and debug it if you want.

Do you use JSDom?
If you have already some sources somewhere of your MVC, I'm interested
to see it.
Cheers,

[Catch22]

Okay, I did some debugging I figured out that in the compiler()
function, the getAutoNodes() is returning the nodes correctly, however
after going through the first loop that calls setsig() (which calls
compiler recursively in the case of a loop), we end up with
'parts' (at the end of the function) that is one more in length than
the fns array of functions.

My guess is that in the recursive call, the setsig() is making changes
to a node that affects the parent call. I can debug further, but I had
a better idea. How about creating a new function that takes the list
of auto nodes and automatically generates directives from that ?

So I created a function called autoDirectives() that does exactly
this, and it seems to work! I am including the patch to pure.js that
accomplishes this. You can test this by calling autoRender2() instead
of autoRender(). I haven't done a thorough test, but the cases the
originally failed works fine with this. I am not handling attribute
appending & prepending, and probably not handling recursive loops
well. Let me know what you think.

And yes I am using JSDom and I have some code for the MVC fraework but
its not too polished. I will send it when its in a better condition.

--- a/purest/pure.js Thu Apr 07 15:50:23 2011 +1000
+++ b/purest/pure.js Wed Apr 13 22:05:38 2011 +1000
@@ -8,6 +8,10 @@

Thanks to Rog Peppe for the functional JS jump
revision: 2.70
+
+ Experimental code by Chandan Kudige - automatically
+ generate directives from the auto nodes.
+ autoRender2() can be used to test this.
*/

var $p, pure = $p = function(){
@@ -91,6 +95,7 @@
f.prototype.compile = plugins.compile || compile;
f.prototype.render = plugins.render || render;
f.prototype.autoRender = plugins.autoRender || autoRender;
+ f.prototype.autoRender2 = plugins.autoRender2 || autoRender2;
f.prototype.find = plugins.find || find;

// give the compiler and the error handling to the plugin context
@@ -621,7 +626,53 @@
context = null;
return this;
}
+
+ // compile the template with autoRender2
+ // run the template function on the context argument
+ // return an HTML string
+ function autoRender2(ctxt){
+ var ans = getAutoNodes(this[0], ctxt);
+ directive = autoDirectives(ans)
+ return this.render(ctxt, directive)
+ }

+ function autoDirectives(ans) {
+ var dir = {}
+ // Find the autonodes inside loops
+ var loopNodes = {}
+ var i = ans.length
+ while (i--) {
+ var an = ans[i]
+ if (an.cspec.prop.match(/([^.]+)\./)) {
+ var loopVar = RegExp.$1
+ if (!loopNodes[loopVar])
+ loopNodes[loopVar] = []
+ loopNodes[loopVar].push(an)
+ ans.splice(i,1)
+ }
+ }
+
+ for (var i=0; i<ans.length; i++) {
+ var type = ans[i].cspec.t
+ var prop = ans[i].cspec.prop
+ var sel = ans[i].cspec.sel
+ if (type === 'str') {
+ dir['.' + sel] = prop
+ } else {
+ var loopDirectives = {}
+ for (var ii=0; ii<loopNodes[prop].length; ii++) {
+ var cspec = loopNodes[prop][ii].cspec
+ loopDirectives['.'+cspec.sel] = cspec.prop
+ }
+ var fullprop = prop + '<-' + prop
+ dir['.' + sel] = {}
+ dir['.' + sel][fullprop] = loopDirectives
+ }
+ }
+ return dir
+ }
+
+
function replaceWith(elm, html) {
var ne,
ep = elm.parentNode,

[Mic (BeeBole)]

Changing something instead of fixing it is always good!
That's very interesting. May be this could be generated inside
getAutoNodes directly, and avoid double loops.

[Catch22]

Good idea, I will create a new version with autoDirectives integrated
in getAutoNodes.
BTW, is there a reason for using text manipulation instead of DOM for
plugging in the templates ?
I worry that for large documents this might get slow. Its probably not
too hard to do that without using
jQuery or any of the other libraries, if the goal is to keep it "pure"

[Mic (BeeBole)]

I don't understand. Which text manipulation?

[Catch22]

I was talking about slicing the html and concatenating it back:

// slice the html string at "Sig"
parts = h.split( Sig );
// for each slice add the return string of
...
return concatenator(parts, pfns);

Its not terrible, but I was just curious.

[Mic (BeeBole)]

This is what makes the lib fast.
But you're right in some cases a DOM manipulation could be more
efficient.
Now the question is how to get rules on that.

Or may be this would be something to investigate for performances:
- the signature is not anymore a string injected in the html but
instead the nodes(targeted by directives) are kept as reference
- the new nodes are rendered in a set of document fragments
- the new nodes replace the old nodes in the DOM.

[Mic (BeeBole)]

I was debugging the autoRender case, but then your comment about text
manipulation sparked something... A DOM only version.

I've sent the code at https://github.com/pure/pure/tree/master/v3_preview
This is a mere proof of concept for now(only a basic selector, no libs
integration,...)

The big differences would be:
1) no more DOM->string->DOM (less code)
2) the DOM version of the template would be available at the
compilation and rendering time(available inside a function
directive!!)
3) the compile should not copy anymore the template by default, to
allow a direct change
4) the compiled template does not return a string anymore, but a DOM
node
5) may be less browsers quirks to deal with
6) and finally for your case :) the autorender will be simpler to
implement

The (3) & (4) will probably break existing code(at least our web app),
but they are cleaner.
May be provide a switch to make them work like before.

The main doubt is about the performances.
But I don't see why this should be any slower than today, especially
on bigger templates.
May be loops will need a special care.

Let's see if this works, but it could be an amazing improvement.

On Apr 14, 1:13 am, Catch22 <kud...@gmail.com> wrote:

[Catch22]

Mic, this version looks very promising ! I will play with it a bit
more and if possible do some performance testing and get back to you.
But the way it works now is exactly what I was looking for in the
first place :)

On Apr 16, 12:20 pm, "Mic (BeeBole)" <tch...@gmail.com> wrote:
> I was debugging the autoRender case, but then your comment about text
> manipulation sparked something... A DOM only version.
>

> I've sent the code athttps://github.com/pure/pure/tree/master/v3_preview

[Catch22]

Mic, I played with the V3 version a bit. Looks like the loops are
still not implemented right ?
I wrote some scripts to test the performance of the V3 vs V2 for
substituting 1000 variables in a large
HTML file.

This is what I found on my Mac (using time command):
./runpure.js bigtest.html 1000 v3 17.95s user 0.17s system 89% cpu
20.257 total
./runpure.js bigtest.html 1000 v2 36.03s user 0.24s system 90% cpu
40.289 total

As I expected the DOM based v3 is twice as fast as the string based
implementation.

Further, for comparision I also wrote a function to do the same
substitution using jsdom selectors (uses Sizzle).
./runpure.js bigtest.html 1000 dom 17.84s user 0.16s system 89% cpu
20.150 total

As you can see, the V3 performance is very close to the direct
substitution with negligible overhead!

I think this justifies going with the DOM based implementation.
When the loops are implemented I would like to test that out too.

You can download my scripts from: https://github.com/kudige/nodify/tree/master/templates/puretest
(Requires installing node js and jsdom to run the scripts)

Cheers,
/CK

[Mic (BeeBole)]

The first loop version is at http://github.com/pure/pure

Nested loops need to be fixed, but it should be enough for a speed
test.
It's document fragment for the loop, I quickly tried with zillions of
rows, and it works fast.
I look forward for your test results.

Cheers,

> ...
>
> read more »

[Catch22]

Mic, you were asking me about the MVC framework. I have a sneak peek
here: http://launchjs.com/
I will be uploading working code soon.

Cheers,

On Apr 25, 5:31 am, "Mic (BeeBole)" <tch...@gmail.com> wrote:
> The first loop version is athttp://github.com/pure/pure

> ...
>
> read more »

[Mic (BeeBole)]

Did you see the HTML5 presentation of the googleIO?
http://bit.ly/l2Xt3N from minute 33:20' they start to talk about an in-
browser kind of templating / data binding.
This could inspire you.

Did you note, their example looks to be inspired by PURE recursive
example: http://bit.ly/4JnsBL :)

> ...
>
> read more »

[Jeff Barczewski]

Mic & CK,

I looked at the test which you used to compare v2 and v3 and I think it might not be measuring the appropriate data.

Mic, if I understand things correctly, once you compile a template, you do not need the DOM any longer to generate rendered HTML string, you simply call the function with data and an HTML string pops out.

So if one was going to use this on the server as CK was testing, I would only use a jsdom once for compiling the template, then after that always just render the compiled template.

If you change the tests to use the precompiled function that, then the bigtest with 1000 replacements run in a loop 1000 times here are the results:

v2     494ms (about 1/2 ms per iteration)

v3  73951ms (about 74 ms per iteration)

Basically the largest overhead in rendering server side is the creation of the jsdom object tree, then next the parsing of the template, both of which can be done once a load time then just use the precompiled function after that.

Since the dom is so wasteful in its memory use, you wouldn't want to be creating and/or hanging onto it for each request, it will be better to use it once then have a function which returns a string.

I have not run any browser side tests yet, but for use on the server, v2 is the way to go for performance using precompiled templates.

Let me know if I am missing something.

Thanks,

Jeff

[Mic (BeeBole)]

The DOM is needed when rendering too.
For instance when setting the value of an attribute, it is the method:
setAttribute that is used.
And not a string concatenation.
Concatenate strings for attributes is not very safe.

I don't think the design of pure.js fits with server rendering.
Even more from the v3.
The browser is about the DOM. The server is about strings.

I'm still looking for a way to have fast loops.
The trouble is to avoid a recompilation of the inner-loop at each
item.