compile a string

[Michael]

hi!

I am writing a templating engine where I generate JavaScript code from a mustache like template.

The templates use asynchronous ways of getting data, partials and stuff like this. So far everything works as expected.

I am now searching for a way to include structures like if / else, for and so on. I think that streamline.js can help me with this. What I haven't found is a way to compile a string in node.js.

For example I have the following code:

var code = "console.log('hello ...');";

code += "setTimeout(_, 1000);";

code += "console.log('... world');"

Now I would expected something like:

var streamlined = require("streamline").compile(code);

to execute the code:

streamlined();

Is it possible to use streamline in this way?

Regards,

Michael

[Bruno Jouhier]

Hi Michael,

You can do it with the transform function:

var compiled = require('streamline/lib/callbacks/transform').transform(code);
eval(compiled);

Bruno

[Michael]

thank you very much Bruno!

I did my job and get things to work:)

My example code is here: http://pastebin.com/5R3BEBkT

The root template (line 40-54) the bottom is transformed to streamline.js. Within this template there is another template included (line 51) which is fetched by the resolvePartial function.

I have three further questions:

1. The runtime function expects a filename as argument, what does this mean? I used "test" in the code but I am not quite sure if this is correct in that case.

2. Is it save to have a single runtime for all requests? Imagine a web server where the runtime is stored as a "global" variable or do i need to "make" a runtime for every call.

3. Do I use streamline.js in the right way?

Regards,

Michael

[Bruno Jouhier]

Good start! I've put my answers below:

On Wednesday, August 15, 2012 9:55:56 AM UTC+2, Michael wrote:

thank you very much Bruno!

I did my job and get things to work:)

My example code is here: http://pastebin.com/5R3BEBkT

The root template (line 40-54) the bottom is transformed to streamline.js. Within this template there is another template included (line 51) which is fetched by the resolvePartial function.

I have three further questions:

1. The runtime function expects a filename as argument, what does this mean? I used "test" in the code but I am not quite sure if this is correct in that case.

The filename is used for error stack traces. Streamline overrides error.stack to give you the stack of pseudo-sync calls, and for this it needs a filename for every function that it compiles. If you set it incorrectly or don't set it at all nothing really bad will happen but you'll get incorrect or undefined filenames in error stacks.
 

2. Is it save to have a single runtime for all requests? Imagine a web server where the runtime is stored as a "global" variable or do i need to "make" a runtime for every call.

Yes it is safe. Anyway, the node.js "require" logic caches the modules. So if you require the same module several times you'll always get the same module object back.
 

3. Do I use streamline.js in the right way?

There is not one right way but many good ways. Most people just write xxx._js or xxx._cs modules and let streamline's require hooks do the transform. Your scenario is a bit special and you need to transform the source and eval it but this is perfectly valid and this is why I made the transform API public. Otherwise, your test code looks fine (how you use the _ marker).

BTW, the requirejs call does not have the standard cb(err, result) signature and you cannot use it directly with streamline but you could write a small wrapper like:

function requirejsWrapper(paths, cb) {
  requirejs(paths, function() {
    cb(null, arguments);
  });
}

Then, if you need to require additional modules inside templates you can do it as:

var moreModules = requirejsWrapper(morePaths, _);

Don't know if you'll really need this but you'll need wrappers like this  if you call async functions that don't have the standard node callback signature.

Bruno

[Michael]

just to be sure to get you right:

var __rt = runtime.runtime("test"),

__func = __rt.__func,

__cb = __rt.__cb,

__trap = __rt.__trap;

I agree that runtime is a singleton. But I call the function runtime.runtime("...") which returns a new object for every call. I expected that some state is saved inside this "runtime" so I was little bit nervous reusing this object.

[Bruno Jouhier]

On Wednesday, August 15, 2012 4:14:33 PM UTC+2, Michael wrote:

just to be sure to get you right:

var __rt = runtime.runtime("test"),

__func = __rt.__func,

__cb = __rt.__cb,

__trap = __rt.__trap;

I agree that runtime is a singleton. But I call the function runtime.runtime("...") which returns a new object for every call. I expected that some state is saved inside this "runtime" so I was little bit nervous reusing this object.

Yes, runtime.runtime(filename) returns a different object every time. This object is basically a container for functions (__func, __cb, __trap, ...). All these functions but one (__func) are the same in all instances returned by runtime.runtime(filename). The only one which is different between instances is __func and it only differs by the way filename is bound. These functions are all designed to be fully reentrant and usable by multiple concurrent requests. So relax, you can use just one, or as many as you want. It won't make a difference (except that you'll use a bit more memory if you allocate several).

If you dig deeper into runtime you will see that it uses a global (__g). May sound problematic at first because concurrent requests will compete on this global but the runtime logic is designed to cope with the fact that this global will be clobbered at every turn of the event loop. So here also, things are safe. And if you need a global context that remains valid across async calls (the equivalent of a TLS in threaded system), streamline gives you one. Take a look at the `globals.context` API in https://github.com/Sage/streamlinejs/blob/master/lib/globals.md. This is something we use extensively to keep track of per-request locales.

Bruno

[Michael]

great  explanation. thanks!