Jscex: makes async programming easier than ever

[Jeffrey Zhao]

Hi all,

 

Here's an introduction to my recent created project "Jscex". The main goal of the project is to simplify the async programming.

 

Everyone knows "callback-based" async programming is much more difficult than traditional. We always write linear code but async breaks the algorithms so we cannot write conditions with "if" statement or loops with "for/while/do", and the combination of async operation is also quite hard.

 

So I create Jscex. We can write normal, linear code to express async method. For example, if we write a very simple static web server in Node.js, we could:

 

var http = require("http");

var fs = require("fs");

var url = require("url");

var path = require("path");

 

var transferFile = function (request, response) {

    var uri = url.parse(request.url).pathname;

    var filepath = path.join(process.cwd(), uri);

 

    // check whether the file is exist and get the result from callback

    path.exists(filepath, function (exists) {

        if (!exists) {

            response.writeHead(404, {"Content-Type": "text/plain"});

            response.write("404 Not Found\n");

            response.end();

        } else {

            // read the file content and get the result from callback

            fs.readFile(filepath, "binary", function (error, data) {

                if (error) {

                    response.writeHead(500, {"Content-Type": "text/plain"});

                    response.write(error + "\n");

                } else {

                    response.writeHead(200);

                    response.write(data, "binary");

                }

 

                response.end();

            });

        }

    });

}

 

http.createServer(function (request, response) {

    transferFile(request, response);

}).listen(8124, "127.0.0.1");

 

But in Jscex, we can only:

 

Jscex.Async.Node.Path.extend(path);

Jscex.Async.Node.FileSystem.extend(fs);

 

var transferFileAsync = eval(Jscex.compile("async", function (request, response) {

    var uri = url.parse(request.url).pathname;

    var filepath = path.join(process.cwd(), uri);

 

    var exists = $await(path.existsAsync(filepath));

    if (!exists) {

        response.writeHead(404, {"Content-Type": "text/plain"});

        response.write("404 Not Found\n");

    } else {

 

        try {

            var data = $await(fs.readFileAsync(filepath));

            response.writeHead(200);

            response.write(data, "binary");

        } catch (ex) {

            response.writeHead(500, {"Content-Type": "text/plain"});

            response.write(ex + "\n");

        }

    }

 

    response.end();

}));

 

The "Jscex.compile" method would get the function code and convert it to a monadic form so it would be executed in async. The code under the "$await" part would be put into callback functions. Some language constructions like "if/try" would also be replaced to keep the algorithms work as we want it. Besides, write extensions/adaptors of async operations (like "existsAsync" and "readFileAsync" method above) for Jscex is just simple lines of code.

 

All the compilation could be done at runtime by Jscex's JIT compiler so no extra work to do, Jscex just keep everything (idioms, execution behaviors, contexts, closures, etc.) for JavaScript programmers. Jscex also provide an AOT compiler which for pre-runtime compilation.

 

Jscex is designed for JavaScript programming for browser originally, but it just rely on the language part without browser APIs, so it can also used in Node.js programming as the sample above.

 

Here's the GitHub page of Jscex: https://github.com/JeffreyZhao/jscex

 

I put a lot of detail with samples on it, and I with that you could read it and give me your opinions of the project. Any feedbacks are warmly welcomed.

 

Thanks.

 

Jeffrey Zhao
Blog: http://blog.zhaojie.me/
Twitter: @jeffz_cn (Chinese) | @jeffz_en (English)

[Pau]

The documentation is excellent.

[Liam]

Interested to hear how this compares with the streamline utility,
which also compiles sync-style code to async...

[Jeffrey Zhao]

Streamline.js is another interesting approch to do the similar thing but
Jscex has its own design principles.

The first rule of building Jscex is "keep everything for JavaScript
programmers" or "Jscex is JavaScript". People write normal JavaScript code
in normal JavaScript editors. Jscex doesn't need any extra compilation
process before running in JavaScript engines. The JIT compiler generates
codes as codes running so there's no change to the execution behavior -
suitable for developing environment. For example, people just fix the bug
and refresh the page (or restart the node process) and the change takes
effect immediately.

Besides, Jscex is not just for async programming. The compiler has general
rules to convert a function to a monadic one and the rest work is just done
by "library". We build "async" library for async programming, we can also
build a "seq" library which brings "lazy sequence" generator ("yield" in
JavaScript 1.7, python or C#).

Wish it helps. :)

Jeffrey Zhao
Blog: http://blog.zhaojie.me/
Twitter: @jeffz_cn (Chinese) | @jeffz_en (English)

--
You received this message because you are subscribed to the Google Groups
"nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en.

[Morteza Milani]

I think the advantage of using JS as language for Node is that it is asyn ( or event driven ) in nature. So changing the nature is a wrong way. I think we should learn to handle the difficulties of writing async code with good practices instead of changing asyn code to serialised one:)

[Jeffrey Zhao]

Well, “serialized code“ doesn’t means “blocked code”, it still has the benefits like “scalable”, “high performance”, etc. Modern languages like Go or F# provide language features for better programmability in async scenarios – Event-based (callback-based) and “linear code” are just two different ways to do the same thing.

 

Also, “async programming linearly” doesn’t means “programming linearly” everywhere. We can still use event-based for this part and linear code in that part. For example, we use a event-based mechanism to handle an http request:

 

http.createServer(function (request, response) {

    transferFileAsync(request, response).start();

}).listen(8125, "127.0.0.1");

 

But when a request comes in, we use linear code to implement the detailed logic:

 

var transferFileAsync = eval(Jscex.compile("async", function (request, response) {

    var uri = url.parse(request.url).pathname;

    var filepath = path.join(process.cwd(), uri);

 

    var exists = $await(path.existsAsync(filepath)); // async

    if (!exists) {

        response.writeHead(404, {"Content-Type": "text/plain"});

        response.write("404 Not Found\n");

    } else {

 

        try {

            var data = $await(fs.readFileAsync(filepath)); // async

            response.writeHead(200);

            response.write(data, "binary");

        } catch (ex) {

            response.writeHead(500, {"Content-Type": "text/plain"});

            response.write(ex + "\n");

        }

    }

 

    response.end();

}));

 

Thanks.

 

Jeffrey Zhao
Blog: http://blog.zhaojie.me/
Twitter: @jeffz_cn (Chinese) | @jeffz_en (English)

[Morteza Milani]

Jeffrey, I know that serialised one doesn't mean blocked code. And I know that async programming linearly doesn’t means programming linearly.

What I'm trying to say is that We should learn to program with callbacks instead of changing the style of coding to linear.

Why not callbacks?
1- because they cause unmanageable code.
2- because debugging is difficult using callbacks.

I think your solution tries to solve the first problem somehow. But there is another solution: using something like this:

    var myCallback = function(req,res){
    };

    http.createServer(myCallback);

I know that your solution does something more like using try catches, but we can do it in the callback function.

And about the second problem, how do you solve it? does your solution cover this issue?
Node tries to handle this problem using its error stacks to trace code.

if node wanted programming style like linear ones, he chose another language. JS is chosen because it is event-driven in its nature. and everyone is happy with that! I love to code using callbacks. it allows me some new kind of abstraction using callback functions. did you ever thought about it?

You know, trying to code parallel is much more useful for years later, so we should learn to live with it! (Again I know that your solution does not mean linear coding)

Thank you!

[Marak Squires]

The main goal of the project is to simplify the async programming.

I fail to see how this approach achieves that goal. An approach like: https://github.com/caolan/async seems a lot more sane.

--

[Bruno Jouhier]

Streamline.js is also Javascript in the sense that there is no new
syntax. It just reserves the _ identifier which is a bit like
reserving $await (and you can choose a different identifier if you
have a conflict with _).

Streamline gives you 3 options to transform the code:
- by compiling it before running it. The compiler produces standalone
modules that don't have any special external dependencies. You can use
these modules in any node.js program.
- by letting the require infrastructure transform the code. In this
case, you need to require('streamline') at the beginning of your
program.
- dynamically at runtime by calling the transform function.

The third option allows you to easily implement something like jscex.
You just need a little utility function:

function transform(fn) {
var str = require("streamline/lib/
transform").transform(fn.toString(), { noHelpers: true });
var result; eval("result = " + str); return result;
}

Here is the streamlined version of the example:

var transferFileAsync = transform(function f(request, response, _) {

var uri = url.parse(request.url).pathname;
var filepath = path.join(process.cwd(), uri);

var exists = path.exists(filepath, __wrap1(_)); // async

if (!exists) {
response.writeHead(404, {"Content-Type": "text/plain"});
response.write("404 Not Found\n");
} else {
try {

var data = fs.readFile(filepath, _); // async

response.writeHead(200);
response.write(data, "binary");
} catch (ex) {
response.writeHead(500, {"Content-Type": "text/plain"});
response.write(ex + "\n");
}
}
response.end();

});

// special wrapper for path.exists single arg callback
function __wrap1(cb) { return function(result) { cb(null,
result); } };

The syntax is a bit different but the idea seems to be the same.

Bruno

[Jeffrey Zhao]

Did you check out the samples on https://github.com/JeffreyZhao/jscex?

 

I also put some comparison between Jscex and other projects. Projects like async.js uses APIs to help async programming, it works in some cases but - in my point of view - not so easy, elegant and flexible as Jscex. In Jscex, people just write traditional JavaScript functions (no extra APIs) and the compiler generates the code (using the async APIs) for us.

 

Jeffrey Zhao
Blog: http://blog.zhaojie.me/
Twitter: @jeffz_cn (Chinese) | @jeffz_en (English)

From: Marak Squires

Sent: Saturday, May 07, 2011 2:45 PM

To: nod...@googlegroups.com

To unsubscribe from this group, send email to mailto:nodejs%2Bunsu...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.

[Jeffrey Zhao]

Thanks for your feedback.

 

I’m not so sure that callback-based programming is the nature of JavaScript. JavaScript is just “non-blocking” programming based on my experiences, we can do more for “non-blocking”, not only event-driven. I agree with you that “code parallel is much more useful for years later”, but it doesn’t me we’d better use the model of callback. More and more modern languages (C#, F#, Go, etc.) provide build-in language feature to support async/parallel programming, since callback based programming is difficult to express logics / be combined / handle error / ..., I still don’t know what your solutions is.

 

For the debugging part. Jscex’s compilation rules is simple and the generated code is easy to debug. I’ll cover it in the introduction page these days.

 

Thank you.

 

Jeffrey Zhao
Blog: http://blog.zhaojie.me/
Twitter: @jeffz_cn (Chinese) | @jeffz_en (English)

From: Morteza Milani

Sent: Saturday, May 07, 2011 1:57 PM

To: nod...@googlegroups.com

[Jeffrey Zhao]

It seems that Jscex and Streamline.js are using the similar approaches. I'll
investigate more on that.

Thank you.

[jason.桂林]

Great, both Jscex and Streamline.js are very cool.

As a compiler, as part of programming language design, I think coffee-script could pay more attention to this aspect.

2011/5/7 Jeffrey Zhao <je...@live.com>

--
Best regards,

Jason Green
桂林

[Bruno Jouhier]

streamline.js works with coffee-script too.

[jason.桂林]

I have write another tool for this aspect, with different syntax. use regex to parse, very simple implement, about 100 lines codes. but do not support try/catch. 

https://github.com/guileen/slippers

I use a //{ as async callback flag.

//example with mongodb
db.open();//{
var err, books = db.collection('books');//{
var err, cursor = books.find();//{
var err, items = cursor.toArray();//{
doSomething(items);

it will 'compile' to 

db.open(function(){
db.collection('books', function(err, books){
books.find(function(err, cursor){
cursor.toArray(function(err, items){
doSomething(items);
});});});});

It is no need to map line number, all the code will be compile to same line for debug. 

2011/5/8 Bruno Jouhier <bjou...@gmail.com>

[Nicolas Chambrier]

I stopped at "eval()" :/ before I posted this message, I thought I should read more then saw this blocking "$await", it sounds like a fail to me, sorry :/

[Bruno Jouhier]

@Nicolas

You should probably look a bit deeper. $await does not mean that code
is going to block, which would be a total failure. It just means that
the code is going to yield and resume.

Same thing with streamline: the _ marks the points where execution
will yield and resume.

Bruno

[Jeffrey Zhao]

Did you check out the introduction page?
https://github.com/JeffreyZhao/Jscex

Please contact me if you find anything unclear.

Jeffrey Zhao
Blog: http://blog.zhaojie.me/

Twitter: @jeffz_cn (Chinese) | @jeffz_cn (English)
-----原始邮件-----
From: Nicolas Chambrier
Sent: Monday, May 09, 2011 3:02 PM
To: nod...@googlegroups.com
Subject: Re: [nodejs] Re: Jscex: makes async programming easier than ever

I stopped at "eval()" :/ before I posted this message, I thought I should
read more then saw this blocking "$await", it sounds like a fail to me,
sorry :/

--

[Tauren Mills]

Just curious how you pronounce this project name? Every time I read it I hear "js sex" or "j sex". Was that on purpose, indicating it's a sexy project? :)

[Jeffrey Zhao]

I pronounce the project “dʒesks”.

 

Actually Jscex is short for “JavaScript Computation EXpressions”, but you just got what I mean. :)

 

Thanks.

 

Jeffrey Zhao
Blog: http://blog.zhaojie.me/
Twitter: @jeffz_cn (Chinese) | @jeffz_cn (English)

> mailto:nodejs%2Bunsu...@googlegroups.com.

> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com.

> To unsubscribe from this group, send email to mailto:nodejs%2Bunsu...@googlegroups.com.

> For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.
>

[Matt]

I think you should stick with j-sex. :-)