sjs REPL api

[moshir...@gmail.com]

I'm trying to implement a simple REPL server.

With node.js, the standard repl module causes an issue as there's no way  to pause async calls.

Suppose I have a standard node.js api :

For instance, with an async api like this 

// node.js async fetch
db.prototype.fetch = function(options, callback){
    ...
    ...
    callback&&callback(null, [...]); // return results to callback
}

In REPL, the following call might return immedialtely :

prompt> x = db.model.persons.fetch();
null
prompt>x;
null
 

What I need is to wait for the result.

So, I stumbled upon your wonderful work with stratifiedjs ( and even found out sjs have a repl module).

I have the idea to adapt the node.js callback based db api which looked like this in node.js

sjsdb.prototype.fetch = function(options){
   waitfor(var results){
       db.prototype.fetch(options, resume);
   }
   return results;
}

I was hoping that using native node.js repl with adapted sjs would get me out of trouble , but the method returns a continuation object...

I would be happy to use sjs repl, unfortunately, doc is lacking and it seems the api is not as rich as standard node equivalent, specifically missing context settings.

Is there any doc about sjs repl ? Does is support context isolation and streams ?

Any idea how i could achieve my goal with sjs (or node.js) ?

Thanks !

[Tim Cuthbertson]

Hi,

I'm afraid the SJS repl module was built mainly for our needs with the
`sjs` commandline REPL, so it doesn't have the same flexibility as
the nodejs version. It's not under-documented, there just isn't anything
configurable to document ;).

However, it should be straightforward enough to wrap the nodejs
repl module with a custom `eval` that runs SJS code.

It looks like you're close, but you're trying to call into an SJS
function from plain JS, which is why you'd see the continuation object
rather than the result.

While it's possible to call into SJS from plain JS with some additional
effort, the easiest thing to do is to just write the wrapper
code in an .sjs module, and then waiting for a suspended function
should just work.

I had a go, and this seems to work:

var sjsCompiler = require('sjs:compile/sjs');
var { extend } = require('sjs:object');
var repl = require('nodejs:repl');
var vm = require('nodejs:vm');

var overrides = {
    require:require,
}

var eval = function(cmd, context, filename, callback) {
    // NOTE: this function is called from plain JS, so the
    // caller can't wait for a return value. But that's OK,
    // we provide the result to `callback` rather than
    // returning anything.

    try {
        // remove leading '(' and trailing ')' that
        // nodejs inserts for some reason
        cmd = cmd.slice(1, cmd.length-1);

        //console.log("COMPILING: " + cmd);
        var js = sjsCompiler.compile(cmd);
        //console.log("RUNNING: " + js);

        extend(context, overrides);

        var result = vm.runInContext(js, context, '(repl)');
        callback(null, result);
    } catch(e) {
        callback(e);
    }
};

repl.start({eval:eval});

I don't know if your code needs access to sjs' `require` function.
If not, you can leave out the `overrides` stuff. I also don't
know why the nodejs repl wraps the code in parens, but that threw
off the SJS compiler so I had to strip them.

The above module should be named with an .sjs extension, and
run using the `sjs` command rather than `node`. You can still
use whatever nodejs modules you need (prefixed with 'nodejs:'),
but you also get access to SJS' modules and functionality (including
waiting for the completion of any suspended result).

BTW, if you're running this as a HTTP server or using nodejs streams,
we've got a few modules that make things a little easier to work with
in SJS than the raw nodejs APIs:

https://conductance.io/reference#sjs:nodejs/stream
https://conductance.io/reference#sjs:nodejs/http::withServer

And if your server is anything bigger than this, you may want to use conductance, which provides a bunch of useful web functionality on top of SJS (e.g routes, generated pages, rich UI, client-server RPC, etc):
https://conductance.io/

Hope that helps, let us know how it goes :)

Cheers,
 - Tim.

[moshir...@gmail.com]

Thank you for our time Tim.

After testing locally, it seems to work, except I'm not able to assign results from an async method.

With an inline class as follows :

var Book  = function(name){

this.name = name;

}

Book.prototype.get = function(){

waitfor(var res){

setTimeout(function(){

resume("RESULT");

}, 1000);

}

return res;

}

And with the repl server initialized with :

var server = repl.start({eval:eval});
server.context.book = new Book();

When in the repl, the runtime actually blocks but the result is not actually returned :

> var x = book.get();

RUNNING: var x;__oni_rt.exseq(this.arguments,this,undefined,[24,__oni_rt.Sc(3,

nction(_oniX){return x=_oniX;},__oni_rt.C(function(){return book.get()},1))])
undefined
> x

RUNNING: __oni_rt.exseq(this.arguments,this,undefined,[24,__oni_rt.Nb(fun

{return x;},3)])

undefined

So it looks like variable x did not get assigned the book.get() result.

With a little hacking, I ended creating a #.set(name, value) method :

server.context.set = function(name, val){ server.context[name] = val;}

Seems it does the trick : 

> set("x", book.get());
...
> x
RESULT

But i'm not very happy with it.

Ps: 

i'm planning to implement something like ipython in node.js, with a custom data api, and charts.

conductance frameworkf seems like a very good fit and i'll keep you posted.

[Alexander Fritze]

Unfortunately SJS doesn't fully play nice with node's `runInContext`:
For asynchronous operations the scopes gets mixed up, which in your
case means that the value gets assigned to a global `x` variable,
which is shadowed by the var you created in the REPL context.
If you run your code without ever declaring `x`, i.e.:

x = book.get();

instead of

var x = book.get();

then things will work as expected because everything is referring to
the global `x`.

SJS's interaction with `runInContext` could be fixed, but I don't
think it is entirely trivial and we haven't got any resources
scheduled to look at this at the moment. It is not a super-high
priority for us, as the isolation you get with `runInContext` isn't
perfect anyway: It is often better to run a single node process per
REPL.

Also, I don't know what exactly you are planning, but if the REPL is
supposed to have graphical capabilities, have you considered running
it not on the node process, but in the browser?
You could then have a multiple REPLs connecting to a common API
exposed by the server (where "the server" could be remote or just a
node process running on the local machine). This is quite easy to do
in conductance ( https://conductance.io ), and the advantage is that
it gives you a lot of control of what exactly you expose to the REPL;
the browser provides a good sandbox.

To give you an idea, a while back we made a prototype of an
"interactive spreadsheet" that has this architecture - see
http://vimeo.com/27732755. If you want to go into this direction (not
the spreadsheet, but the general architecture), drop us a line. We're
happy to help!

Cheers,
Alex

> --
> You received this message because you are subscribed to the Google Groups
> "StratifiedJS" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to stratifiedjs...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.