[ANN] node-candle module for callbacks that are freed either by resolving or timing out

[Alexey Kupershtokh]

Here it is: https://github.com/AlexeyKupershtokh/node-candle

it's similar to:

https://github.com/coolaj86/futures/tree/v2.0/future

and

https://github.com/temsa/addTimeout

to some extent, except that the callbacks are able to free in my case allowing to avoid leaks.

As an yet another example, if you use a callback wrapped by addTimeout(timeout, cb) or future.once(cb).setTimeout(timeout) as an ACK callback for a socket.io request that is never acknowledged, this wrapepd callback would exist till the socket is disconnected.

Any response is highly appreciated.

[Mark Hahn]

This appears to be quite useful.

How does this compare to putting the timeout and once-filter around a normal call?  How much code does it save?

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[Alexey Kupershtokh]

This is gonna be a lot of code.

1) we need to pass something to the other side. Let's use incremental request_id.

2) we need a structure that would be an interruptable intermediate between real callback and the one that handles answers. Let's use the EventEmitter for saving code.

3) we need to call x = setTimeout and clearTimeout(x). And keep the x variable somewhere.

// global structures

var id = 0;

var e = new EventEmitter;

// global 'response' message handler

socket.on('response', function() { e.emit.apply(e, arguments); })

// a single callback assignment & sending the request

var real_callback = function(err, response) { ... };

var request_id = (function(callback, timeout) {

  var x = setTimeout(function(id) {

    e.emit(request_id, 'timeout')

  }, timeout, ++request_id);

  e.once(request_id, function() {

    clearTimeout(x);

    callback.apply(null, Array.prototype.slice(arguments, 1));

  });

  return request_id;

})(real_callback, timeout);
socket.send('request', request_id);

This is similar to this code using candle:

// global structures

var c = new candle;

// global 'response' message handler

socket.on('response', function() { c.resolve.apply(c, arguments); })

// a single callback assignment & sending the request

var real_callback = function(err, response) { ... };

var request_id = c.add(real_callback, timeout);

socket.send('request', request_id);

(I haven't tested the code, wrote it just to show how it works)

четверг, 1 ноября 2012 г., 1:22:20 UTC+7 пользователь Mark Hahn написал:

[Alexey Kupershtokh]

I've published a new version. Re-worked it heavily, updated the docs, added more example, tests, travis-ci, etc.

Now you will have to add timeout manually using a separate call.

I'm going to add a method addWithTimeout(cb, timeout) or something similar for syntax sugar. add() is intended to eventually support extra args, that will mix with resolve extra args and all of them will be passed into the callback.

среда, 31 октября 2012 г., 18:48:56 UTC+7 пользователь Alexey Kupershtokh написал:

[jmar777]

Looks like an interesting idea.  We had to build some similar functionality for wrapping some fairly indeterminate behavior to a request/response model.  Part of our solution there was a utility that at least offered some of the timeout handling: https://github.com/jmar777/cb

Using the above, we then just had a map that kept track of callbacks keyed by an id (which looks like that's part of the problem candle addresses).  Nice work!

[Alexey Kupershtokh]

Thank you :) It's very pleasant to meet a person that feels the same problem very well!

Yes, one of the key points is exactly handling the map of callbacks.

Do I understand correctly that your cb module stands in one row with addTimeout and future and doesn't provide freeing callbacks on it's own?

Since you've written a similar module too, you would likely want to know that one of high speed factors of the module was using optimized versions of setTimeout (1.05..1.5 times) and clearTimeout (20-30 times faster :)). I've created a pull request ( https://github.com/joyent/node/pull/4193 ) for node.js. More details and benchmarks are here - https://github.com/joyent/node/issues/4225#issuecomment-9971557

пятница, 2 ноября 2012 г., 20:03:14 UTC+7 пользователь jmar777 написал:

[jmar777]

Very interesting - I wasn't aware of the set/clearTimeout optimizations.  And yes, you're correct about the scope of my module - it's just a lightweight utility for handling timeouts and enforcing certain callback execution semantics.

[Alexey Kupershtokh]

jmar777,

https://github.com/AlexeyKupershtokh/node-candle/wiki/Todo

check out the table please

суббота, 3 ноября 2012 г., 4:46:11 UTC+7 пользователь jmar777 написал:

[jmar777]

Nice work on the comparisons.  A couple questions:

• For "configurable error handler call", are you referring to timeouts only?  If so the description of `cb` is accurate, otherwise not exactly.
• Not sure what is being compared here: "can forcibly freecallbackbefore ..."

Also, it appears that your table has inspired some `cb` pull requests to reach feature parity with node-candle: https://github.com/jmar777/cb/pulls

I love Open Source :)

[Alexey Kupershtokh]

Wow. Jason Brumwell is amazing :)

> For "configurable error handler call", are you referring to timeouts only?  If so the description of `cb` is accurate, otherwise not exactly.

Yes, probably I should change the definition. What I meant is "does it allow to call the same callback on timeout with args different from (new TimeoutError)?". Especially this case is useful when using some 3rd party libs' callbacks that always expect to be called as (err, result), as in the case with async.parallel. Am I mistaken regarding cb in this case?

> Not sure what is being compared here: "can forcibly freecallbackbefore ..."

In the case of cb there is a function returned from the cb() call. Then you pass this function as a callback to somewhere where it's wired with an additional ref and even if it will be timed out, it will still persist in the memory. In the case of cb, the only thing remain after a callback is timed out: it's the id that was assigned to this callback. And more than that, this id will remain not in the process where the callback was assigned but rather on a different server.

--

[Alexey Kupershtokh]

you should read it:
> In the case of CANDLE, the only thing remain after a callback is timed out OR DELETE()D

[Jason Brumwell]

@Alexey great work on candle, I was looking for something like this for a project and it had the feature set I needed. The api just didn't feel right for me, and when I spec'd out how I would write it, it was very close to cb, so I spent a couple of hours adding the feaures it lacked.

I wanted to also thank you for creating that table, it outlined a couple of features that I hadn't thought of but could definitely make use of.

On Wednesday, October 31, 2012 7:48:56 AM UTC-4, Alexey Kupershtokh wrote:

[Alexey Kupershtokh]

Jason, in your turn you've inspired me how to improve candle's usage syntax :) I'll try to implement this later.

Also I've added a benchmark to compare our projects - https://github.com/AlexeyKupershtokh/node-candle/blob/master/benchmark/compare.js

My numbers are in the table too.

Please check if the benchmark code is fair enough. It measures only the sides that are applicable to the candle.

суббота, 10 ноября 2012 г., 22:09:28 UTC+7 пользователь Jason Brumwell написал:

[Alexey Kupershtokh]

The benchmark seems broken: I've added gc() & output memory after it in on('cycle') and the memory usage is growing.

воскресенье, 11 ноября 2012 г., 16:37:53 UTC+7 пользователь Alexey Kupershtokh написал:

[Alexey Kupershtokh]

https://github.com/joyent/node/issues/4273

Seems that's it.

Also cb uses process.nextTick and feels bad in a synchronous benchmark so had to add { async: true } to it's benchmarks.

понедельник, 12 ноября 2012 г., 0:06:29 UTC+7 пользователь Alexey Kupershtokh написал:

[Jason Brumwell]

Definitely I think nextTick should be optional, removing it almost doubles ops/sec, thanks for setting this up interested to see where the differences are. I'll issue a PR to make it optional

On Wednesday, October 31, 2012 7:48:56 AM UTC-4, Alexey Kupershtokh wrote:

[Alexey Kupershtokh]

Cool.

Could you also check a "candelabrum" branch of node-candle?

https://github.com/AlexeyKupershtokh/node-candle/blob/candelabrum/candle3.js

It's implements the syntax:

var c = new Candelabrum();

var id = c.add(callback).setTimeout().getId();

c.get(id).resolve();

Though it's about 15% (0.85x) slower in the add-settimeout-resolve scenario and about 40% slower (0.6x) in add-resolve scenario.

понедельник, 12 ноября 2012 г., 7:08:01 UTC+7 пользователь Jason Brumwell написал: