Hello everybody. I'm a huge fan of node.js and ve're actively using it in our project for creating various services.
Recently we've faced a problem of significant ram overhead required to run each node.js instance (about 40Mb). This is an issue since we are running a number of different node.js services on one machine, and the memory is limited.
So what we trying to use is run multiple services in single node.js process using vm.runInNewContext(), but compared this approach is missing some functionality:
1) We cannot monitor amount of memory allocated by each subprocess. 2) We cannot cancel all setTimeouts and process.nextTicks for given process to effectively stop its execution. 3) We cannot get a list of open sockets and file descriptors for given process.
With this functionality we could use node.js similar to erlang - run a number of processes inside a single vm - and use some erlang features (like 'let it fail') that would certanly lead us to world domination ;)
So, is there any projects targeted to solve similar problem?
Isolates were taken a look at a while ago. They are similar to what you want, but still unable to do so. It would generally be more sane to just scale out processes was what was decided. The edge cases of giving multiple "light weight" isolates proved to be very difficult to prevent Isolates from affecting each other (env variables, cwd, ulimits, etc.). If you are running large numbers of processes and having memory issues I might recommend you look into where that memory is coming from, node itself is around 10Mb overhead. The rest is most likely some kind of cache or non-GCable objects in your application. Note: Code source is cached, and coffeescript can compound things in unexpected ways (not always bad, inspect yourself).
On Saturday, October 13, 2012 4:29:17 AM UTC-5, Alexey Guskov wrote:
> Hello everybody. I'm a huge fan of node.js and ve're actively using it in > our project for creating various services.
> Recently we've faced a problem of significant ram overhead required to run > each node.js instance (about 40Mb). This is an issue since we are running a > number of different node.js services on one machine, and the memory is > limited.
> So what we trying to use is run multiple services in single node.js > process using vm.runInNewContext(), but compared this approach is missing > some functionality:
> 1) We cannot monitor amount of memory allocated by each subprocess. > 2) We cannot cancel all setTimeouts and process.nextTicks for given > process to effectively stop its execution. > 3) We cannot get a list of open sockets and file descriptors for given > process.
> With this functionality we could use node.js similar to erlang - run a > number of processes inside a single vm - and use some erlang features (like > 'let it fail') that would certanly lead us to world domination ;)
> So, is there any projects targeted to solve similar problem?
2 and 3 is quite doable using domains, and 1st could be achievable with v8 debugger.
š
It's a hack, but nevertheless it's possible, if all your processes follow some rules and don't mess with each other. But I'm afraid that an infinite loop in a service will screw the whole process anyway.
Hello everybody. I'm a huge fan of node.js and ve're actively using it in our project for creating various services.
Recently we've faced a problem of significant ram overhead required to run each node.js instance (about 40Mb). This is an issue since we are running a number of different node.js services on one machine, and the memory is limited.
So what we trying to use is run multiple services in single node.js process using vm.runInNewContext(), but compared this approach is missing some functionality:
1) We cannot monitor amount of memory allocated by each subprocess.
2) We cannot cancel all setTimeouts and process.nextTicks for given process to effectively stop its execution.
3) We cannot get a list of open sockets and file descriptors for given process.
With this functionality we could use node.js similar to erlang - run a number of processes inside a single vm - and use some erlang features (like 'let it fail') that would certanly lead us to world domination ;)
So, is there any projects targeted to solve similar problem?
> 2 and 3 is quite doable using domains, and 1st could be achievable with v8 > debugger.
> It's a hack, but nevertheless it's possible, if all your processes follow > some rules and don't mess with each other. But I'm afraid that an infinite > loop in a service will screw the whole process anyway.
> -- > // alex
> Hello everybody. I'm a huge fan of node.js and ve're actively using it in > our project for creating various services.
> Recently we've faced a problem of significant ram overhead required to run > each node.js instance (about 40Mb). This is an issue since we are running a > number of different node.js services on one machine, and the memory is > limited.
> So what we trying to use is run multiple services in single node.js > process using vm.runInNewContext(), but compared this approach is missing > some functionality:
> 1) We cannot monitor amount of memory allocated by each subprocess.
> 2) We cannot cancel all setTimeouts and process.nextTicks for given > process to effectively stop its execution.
> 3) We cannot get a list of open sockets and file descriptors for given > process.
> With this functionality we could use node.js similar to erlang - run a > number of processes inside a single vm - and use some erlang features (like > 'let it fail') that would certanly lead us to world domination ;)
> So, is there any projects targeted to solve similar problem?
By the way, i believe the way google chrome deals with multiple pages and browser addons is very similar to what i'm trying to achieve: isolated js vm instances inside a single process. If anyone knows chrome internals, please correct me.
суббота, 13 октября 2012 г., 13:29:17 UTC+4 пользователь Alexey Guskov написал:
> Hello everybody. I'm a huge fan of node.js and ve're actively using it in > our project for creating various services.
> Recently we've faced a problem of significant ram overhead required to run > each node.js instance (about 40Mb). This is an issue since we are running a > number of different node.js services on one machine, and the memory is > limited.
> So what we trying to use is run multiple services in single node.js > process using vm.runInNewContext(), but compared this approach is missing > some functionality:
> 1) We cannot monitor amount of memory allocated by each subprocess.
> 2) We cannot cancel all setTimeouts and process.nextTicks for given > process to effectively stop its execution.
> 3) We cannot get a list of open sockets and file descriptors for given > process.
> With this functionality we could use node.js similar to erlang - run a > number of processes inside a single vm - and use some erlang features (like > 'let it fail') that would certanly lead us to world domination ;)
> So, is there any projects targeted to solve similar problem?
On Mon, Oct 15, 2012 at 8:44 PM, Alexey Guskov <kvasdo...@gmail.com> wrote:
> By the way, i believe the way google chrome deals with multiple pages and
> browser addons is very similar to what i'm trying to achieve: isolated js
> vm instances inside a single process. If anyone knows chrome internals,
> please correct me.
> суббота, 13 октября 2012 г., 13:29:17 UTC+4 пользователь Alexey Guskov
> написал:
>> Hello everybody. I'm a huge fan of node.js and ve're actively using it in
>> our project for creating various services.
>> Recently we've faced a problem of significant ram overhead required to
>> run each node.js instance (about 40Mb). This is an issue since we are
>> running a number of different node.js services on one machine, and the
>> memory is limited.
>> So what we trying to use is run multiple services in single node.js
>> process using vm.runInNewContext(), but compared this approach is missing
>> some functionality:
>> 1) We cannot monitor amount of memory allocated by each subprocess.
>> 2) We cannot cancel all setTimeouts and process.nextTicks for given
>> process to effectively stop its execution.
>> 3) We cannot get a list of open sockets and file descriptors for given
>> process.
>> With this functionality we could use node.js similar to erlang - run a
>> number of processes inside a single vm - and use some erlang features (like
>> 'let it fail') that would certanly lead us to world domination ;)
>> So, is there any projects targeted to solve similar problem?
Thats' right, but it's doesn't always create a separate process, sometimes
multiple tabs run in single process (when they're from same domain and from
same origin). Besides, chrome addons certanly live in same process, but are
somehow isolated from each other and js in pages. I suppose they're created
by single vm, memory overhead would be significant otherwise.
Anyway, it must be possible to create multiple vm's in v8 and isolates
feature seems to be what i'm looking for. It's not implemented, however,
and i don't know if it will be. So for now i'll take a look on what can be
done with domains.
> Actually chrome has multiple processes running, i think its creating one
> for every tab you have open
> On Mon, Oct 15, 2012 at 8:44 PM, Alexey Guskov <kvasdo...@gmail.com>wrote:
>> By the way, i believe the way google chrome deals with multiple pages and
>> browser addons is very similar to what i'm trying to achieve: isolated js
>> vm instances inside a single process. If anyone knows chrome internals,
>> please correct me.
>> суббота, 13 октября 2012 г., 13:29:17 UTC+4 пользователь Alexey Guskov
>> написал:
>>> Hello everybody. I'm a huge fan of node.js and ve're actively using it
>>> in our project for creating various services.
>>> Recently we've faced a problem of significant ram overhead required to
>>> run each node.js instance (about 40Mb). This is an issue since we are
>>> running a number of different node.js services on one machine, and the
>>> memory is limited.
>>> So what we trying to use is run multiple services in single node.js
>>> process using vm.runInNewContext(), but compared this approach is missing
>>> some functionality:
>>> 1) We cannot monitor amount of memory allocated by each subprocess.
>>> 2) We cannot cancel all setTimeouts and process.nextTicks for given
>>> process to effectively stop its execution.
>>> 3) We cannot get a list of open sockets and file descriptors for given
>>> process.
>>> With this functionality we could use node.js similar to erlang - run a
>>> number of processes inside a single vm - and use some erlang features (like
>>> 'let it fail') that would certanly lead us to world domination ;)
>>> So, is there any projects targeted to solve similar problem?
> Thats' right, but it's doesn't always create a separate process, sometimes multiple tabs run in single process (when they're from same domain and from same origin). Besides, chrome addons certanly live in same process, but are somehow isolated from each other and js in pages. I suppose they're created by single vm, memory overhead would be significant otherwise.
> Anyway, it must be possible to create multiple vm's in v8 and isolates feature seems to be what i'm looking for. It's not implemented, however, and i don't know if it will be. So for now i'll take a look on what can be done with domains.
> 2012/10/16 Michal Kruk <kru...@gmail.com>
> Actually chrome has multiple processes running, i think its creating one for every tab you have open
> On Mon, Oct 15, 2012 at 8:44 PM, Alexey Guskov <kvasdo...@gmail.com> wrote:
> By the way, i believe the way google chrome deals with multiple pages and browser addons is very similar to what i'm trying to achieve: isolated js vm instances inside a single process. If anyone knows chrome internals, please correct me.
> суббота, 13 октября 2012 г., 13:29:17 UTC+4 пользователь Alexey Guskov написал:
> Hello everybody. I'm a huge fan of node.js and ve're actively using it in our project for creating various services.
> Recently we've faced a problem of significant ram overhead required to run each node.js instance (about 40Mb). This is an issue since we are running a number of different node.js services on one machine, and the memory is limited.
> So what we trying to use is run multiple services in single node.js process using vm.runInNewContext(), but compared this approach is missing some functionality:
> 1) We cannot monitor amount of memory allocated by each subprocess.
> 2) We cannot cancel all setTimeouts and process.nextTicks for given process to effectively stop its execution.
> 3) We cannot get a list of open sockets and file descriptors for given process.
> With this functionality we could use node.js similar to erlang - run a number of processes inside a single vm - and use some erlang features (like 'let it fail') that would certanly lead us to world domination ;)
> So, is there any projects targeted to solve similar problem?
