v0.8 Roadmap

[Ryan Dahl]

We're just starting development on the new branch of Node. New
development continues in the master branch - bug fixes go into the
v0.6 branch. The current target for a v0.8 release is early January.
We will continue weekly v0.6 releases throughout v0.8 development.

Node v0.8 roadmap
-----------------

- Move the remaining platform functions from src/platform-*.cc into
libuv. (Everyone)
https://github.com/joyent/node/issues/2132

- Get rid of WAF. All platforms will build using GYP. (Ben, Ryan)
https://github.com/joyent/node/issues?labels=gyp&sort=created&direction=desc&state=open&page=1

- Isolates. Node will allow users to spawn "child processes" that actually
run in a thread. We have to get rid of all the global variables in node.
Compiled extensions need know which isolate they're targeting, and we need
to decide if we want to load the extension multiple times or just once.
Also, some changes to libuv are necessary, since we will have to completely
clean up a loop. Finally we'll have to deal with debugging a multi-threaded
node process. (Ben, Ryan)
https://github.com/joyent/node/issues/2133

- Domains. Domains provide a lightweight isolation mechanism for all i/o
related to a particular network connection (e.g. an incoming http request).
If an unhandled error is encountered, all i/o local to that particular
domain is canceled and all handles are cleaned up. An experimental
implementation can be found at https://github.com/joyent/node/commits/domains.
Some of the early work for domains will be used in Isolate support (e.g.
cleaning up handles when an Isolate is killed) (Bert, Ryan)
https://github.com/joyent/node/issues/2134

- Better allocator. Currently node uses a memory pool allocator to provide
read buffers for incoming data streams. This allocator is only
reasonably efficient for network connections and local pipes on Unix, and on
Windows when zero reads are used. For file i/o and buffered reads on Windows
we need a better story. (Igor)
https://github.com/joyent/node/issues/2135

- Addons. We need to define an easy and suggested way of building
extensions, which should be similar across all supported platforms. (Everyone)
https://github.com/joyent/node/issues/2136

[Mikeal Rogers]

On Nov 16, 2011, at November 16, 20111:36 PM, Ryan Dahl wrote:

> - Isolates. Node will allow users to spawn "child processes" that actually
> run in a thread. We have to get rid of all the global variables in node.
> Compiled extensions need know which isolate they're targeting, and we need
> to decide if we want to load the extension multiple times or just once.
> Also, some changes to libuv are necessary, since we will have to completely
> clean up a loop. Finally we'll have to deal with debugging a multi-threaded
> node process. (Ben, Ryan)
> https://github.com/joyent/node/issues/2133

I'm wondering what the plan is intended use is for isolates.

If I remember correctly, the main reason we want to use them to get away from using multiple system processes as we do now for processing concurrency, mainly because Windows really wants you to use threads.

If that is the case, will the current cluster APIs, and most node APIs in general, remain unchanged and as "dumb" or "smart" about clustering as they are today? For example: using cluster today you will have an http module per process with a separate pool global to that process. When I see notes like "remove all global variables" I wonder if there is a plan to change this behavior.

[Liam]

On Nov 16, 1:58 pm, Mikeal Rogers <mikeal.rog...@gmail.com> wrote:
> On Nov 16, 2011, at November 16, 20111:36 PM, Ryan Dahl wrote:
>
> > - Isolates. Node will allow users to spawn "child processes" that actually
> >  run in a thread. We have to get rid of all the global variables in node.
> >  Compiled extensions need know which isolate they're targeting, and we need
> >  to decide if we want to load the extension multiple times or just once.

Will this break most compiled modules again?

> If I remember correctly, the main reason we want to use them to get away from using multiple system processes as we do now for processing concurrency, mainly because Windows really wants you to use threads.

Seems like we'd prefer processes to threads for cp.fork() & cluster
(at least on Unixes) so that a failure in one (e.g. module segfault)
won't take them all down?

[Tomasz Janczuk]

Isolates support is a very interesting feature. I've run some experiments related to application density using various isolation constructs V8 offers (details at http://github.com/tjanczuk/denser and http://cloud.github.com/downloads/tjanczuk/denser/High%20density%20server%20side%20JavaScript.pdf). In the results I got using multiple V8 isolates to run several copies of the same JS application in a single process resulted in ~2x reduction of memory use compared to running the same number of applications each in its own process. 

[Liam]

On Nov 16, 1:36 pm, Ryan Dahl <r...@tinyclouds.org> wrote:
> - Domains. Domains provide a lightweight isolation mechanism for all i/o
>   related to a particular network connection (e.g. an incoming http request).
>   If an unhandled error is encountered, all i/o local to that particular
>   domain is canceled and all handles are cleaned up. An experimental

>   implementation can be found athttps://github.com/joyent/node/commits/domains.

>   Some of the early work for domains will be used in Isolate support (e.g.
>   cleaning up handles when an Isolate is killed) (Bert, Ryan)
>  https://github.com/joyent/node/issues/2134

If the main purpose of a Domain is to turn a thrown exception into an
event, why not try/catch/finally terminology?

var ta = process.tryAsync // or tryCatch, etc
ta.on('catch', function(err) {}) // called if obstreperous
ta.on('finally', function() {}) // called either way

io.async(..., function(obstreperous) {
if (obstreperous) throw obstreperous;
ta.finally();
})

[Dave Clements]

I was under the impression that threads create problems that node had
solved?

[Liam]

On Nov 16, 3:32 pm, Dave Clements <huperekch...@googlemail.com> wrote:
> I was under the impression that threads create problems that node had
> solved?

Managing concurrent access to shared state is the main problem with
threads. These threads would not share any state; they're functionally
equivalent to multiple Node processes.

[Dave Clements]

Ahh right, thanks Liam.

[Liam]

Tomasz, your project says it only runs on Windows. Done any testing on
Unixes?

Your slides don't show how the RAM per thread+isolate scales over N
threads. How many threads per process have you tried?

Also what if any of the app environment is in shared libraries? If
none, wouldn't that reduce per-process overhead?

On Nov 16, 2:27 pm, Tomasz Janczuk <tom...@janczuk.org> wrote:
> Isolates support is a very interesting feature. I've run some experiments
> related to application density using various isolation constructs V8 offers

> (details athttp://github.com/tjanczuk/denserandhttp://cloud.github.com/downloads/tjanczuk/denser/High%20density%20se...).

[jmar777]

I don't think this was ever discussed in the context of a particular
version, but I recall at one point there were discussions about
bringing Promises back into core. Is this now off the road map
entirely, or just not currently slated for 0.8?

[Aikar]

jmar: do you mean "if a CB is not specified, return a promise instead" kind of approach?

I don't think that works because not all async functions have undefined returns (ie: they actually return something, for example stream.write)

For cases like that, you would need yet another function to return a promise...

[Axel Kittenberger]

> (ie: they actually return something, for example stream.write)

stream.write has no callback. Just saying.

[Aikar]

ah... net stream.write is what i meant... thought .write was down on the stream level.

[Dean Landolt]

On Thu, Nov 17, 2011 at 10:35 AM, Aikar <xdr...@gmail.com> wrote:

jmar: do you mean "if a CB is not specified, return a promise instead" kind of approach?

I don't believe it was ever the idea to bring promises back into the API, just to use them for some core functionality. So it's not really something that would belong in the roadmap.

[jmar777]

> I don't believe it was ever the idea to bring promises back into the API,
> just to use them for some core functionality. So it's not really something
> that would belong in the roadmap.

Ahh, cool. I personally kind of liked the idea, but was more so just
curious. Thanks!

[Mikeal Rogers]

I think what Ryan meant was that "promisey" objects were going to go back in to core and that we may want to standardize them a bit more.

We have several domain specific objects, which are streams, which also resolve several states at some time in the future. Files, sockets, and http among them. While the stream interface is specified there are several other states in the objects that is not specified nor is the interface entirely implemented for people to extend.

-Mikeal

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[Ryan Dahl]

It's not off the roadmap but it's not a "must have" for 0.8. This is
something Isaac will be looking at after NPM is fully functioning on
Windows. Bring your API proposals to him.

[Dean Landolt]

On Thu, Nov 17, 2011 at 11:36 AM, Mikeal Rogers <mikeal...@gmail.com> wrote:

I think what Ryan meant was that "promisey" objects were going to go back in to core and that we may want to standardize them a bit more.

We have several domain specific objects, which are streams, which also resolve several states at some time in the future. Files, sockets, and http among them. While the stream interface is specified there are several other states in the objects that is not specified nor is the interface entirely implemented for people to extend.

Thanks for the clarification Mikeal -- I know exactly what you mean. It's refreshing to hear that this will get some intention.

[Liam]

Hoping for thoughts on this...

[Liam]

Anyone else with me or against on this?

[Mikeal Rogers]

The complexity of maintaining two paths (processes and threads) across posix and windows might be a pain in the ass.

Also, in the future there are optimizations we can make if we are all in the same process.

For instance, WebWorkers are shared nothing as well but Chrome has done some recent work to allow you to pass messages to them by reference rather than suffering serialization.

-Mikeal

[Tomasz Janczuk]

The particular numbers I show on the slide were obtained by running 50
instances of an HTTP long polling web application (https://github.com/
tjanczuk/denser/blob/master/src/denser/samples/chat2.js) either as 50
separate processes or 50 isolates each on its own thread in a single
process, averaged over the number of apps. The memory metric I used is
"private working set", which on Windows is the amount of physical
memory utilized by a process that cannot be shared with other
processes (so shared libs are excluded; in this case the v8.dll on
windows was excluded).

I did not run an experiment on a *nix that would utilize V8 isolates.
I did however experiment with running multiple apps in a single node
process on Linux using the vm module, using separate script contexts
for each app. This is of course much more lightweight than using
isolates and does not provide that level of resource isolation, plus
does not support multi-threading, but from the perspective of memory
measurements it provides an idea of the memory footprint below which
isolates will never let you get. So again I took 100 instances of a
node.js http long polling web chat application (https://github.com/
tjanczuk/denser/blob/master/src/nodejs/samples/chat2.js) and run
either in 100 processes on linux or 100 script contexts inside a
single process. The memory metric I used was non-shared memory
utilized by the process(es) averaged over the number of apps. V8 was
linked in as a shared lib. The variant using script contexts ended up
utilizing 21x less memory per application that the variant using
processes.

On Nov 16, 10:11 pm, Liam <networkimp...@gmail.com> wrote:
> Tomasz, your project says it only runs on Windows. Done any testing on
> Unixes?
>
> Your slides don't show how the RAM per thread+isolate scales over N
> threads. How many threads per process have you tried?
>
> Also what if any of the app environment is in shared libraries? If
> none, wouldn't that reduce per-process overhead?
>
> On Nov 16, 2:27 pm, Tomasz Janczuk <tom...@janczuk.org> wrote:
>
>
>
>
>
>
>
> > Isolates support is a very interesting feature. I've run some experiments
> > related to application density using various isolation constructs V8 offers

> > (details athttp://github.com/tjanczuk/denserandhttp://cloud.github.com/downloads......).

[Liam]

On Nov 17, 11:40 am, Tomasz Janczuk <tom...@janczuk.org> wrote:
> The particular numbers I show on the slide were obtained by running 50
> instances of an HTTP long polling web application (https://github.com/
> tjanczuk/denser/blob/master/src/denser/samples/chat2.js) either as 50
> separate processes or 50 isolates each on its own thread in a single
> process, averaged over the number of apps. The memory metric I used is
> "private working set", which on Windows is the amount of physical
> memory utilized by a process that cannot be shared with other
> processes (so shared libs are excluded; in this case the v8.dll on
> windows was excluded).

It would be interesting to see the numbers across a range of threads.
(It'd also be nice to have a script to automate the tests so ppl can
run it in different contexts.)

It would also be helpful to see an application that has far higher
memory demands, and uses some common third party modules...

[Ryan Dahl]

There is no memory sharing between isolates.

[Liam]

On Nov 17, 11:57 am, Ryan Dahl <r...@tinyclouds.org> wrote:

Exactly, so the 2x benefit described seems unlikely for real-world
apps.

[Bert Belder]

I don't think Tomasz' experiments used memory sharing. He was just
measuring the 'overhead' of running a separate process.

[Tomasz Janczuk]

I agree the benefit would greatly diminish for memory-bound node
applications since the shared global state of the process would be a
very small fraction of the private non-shared state across isolates. I
am curious if you have you seen a lot of node apps with large memory
consumption in the "real world"? I chose HTTP long polling web chat
app as a benchmark based on the assumption that node works
particularly well the class of apps that need to orchestrate IO while
doing a lot of non-CPU and non-memory intensive waiting in between. I
think such apps would actually show reduced average memory footprint
when hosted in isolates as opposed to individual processes.

[Liam]

Right, so shouldn't that be given as KB per process (on various OSes)
not a ratio?

And then the question is whether that cost is worth the benefits of a
process, e.g. it can't crash other processes.

Maybe it's not; I don't have an opinion. Ryan's on the record saying
"processes are the tool for concurrency," so this seems like a
significant shift?

[Ryan Dahl]

We're giving people the option of starting a new Node either as a
standalone process or as a new thread/isolate. Most people will want
to continue using processes as their method of parallelizing because
it's most robust. However some people will want to use threads because
with an addon they can do fancy synchronization between the isolates.
The isolates feature is to enable advanced parallelizing techniques in
Node addons.

[Liam]

On Nov 17, 12:53 pm, Tomasz Janczuk <tom...@janczuk.org> wrote:
> I agree the benefit would greatly diminish for memory-bound node
> applications since the shared global state of the process would be a
> very small fraction of the private non-shared state across isolates. I
> am curious if you have you seen a lot of node apps with large memory
> consumption in the "real world"?

I think it's safe to say that the appeal of Javascript will draw in
developers who push Node into every conceivable role, so "Node is for
x & y sorts of apps" probably isn't a durable assertion. I think we'll
see Node displace Java to some extent, as its ecosystem matures.

It's not hard to eat memory in JS. For instance, it's natural to use
an Object as a key:value cache, and an Array as a queue. I suspect
those aren't the most efficient structures for those needs, but it
will be common practice.

[Liam]

On Nov 17, 1:31 pm, Ryan Dahl <r...@tinyclouds.org> wrote:

Aha. Got it. Good idea.

Synchronization like... sharing data via a Buffer?

[Mark Hahn]

>  He asked a question in a silly way and some of us were pretty rude to him.  

We did not get rude until the skit got really silly.  He kept repeating the same mantra over and over even though he got a lot of replies explaining the same thing to him over and over.  We went for an extended period of time being very polite.

[Liam]

Mark I think you missed your intended thread target :-)

[Mark Hahn]

Oooops.  Sorry.

[Diogo Resende]

On Thu, 17 Nov 2011 13:34:02 -0800 (PST), Liam wrote:
> It's not hard to eat memory in JS. For instance, it's natural to use
> an Object as a key:value cache, and an Array as a queue. I suspect
> those aren't the most efficient structures for those needs, but it
> will be common practice.

I agree, but that could change if the core had a specific struct to
handle those 2 kinds of "bad" usage (something similar to Buffer) or
perhaps a C module..

---
Diogo R.

[Dean Landolt]

<3 

 

Aha. Got it. Good idea.

Synchronization like... sharing data via a Buffer?

Synchronization like sharing deeply frozen data structures, or some kind of pass-and-forget reference semantics, or some other interesting model.

[Ted Young]

Shouldn't the node list be single threaded, to prevent these kinds of user errors?

On Nov 17, 2011, at 2:16 PM, Mark Hahn wrote:

[Tomasz Janczuk]

Regarding isolates and domains: is it the intent that when code
running in an isolate fails (e.g. unhandled exception), domains will
allow node to clean up global state associated with that isolate only,
and as a result leave remaining isolates running? Or is the entire
process going down if code in a single isolate fails?

On Nov 16, 1:36 pm, Ryan Dahl <r...@tinyclouds.org> wrote:
> We're just starting development on the new branch of Node. New
> development continues in the master branch - bug fixes go into the
> v0.6 branch. The current target for a v0.8 release is early January.
> We will continue weekly v0.6 releases throughout v0.8 development.
>
> Node v0.8 roadmap
> -----------------
>
> - Move the remaining platform functions from src/platform-*.cc into
> libuv. (Everyone)
>  https://github.com/joyent/node/issues/2132
>
> - Get rid of WAF. All platforms will build using GYP. (Ben, Ryan)
>  https://github.com/joyent/node/issues?labels=gyp&sort=created&directi...

>
> - Isolates. Node will allow users to spawn "child processes" that actually
>   run in a thread. We have to get rid of all the global variables in node.
>   Compiled extensions need know which isolate they're targeting, and we need
>   to decide if we want to load the extension multiple times or just once.

>   Also, some changes to libuv are necessary, since we will have to completely
>   clean up a loop. Finally we'll have to deal with debugging a multi-threaded
>   node process. (Ben, Ryan)
>  https://github.com/joyent/node/issues/2133

>
> - Domains. Domains provide a lightweight isolation mechanism for all i/o
>   related to a particular network connection (e.g. an incoming http request).
>   If an unhandled error is encountered, all i/o local to that particular
>   domain is canceled and all handles are cleaned up. An experimental
>   implementation can be found athttps://github.com/joyent/node/commits/domains.
>   Some of the early work for domains will be used in Isolate support (e.g.
>   cleaning up handles when an Isolate is killed) (Bert, Ryan)
>  https://github.com/joyent/node/issues/2134
>

> - Better allocator. Currently node uses a memory pool allocator to provide
>   read buffers for incoming data streams. This allocator is only
>   reasonably efficient for network connections and local pipes on Unix, and on
>   Windows when zero reads are used. For file i/o and buffered reads on Windows
>   we need a better story. (Igor)
>  https://github.com/joyent/node/issues/2135
>
> - Addons. We need to define an easy and suggested way of building
>   extensions, which should be similar across all supported platforms. (Everyone)
>  https://github.com/joyent/node/issues/2136

[Ryan Dahl]

On Nov 17, 2011 8:43 PM, "Tomasz Janczuk" <tom...@janczuk.org> wrote:
>
> Regarding isolates and domains: is it the intent that when code
> running in an isolate fails (e.g. unhandled exception), domains will
> allow node to clean up global state associated with that isolate only,
> and as a result leave remaining isolates running? Or is the entire
> process going down if code in a single isolate fails?

If one isolate fails we will continue execution. We need the ability
to clean up and open handles and I/O requests. The current plan is
this (subject to change): each isolate has many domains, each domains
has many handles. When an isolate goes down it kills all of its
domains, which in turn kills all of its handles.

The intent with domains is simply to give the user more insight and
better control over the I/O they are preforming by grouping related
handles together. For example, if timers are set up during the process
of handling an HTTP request and the HTTP connection is terminated
because a peer hit the stop button on their browser, it would be nice
to be able to automatically cancel those timers.

[Mikeal Rogers]

>
> If one isolate fails we will continue execution. We need the ability
> to clean up and open handles and I/O requests. The current plan is
> this (subject to change): each isolate has many domains, each domains
> has many handles. When an isolate goes down it kills all of its
> domains, which in turn kills all of its handles.
>
> The intent with domains is simply to give the user more insight and
> better control over the I/O they are preforming by grouping related
> handles together. For example, if timers are set up during the process
> of handling an HTTP request and the HTTP connection is terminated
> because a peer hit the stop button on their browser, it would be nice
> to be able to automatically cancel those timers.

This is a great explanation. +1

[geddesign]

You should consider using [Trello](http://trello.com) for the Node roadmap. That would be a nice way of letting people know what's coming and what's being worked on, and devs can vote on the things in the backlog they'd love to see implemented.

[Liam]

On Nov 17, 9:36 pm, Ryan Dahl <r...@tinyclouds.org> wrote:
> If one isolate fails we will continue execution. We need the ability
> to clean up and open handles and I/O requests. The current plan is
> this (subject to change): each isolate has many domains, each domains
> has many handles. When an isolate goes down it kills all of its
> domains, which in turn kills all of its handles.

What constitutes failure of an isolate? Uncaught exceptions, and...?

[Paddy Byers]

Hi,

What constitutes failure of an isolate? Uncaught exceptions, and...?

I think it would be any exit of an isolate, under either normal conditions (process.exit(), or no remaining watchers) or abnormal conditions (ie uncaught or fatal exception). The isolate lifecycle should be indistinguishable from that of any normal node process (with the exception of handling of external signals, because it would not be possible to direct a signal at a single isolate only).

Paddy

[Tomasz Janczuk]

Besides IO and open handles, is there anything else that gets clean up
when an isolate goes away? For instance, what happens with other
isolates or processes spawned by that isolate?

There is a certain level of resemblance between isolates and
processes, I wonder to what extent the behavioral model of am isolate
and a process will be reconciled in node? Specifically, when I write a
node app, do I need to be explicit if it is to run in a process or an
isolate, or will node runtime provide an environment that abstract
away the differences?

[Paddy Byers]

Hi,

Besides IO and open handles, is there anything else that gets clean up
when an isolate goes away? For instance, what happens with other
isolates or processes spawned by that isolate?

There is a certain level of resemblance between isolates and
processes, I wonder to what extent the behavioral model of am isolate
and a process will be reconciled in node? Specifically, when I write a
node app, do I need to be explicit if it is to run in a process or an
isolate, or will node runtime provide an environment that abstract
away the differences?

I can't speak for the core team, and I suppose some of the details are still under discussion. But for my application I interpreted the need to be:

- behave identically to processes wherever possible;

- be explicit about the differences in behaviour when you can't.

So, if an isolate spawns a child process, the child continues to run when the isolate exits. Similarly, if an isolate forks another isolate, the child will continue to run once if the parent exits. For most applications, there will be no difference seen by the application code between the isolate and process case.

Those things that are process-wide - such as process.setuid() - cannot be faithfully emulated in isolates, so the action would apply to all isolates.

env and cwd are process-wide but you could make some kind of effort to fake these as being independent between multiple isolates in a process. In the case of cwd, I've made no effort to do that - a call to chdir() will take effect for all isolates in a process - but for my use-case this was irrelevant. In the case of env I have the usual process-wide env, and local isolate overrides that are initialised by the env passed in to fork(). There is a well-defined behaviour for these, but they are different in some situations from the conventional case; for example, they are not transitive (ie a child of a child doesn't inherit the local overrides). Again, that's enough for my use-case, but I don't know if that's a useful behaviour in the general case.

Paddy

[Dean Landolt]

On Fri, Nov 18, 2011 at 1:26 PM, Paddy Byers <paddy...@gmail.com> wrote:

Hi,

Besides IO and open handles, is there anything else that gets clean up
when an isolate goes away? For instance, what happens with other
isolates or processes spawned by that isolate?

There is a certain level of resemblance between isolates and
processes, I wonder to what extent the behavioral model of am isolate
and a process will be reconciled in node? Specifically, when I write a
node app, do I need to be explicit if it is to run in a process or an
isolate, or will node runtime provide an environment that abstract
away the differences?

I can't speak for the core team, and I suppose some of the details are still under discussion. But for my application I interpreted the need to be:

- behave identically to processes wherever possible;

- be explicit about the differences in behaviour when you can't.

So, if an isolate spawns a child process, the child continues to run when the isolate exits. Similarly, if an isolate forks another isolate, the child will continue to run once if the parent exits.

What are you implying here? I'm under the impression that regardless of whether your in an isolates environment or not, the child_process module will continue to spawn real os processes. At least, I sure hope. The whole isolates pseudo-process worker thing is great and all, but it should get its own API. The child_process API (other than being unfortunately named) is just fine -- there's no need to remove this feature (and I don't believe the core team intends to).

 

For most applications, there will be no difference seen by the application code between the isolate and process case.

Those things that are process-wide - such as process.setuid() - cannot be faithfully emulated in isolates, so the action would apply to all isolates.

 

I don't believe the intent of isolates is to emulate processes. It's an abstraction over processes. Maybe it'll be easier if we just called them "workers" -- and we can say there are thread-backed workers and process-backed workers -- both would have a shared API surface. Which one you choose just depends on your application's needs (thread-backed workers could enable efficient memory sharing tricks, process workers on *nix could open the door to CoW fork tricks).

Those in the know: is this a fair characterization?

 

env and cwd are process-wide but you could make some kind of effort to fake these as being independent between multiple isolates in a process. In the case of cwd, I've made no effort to do that - a call to chdir() will take effect for all isolates in a process - but for my use-case this was irrelevant. In the case of env I have the usual process-wide env, and local isolate overrides that are initialised by the env passed in to fork(). There is a well-defined behaviour for these, but they are different in some situations from the conventional case; for example, they are not transitive (ie a child of a child doesn't inherit the local overrides). Again, that's enough for my use-case, but I don't know if that's a useful behaviour in the general case.

Paddy

--

[Paddy Byers]

Hi,

What are you implying here? I'm under the impression that regardless of whether your in an isolates environment or not, the child_process module will continue to spawn real os processes. At least, I sure hope.

Yes, you're right - if I implied otherwise, it wasn't intended.

 

The whole isolates pseudo-process worker thing is great and all, but it should get its own API. The child_process API (other than being unfortunately named) is just fine -- there's no need to remove this feature (and I don't believe the core team intends to).

I agree. (But .. child_process.fork() is already a different API from spawn(), exec() etc, and it has an options argument that could be used to specify one behaviour or another. But the core team might choose to expose the functionality in a different way.)

 

I don't believe the intent of isolates is to emulate processes. It's an abstraction over processes. Maybe it'll be easier if we just called them "workers" -- and we can say there are thread-backed workers and process-backed workers -- both would have a shared API surface. Which one you choose just depends on your application's needs (thread-backed workers could enable efficient memory sharing tricks, process workers on *nix could open the door to CoW fork tricks).

I agree here as well. I wasn't saying that the idea is to emulate processes; only to emulate already-well-understood behaviour that we get today with process-backed workers wherever it is possible and makes sense, so existing modules and applications are broadly agnostic to the environment they're run in.

But as I said at the start, don't take my word for it; it was just a personal take on it.

Thanks - Paddy

[Ben Noordhuis]

Yes.

[George Snelling, Seattle]

Ryan, 

I don't understand how the core team prioritizes outstanding bugs/issues vs new work. See

https://github.com/joyent/node/issues?direction=desc&labels=&milestone=&page=1&sort=created&state=open

Am i searching the wrong query?   From that link today with my perms on github it looks like there are 400+ issues with no milestone assigned.  

Please consider something like "Fix Bugs, Performance, Refactor" as themes for 0.8.  No new features.  Finish the features you already have.  

Thanks, 

-George

[Mark Hahn]

>  Please consider something like "Fix Bugs, Performance, Refactor" 

Wouldn't that be done in 0.7?

[Liam]

Lots of them are feature requests which the core team considers low-
priority.

On Dec 7, 1:48 am, "George Snelling, Seattle" <georg...@gmail.com>
wrote:

> Ryan,
>
> I don't understand how the core team prioritizes outstanding bugs/issues vs
> new work. See
>

> https://github.com/joyent/node/issues?direction=desc&labels=&mileston...

[jmar777]

Low-risk bug fixes generally get back-ported into stable branches as
well (i.e., 0.6) as long as they don't require significant refactoring
or introduce API changes.

[Ryan Dahl]

Yes, there are many outstanding issues - but they vary widely in
importance. We have a good handle on how stable the code bases is and
we communicate that stability via version numbers - not via open
GitHub issues. If someone posts an issue for "Support auto detecting
OpenSSL in OpenBSD" we will not close it - but we also don't jump to
fix it. Our features for v0.8 are very minimal in from the user's
perspective and largely amount to simplifying and cleaning the code
base (isolates, domains).

[john.tiger]

On 06/25/2012 06:43 PM, GuoQing Deng wrote:

Free Node.js eBook update: http://www.heronote.com/files/Node.js.htm

free but in a non-free compiled html format ?  hope you can publish it in an open source web standard  format like regular html / css or even pdf

[Mark Hahn]

> Free Node.js eBook update: http://www.heronote.com/files/Node.js.htm

I downloaded the .chm version and opened it in win 7.  The contents on the left look ok but the right pane just says "Navigation to the webpage was canceled".

 

[Jorge]

On 26/06/2012, at 02:43, GuoQing Deng wrote:

> Free Node.js eBook update: http://www.heronote.com/files/Node.js.htm

> "Furthermore, users of Node are free from worries of dead-locking the process—there are no locks."

Neither in a bunch of httpd processes.

> "But what about multiple-processor concurrency? Aren't threads necessary to scale programs to multi-core computers?"

-YES: Threads are necessary to scale programs to multi-core computers.

> "Processes are necessary to scale to multi-core computers"

-Processes *are* threads.

> "not memory-sharing threads."

-Processes *are* threads running in isolated memory spaces, and it's so for security reasons *not* for performance reasons.
-The ability to share memory is always a plus.
-The threads in a process share what they want to share and if they want to. They can also share nothing as processes do.

> "The fundamentals of scalable systems are fast networking and non-blocking design—"

-But node uses multiple threads itself both to *parallelize* IO and to avoid blocking on IO. But it doesn't expose any threads API to its users.

> "the rest is message passing."

-There's no fastest way to pass a message than via a pointer and you can't do that across processes but you can do it across threads.

> "In future versions, Node will be able to fork new processes (using the Web Workers API ) which fits well into the current design."

-In other words, to exploit multiple cores node will do just what everybody else has been doing until now, say, httpd: fork itself.


threads_a_gogo(*) gives node users the power to parallelize the execution of their javascript code within a single node process. Node couldn't benefit before threads_a_gogo of multi-core cpus simply because it was lacking a threads API. But threads_a_gogo gives you one:

(*) <https://github.com/xk/node-threads-a-gogo/>
--
Jorge.