Sane arguments for when you shouldn't use node.js? (no trolls please)
John Hamelink
the super-trolling ( http://teddziuba.com/2011/10/node-js-is-cancer.html
), I'd like to get to the bottom of the question: when is it not right
to use node.js and why? Also, what significant drawbacks to the
language are there, that are unique to node?
Again, no trolls please - I don't want to start a flamewar, I simply
want to broaden my understanding of node's appropriateness.
Ben Noordhuis
- Serious number crunching à la NumPy. JS as a language is poorly
suited for it and Node inherits that shortcoming. It's not impossible
but you could pick a better solution.
- COBOL-style batch jobs. You can do that with Node but it's overkill. Use Perl.
- Real-time applications (hard real-time, not "web" real-time). I
suppose that's a shortcoming that (almost?) all scripting languages
have.
- Running in really constrained environments. Your home router or
smartphone probably doesn't count - my Nokia N900 has more muscle than
my 2000 desktop machine.
Micheil Smith
> - Real-time applications (hard real-time, not "web" real-time). I
> suppose that's a shortcoming that (almost?) all scripting languages
> have.
What makes this so? I'm interested to know more here.
– Micheil.
Diogo Resende
Correct me Ben if I'm wrong but you're talking about really real-time
apps,
that usually need a real-time kernel. These kind of apps are not suited
for
probably any scripting language because there are constraints in this
apps
that these languages (still) don't have. Like:
"If an event is not processed in a certain time (ms), it's not worth
being
processed and should that should be logged/treated in a different way."
Real-life example: robots.
---
Diogo R.
Matt
- Serious number crunching à la NumPy. JS as a language is poorlysuited for it and Node inherits that shortcoming. It's not impossible
but you could pick a better solution.
friesnow
http://blog.nodejs.org/2011/10/04/an-easy-way-to-build-scalable-network-programs/
The comments and several linked success stories have more discussion.
On Oct 6, 7:41 am, John Hamelink <s0l1dsnak3...@gmail.com> wrote:
> Following on from the "meta-programming?" thread, and in the face of
Ben Blair
anything that requires sub-millisecond average latency.
True hard real-time systems have to provide latency guarantees (think
fly-by-wire systems in an airplane). Languages without static typing,
or run-times with garbage collection cannot provide that level of
predictability.
There's a huge area between hard real-time and web real-time (where
you're usually limited by internet latency in the 10s of
milliseconds). To me, this is were it things are currently very
interesting. We've got a distributed event processing system with
round-trip latencies well under half a millisecond. In principle,
Node.js could absolutely perform at this level. But we've found it's
not quite there yet.
I should be clear that we're not just copying TCP buffers. If you're
decoding messages out of your buffer and applying some logic to them,
javascript will necessarily spend time doing property lookups and
existence checks. Basically dealing with duck-typing overhead. With a
good JIT like V8 or SpiderMonkey, this isn't much time, but it is a
lot more than zero. The semantics of the language prevent even a
tracing JIT like SpiderMonkey from making the kind of assumptions that
a developer writing in a staticly typed language can make.
In our case, we don't know at compile time what our message structures
will be, so we dynamically generate an optimized wire protocol and
codex for each new message type we see. That's expensive, but we only
do it once, then amortize that over hundreds of thousands of
subsequent messages each second. This works well for us and is not
something you could do in pure Node.
We're working on some benchmarks comparing messaging latency and
throughput in Node.js, C++ and .NET implementations. I'll post to the
list when we have the numbers ready.
On the other hand, I want to emphasize that there are really very few
situations like this where the penalty for being duck typed matters.
We're using Node as the web gateway to our system and it works
wonderfully for that purpose. In general, I'd say always use something
easy like Node first, then only try harder things when you can prove
that the easy way won't work for you.
- Ben
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
Daniel Ly
Isaac Schlueter
1. Database. While node might be great for managing the logic to keep
a ring or cluster alive and keep track of which nodes have which bits
of data in a distributed hash table, it's a bad choice for doing the
file system jiggery pokery, imo, especially as the files get larger
than JavaScript's maximum integer precision boundary. Better to use a
backend written in C, and access it via a compiled module, or message
passing.
2. Protein-folding, image processing, video encoding, SETI data
processing, and other large-scale cpu-intensive tasks. These things
really need to be written in C, probably split across multiple
threads. That's not to say you couldn't have a node program that
dispatched jobs to such a system, and then served the results to a
client.
3. Anything else where the CPU utilization is going to introduce
significant latency that will affect the IO responsiveness of the
system.
Node is best for IO-bound applications that use core web technologies.
Many many applications are IO-bound. I would suggest that the term
"network program" really *only* properly refers to IO-bound
applications that use core web technologies, so calling it "a system
for network programming" makes a lot of sense.
On Thu, Oct 6, 2011 at 15:22, Daniel Ly <nal...@gmail.com> wrote:
> Anything which requires 64 bit integer math. Javascript does not have 64 bit
> integer.
>
Marak Squires
--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
Isaac Schlueter
I think, a lot of times, you have to make your technology decisions
based on the community that you want to attract. If you're looking to
attract people who are really passionate about JavaScript and the web,
then node might be a good choice for that reason. However, if you're
more interested in stability, development cost, and predictability,
than creativity and cutting-edge functionality (which is certainly
valid in many cases) then you might be better off with PHP, Rails, or
.NET.
Joshua Kehn
Thomas Shinnick
On Thursday, October 6, 2011 6:03:04 PM UTC-5, Isaac Schlueter wrote:
I would not write the following sorts of applications in node:1. Database. While node might be great for managing the logic to keep
a ring or cluster alive and keep track of which nodes have which bits
of data in a distributed hash table, it's a bad choice for doing the
file system jiggery pokery, imo, especially as the files get larger
than JavaScript's maximum integer precision boundary. Better to use a
backend written in C, and access it via a compiled module, or message
passing.
Isaac Schlueter
start splitting files up to keep them under a 2^50 bytes, or rewrite
the fs lib to use some kind of bigint tuple or ctype int64 buffers, or
you'll have to juggle bunches of tiny files if you keep each record in
a separate file, and so on.
In the end, the complexity (imo) is not worth it, since gains of
working in a friendlier-than-c language would start to add complexity
to your program rather than reduce it. Better to do the lowlevel
lifting in C, and then use node to manage the cluster communication
and api layer.
Just saying "This will only work up to 2^50 bytes" is not a great
story for something that calls itself a "database".
Thomas Shinnick
Right, but that means that your database is either going to have to
start splitting files up to keep them under a 2^50 bytes, or rewrite
the fs lib to use some kind of bigint tuple or ctype int64 buffers, or
you'll have to juggle bunches of tiny files if you keep each record in
a separate file, and so on.In the end, the complexity (imo) is not worth it, since gains of
working in a friendlier-than-c language would start to add complexity
to your program rather than reduce it. Better to do the lowlevel
lifting in C, and then use node to manage the cluster communication
and api layer.Just saying "This will only work up to 2^50 bytes" is not a great
story for something that calls itself a "database".
>> ... bad choice for doing the
>> file system jiggery pokery, imo, especially as the files get larger
>> than JavaScript's maximum integer precision boundary.
Oh, and fershur when I said "cavilling caveators" I was describing myself... :-)
Jorge
On 07/10/2011, at 04:12, Isaac Schlueter wrote:
> Right, but that means that your database is either going to have to
> start splitting files up to keep them under a 2^50 bytes, or rewrite
> the fs lib to use some kind of bigint tuple or ctype int64 buffers, or
> you'll have to juggle bunches of tiny files if you keep each record in
> a separate file, and so on.
>
> In the end, the complexity (imo) is not worth it, since gains of
> working in a friendlier-than-c language would start to add complexity
> to your program rather than reduce it. Better to do the lowlevel
> lifting in C, and then use node to manage the cluster communication
> and api layer.
>
> Just saying "This will only work up to 2^50 bytes" is not a great
> story for something that calls itself a "database".
JFTR, the limit is 2^53, which is 8192 terabytes.
n=0
do {
a= Math.pow(2,++n);
if (a === a+1) break;
} while (1)
9007199254740992
n
53
Math.pow(2,53)/(1024*1024*1024*1024)
8192
--
Jorge.
Jorge
> 3. Anything else where the CPU utilization is going to introduce
> significant latency that will affect the IO responsiveness of the
> system.
Just put any cpu-bound task (for example filling a template) that takes over 1ms in the request handler, and it will slow down node to under 1k reqs/s.
You guys need to find a (good, cheap, efficient) way to send these kind of tasks to the background.
Because for node to run fast, its event loop must be spinning fast, and for that to happen cpu-bound tasks must be run in parallel, asynchronously, just like the io.
runInBackground(taskFunction, dataObject, callback)
--
Jorge.
Alejandro Pimentel
--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
Atte. ISC Alejandro Pimentel Ríos
.::Digitalizando tus sueños::.
XIMA Productions
Tel. 26160580 y 86364234
-------------------------------------------
Maybe you aren't a shadow on the water...
...but instead, a fish that breaches the water's surface.
Garann Means
expand on why not? Does it have to do more with retrieving the data to
populate the templates, or with templating/markup generation itself?
Liam
number of load-balanced concurrent node instances. Isn't that a good
solution for the problem you raise?
Dean Landolt
@Jorge, Ry stated that in near-future, we'll be able to specify a
number of load-balanced concurrent node instances. Isn't that a good
solution for the problem you raise?
Dean Landolt
Joshua Kehn
Liam
subordinate thread/process vs doing it inline on one of several load-
balanced processes running the same code?
In neither case do you stall the main request processor.
Matt
Is there a difference between handing off cpu-intensive stuff to a
subordinate thread/process vs doing it inline on one of several load-
balanced processes running the same code?
In neither case do you stall the main request processor.
Liam
> > Is there a difference between handing off cpu-intensive stuff to a
> > subordinate thread/process vs doing it inline on one of several load-
> > balanced processes running the same code?
>
> > In neither case do you stall the main request processor.
>
> It's not about stalling the event loop. That's a solved problem. The problem
> is in passing data between the two, requiring copies to be made, and
> potentially sending large amounts of data over pipes.
processes, unless perhaps to pull data from a shared cache service.
A cache service which delivers data via mmap/signal could be accessed
by a C++ Node module. Is there a common mmap-based IPC protocol for
such things? If not, it's really up to the service-builder to
implement such a thing.
Rob
although it is often misused that way. It means that time as a
parameter or constraint has entered into the application (semantics).
In terms of timing constraints, there are inherent reasons in the
application (due to the physics, human sensory limitations,
psychological tolerance of delay, regulatory requirements, etc) that
the response to an event must be completed within a maximum period of
time time. This applies both to input sampling and computation of
outputs. The time limitations can be submicroseconds for certain kinds
of signal processing, microseconds for a some controls systems,
milliseconds, seconds, minutes, hours, days, years, etc for other
applications. The point is that you have know those scheduling limits
of the events in advance. There are classes of non-preemptive real
time scheduling algorithms that could be applied to (implemented in)
Node to guarantee adequate performance, although this kind of analysis
is typically not trivial.
Agreed Node under current technologies will not be a good candidate
for fast microsecond applications and likely the prudent application
should stay well into multiples of milliseconds and seconds at a
minimum to be safe. But it is up to the designers and implementers of
both preemptive and non-preemptive real time systems to do the
scheduling analysis to guarantee adequate performance.
In practice, most systems simply run fast enough and the designers
have sufficient experience or general knowledge exists that they do
not run (too often) into the timing constraints. Benchmarking is a
form of testing and while useful for proving a system is too slow by
observation, it can never by itself prove that the system is fast
enough. In other words the vast major of systems are built using
heuristics/experience. If you have sufficient resources to stay away
from borderline troubling cases, you win. The discussion here is about
those rules of thumb for Node. But remember as the Big Bad Troll
pointed out, if your feelings about how long it will take to complete
a response are too low, you will have trouble. If you are running
close to the effective limits of Node, garbage collection will almost
certainly be the thing that introduces dangerous variation leading to
overrunning acceptable event response. Best not to get near to that
fine a level of timing.
On Oct 6, 7:44 am, friesnow <fries...@gmail.com> wrote:
> Also see:
>
> http://blog.nodejs.org/2011/10/04/an-easy-way-to-build-scalable-netwo...
Bradley Meck
Jorge
Yes I think there's a difference.
Case 1: Say you have a 4 cores machine, and you're load balancing to 4 identical node processes. Your request handler blocks for 5 seconds to complete request A, and 1ms for request B, then it gets 5 requests at once in this order: A,A,A,A,B. All the 4 processes will block for 5 seconds while serving A, so it's going to take more than 5 seconds to dispatch B.
Case 2: You have a 4 cores machine, and a *single* node process than can run background tasks in background threads asynchronously (runInBackground(taskFunction, data, callback)). Your request handler takes 5 seconds to complete for request A, and 1ms for request B, but they're handled async, non-blockingly, just as node does with io. Then you get 5 requests at once in this order: A,A,A,A,B. It spawns 4 background threads to handle reqs of type A, and then handles the request of type B immediately, without having to wait for A to complete.
It's also easier to program when you have a single program context. Say, for example, that these four A requests were identical and coming from the same session/authenticated user. It's easier to deal with that in a single node process, than when they end up in 4 different, separate processes.
Think what would be node's behaviour if it had only the sync io methods. No matter how many node instances you launched, it would still suck badly. That's the scenario wrt cpu-bound tasks.
--
Jorge.
Liam
So, er... why are Ry & Co doing load balancing, rather than a
child_process pool & queue under runInBackground api? (Admittedly, a
userland pool/queue module would be easy to cook up in pure JS.) I
suppose load balancing works across multiple hosts...
Nicolas Chambrier
It may be completely dumb but... The "vm" module couldn't be multicore-capable (maybe it is actually already) ? This would ease the whole context thing...
Isaac Schlueter
What template language are you using? I've used EJS with Express
quite a bit, and it's extremely snappy. Could you share some code or
benchmarks that show it being slow? Otherwise, it seems a bit fuddy.
Matt
> On Oct 7, 8:04 pm, Matt <hel...@gmail.com> wrote:
>> On Fri, Oct 7, 2011 at 10:51 PM, Liam <networkimp...@gmail.com> wrote:
>>> Is there a difference between handing off cpu-intensive stuff to a
>>> subordinate thread/process vs doing it inline on one of several load-
>>> balanced processes running the same code?
>>
>>> In neither case do you stall the main request processor.
>>
>> It's not about stalling the event loop. That's a solved problem. The problem
>> is in passing data between the two, requiring copies to be made, and
>> potentially sending large amounts of data over pipes.
>
> The point of load-balancing is that no data need be shuttled between
> processes, unless perhaps to pull data from a shared cache service.
If you load balance between 1 process per CPU on an 8 CPU box then you just have 8 processes that can get locked up instead of 1. It's still the same problem, just on a bigger scale.
As Ben points out - this isn't a problem that Node can solve (yet). But it's still a better programming environment than most.
rtweed
> So, er... why are Ry & Co doing load balancing, rather than a
> child_process pool & queue under runInBackground api? (Admittedly, a
> userland pool/queue module would be easy to cook up in pure JS.) I
> suppose load balancing works across multiple hosts...
https://github.com/robtweed/Q-Oper8
I agree - this kind of thing needs baking into the lower layers of the
architecture
Dean Landolt
I'm not so sure I buy the claims here about templating being cpu-intensive.
What template language are you using? I've used EJS with Express
quite a bit, and it's extremely snappy. Could you share some code or
benchmarks that show it being slow? Otherwise, it seems a bit fuddy.
Ted Young
templating engine chunk it's processing with nextTick, so that your
slow requests would not block the event loop. That's the goal isn't
it? Processing multiple requests in the same thread without any
context switching or extra serializing?
Ted
Liam
Now... I don't suppose you've considered adding a mmap-based IPC
mechanism to this? That would ship requests & results in Buffers? :-)
rtweed
>
> Now... I don't suppose you've considered adding a mmap-based IPC
> mechanism to this? That would ship requests & results in Buffers? :-)
think you just took me out of my depth :-)
Rob
Dean Landolt
Liam
from it on receive.
Jorge
Kudos to them for addressing case 1, that's a Good Thing™ !
But why do they deny the problem and/or the need to go for 2 as well ? I don't know.
ISTR ry said once that 'it involves too much magic'.
- It's not possible to hand an object from a v8/JS context to another (they'd have to devise a way to do it)
- To achieve shared-nothingness and speed, we'd need a way to pass-and-forget them (too much magic).
- node is not thread-safe (would need its own 'isolates' branch á la v8)
- ... ?
--
Jorge.
Liam
from an mmap, no? And for a separate script instead of a function from
the main script?
Giovanni Giorgi
to manage x subnodes mailbox. Every subnodes can send async request to
others, and query on its queue. You ask The main node server to spawn
anorher subnode sending it a message. All async
Inviato da iPhone
Il giorno 09/ott/2011, alle ore 19.35, Liam <networ...@gmail.com>
ha scritto: