Node.js vs. Scala - "Scaling in the large"

[Srirangan]

Hi,

I'm looking for responses to the points raised here.

- Sri

--
Srirangan  |  About  Blog  GitHub  LinkedIn  Twitter

[Mark Hahn]

The author implies that if you use Node you are locked into Node.  This is a big assumption.  Why is this true more for node than scala?

This isn't true for me.  I've used node in combination with Apache and I've used processes with threads inside a node-based app.  I'm also keeping ngen in my arsenal of tools for when I need it.

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[Liam]

Thanks for the link. It's a good read with a very valid point; Node's
not a silver bullet for massive-scale contexts.

It IS designed to be more efficient and easy-to-use than the other
readily-available options for network apps -- PHP, RoR, Python, et al.

On Sep 2, 8:55 pm, Srirangan <sriran...@gmail.com> wrote:
> Hi,
>
> I'm looking for responses to the points raised

> here<http://al3x.net/2010/07/27/node.html>
> .
>
> - Sri
>
> --
> Srirangan  |  About <http://srirangan.net/about>  Blog<http://srirangan.net>
>   GitHub <https://github.com/Srirangan>
> LinkedIn<http://www.linkedin.com/in/srirangan>
>   Twitter <http://twitter.com/srirangan>

[Mo Cheng]

Scalability is basically a economic problem. If your system can serve billions of visits with acceptable cost, it really doesn't matter your system is on Node.js or Apache or anything else.

The cost includes:

1) engineering cost. e.g. wages of engineers

2) operation cost. e.g. electronic and hardware bill 

Node.js may help to have same single machine serve more visits due to asynchronous IO, but it may not reduce other cost such as engineering cost.

Thanks,

-Morgan

--

[Floby]

I'm not sure I agree with this sentence :
" threads are constructs for a single computer, and events are
constructs for a single CPU"

On Sep 3, 10:08 am, Mo Cheng <morgan.chen...@gmail.com> wrote:
> Scalability is basically a economic problem. If your system can serve
> billions of visits with acceptable cost, it really doesn't matter your
> system is on Node.js or Apache or anything else.
>
> The cost includes:
> 1) engineering cost. e.g. wages of engineers
> 2) operation cost. e.g. electronic and hardware bill
>
> Node.js may help to have same single machine serve more visits due to
> asynchronous IO, but it may not reduce other cost such as engineering cost.
>
> Thanks,
> -Morgan
>
>
>
>
>
>
>
> On Sat, Sep 3, 2011 at 11:55 AM, Srirangan <sriran...@gmail.com> wrote:
> > Hi,
>

> > I'm looking for responses to the points raised here<http://al3x.net/2010/07/27/node.html>
> > .
>
> > - Sri
>
> > --

> > Srirangan  |  About <http://srirangan.net/about>  Blog<http://srirangan.net>
> >   GitHub <https://github.com/Srirangan>  LinkedIn<http://www.linkedin.com/in/srirangan>
> >   Twitter <http://twitter.com/srirangan>
>

[Mo Cheng]

I'd like to say:

Thread should be system level concept; events are concept of application level.

-Morgan

[Joe Developer]

I agree that scaling is hard, that is why it pays to 'scale' as little as possible - which is where doing more on less pays dividends. 

[ambert ho]

That blog post is stating nothing controversial - it's pretty much done in the same manner as the posts done by many DBAs in a similar admonishing counterpoint to the "nosql fanboy" movement.

Most experienced developers know this already, I think the authors of these blog posts are just irritated by the type of person who says "MongoDB is faster because it's in memory" or "CouchDB is faster because you don't need to join data." <--- and "Node.js scales because it doesn't use threads" is a similar irritating statement.

Ambert

[Nicholas Campbell]

Its not that those statements are necessarily incorrect but they aren't substantiated. They leave out a massive amount of additional information- information that would be useful to validate such claims. It is the same as fighting against someone who finds a single usecase where a technology doesn't work well and claims that the tech is terrible when if fact its terrible for that usecase.

Mostly though it is that people will make those claims without having done any research. Pick the right tool for the job.

- Nick Campbell

http://digitaltumbleweed.com

[Naouak]

The author implies that if you do web developpement, you don't know a shit about concurrency and is insultingg web developpers (implying they are dumber tham regular programmers).

The author never mentions the javascript workers : the way to do threading in javascript. Most of his argument about if you are using node, you are bind to use event based is not relevant considering workers.

Le 3 sept. 2011 07:03, "Mark Hahn" <ma...@boutiquing.com> a écrit :
> The author implies that if you use Node you are locked into Node. This is a
> big assumption. Why is this true more for node than scala?
>
> This isn't true for me. I've used node in combination with Apache and I've
> used processes with threads inside a node-based app. I'm also keeping ngen
> in my arsenal of tools for when I need it.
>
> On Fri, Sep 2, 2011 at 8:55 PM, Srirangan <srir...@gmail.com> wrote:
>
>> Hi,
>>

>> I'm looking for responses to the points raised here<http://al3x.net/2010/07/27/node.html>
>> .
>>
>> - Sri
>>
>> --

>> Srirangan | About <http://srirangan.net/about> Blog<http://srirangan.net>
>> GitHub <https://github.com/Srirangan> LinkedIn<http://www.linkedin.com/in/srirangan>
>> Twitter <http://twitter.com/srirangan>
>>

[Marco Rogers]

The author implies that if you do web developpement, you don't know a shit about concurrency and is insultingg web developpers (implying they are dumber tham regular programmers).

The author isn't insulting anybody except those who don't take the topic seriously and buy into pre-canned answers. Those people, along with people who read too much into things and get defensive for no reason, are the primary reason we can't have helpful discussions about these things.

:Marco

[Branko Vukelić]

On 2011-09-03 02:31 -0700, Floby wrote:
> I'm not sure I agree with this sentence :
> " threads are constructs for a single computer, and events are
> constructs for a single CPU"

I'm not an expert, but I believe it goes something like threads that run
on multiple CPU cores as opposed to event loop that runs in a single
thread (and therefore on a single CPU core).

--
Branko Vukelic
bra...@brankovukelic.com
bra...@herdhound.com

IDEA MACHINE
www.brankovukelic.com

Lead Developer
Herd Hound (tm) - Travel that doesn't bite
www.herdhound.com

Love coffee? You might love Loveffee, too.
loveffee.appspot.com

[Ted Young]

I don't think he's being insulting.

The real point he's making is pretty straight forwards: in node, there is only one concurrency model.  A number of other platforms offer multiple concurrency models.  If you want access to one of those other models down the line, you will have to carve off that part of your application and rewrite it in another language.  And there's definitely an added cost in throwing another language into the mix, depending on what parts of your application need to be ported it could be way more of a rewrite than if you could write it all in one language.  So... why not use a platform that already supports all of the concurrency models? I.E. Scala on the JVM?  It's a valid enough point when choosing a platform, if you can't predict the problems your application is going to hit 2 years down the line.

How much of this can be mitigated by writing C++ addons?  What of the possibility of writing threaded and actor-based components in C++?  Node already exposes a number of treaded activities in this manner, but the documentation indicates that it's a painful process, and I've never written one.  I don't have a big ego, so it's easy for me to ask naive questions like this.  :)

Also, one last thing: I find node encourages me to break my applications up into small, independent components, and asynchronous api's make it easy to refactor a component to use a different communications protocol.  So adding components in other languages that run in separate processes is potentially easier with node, as it's already pushing you in that direction.

Ted

[Branko Vukelić]

On 2011-09-03 16:31 -0700, Ted Young wrote:
> I don't think he's being insulting.

Agreed.

> The real point he's making is pretty straight forwards: in node,
> there is only one concurrency model. A number of other platforms
>

> SNIPPAGE

Yes, author does make a valid point, and I fully agree. Yet I feel most
software engineers get trapped in technical discussion without ever
realizing the simple truth: we are using Node, Scala, Erlang, whatever,
because we LOVE the tools we use. It makes us happy, excited, and more
productive thanks to the happiness. When we are no longer happy, we move
on, and find a new tool that makes us even more happy. If the tools fail
us in strictly technical sense, it also makes us unhappy, and we look
for new ones.

Software engineers may seem more rational than non-programmers, but we
are all humans after all, and we act irrationally most of the time. Why
do you think there are so many different tools out there? If it weren't
for this particularly irrational aspect of people in our industry, we
would probably have one language that has everything we need and we
wouldn't be arguing about leading commas and curly braces.

[Rob Evans]

I'd like to jump in and make a few points if I may.

I am the developer over at http://www.isogenicengine.com, a JavaScript game engine with built-in multiplayer networking that runs in both front and back-end from the same code base using Node.js for server-side execution.

As you can probably imagine, scalability is a pretty damn important issue when designing a network-based game engine. Isogenic needs to be fast and efficient on both sides of the network connection and able to handle many concurrent clients.

I actually agree completely that it's not about which tool is better or worse "overall"... but rather that you need to pick one for your specific use case.

You can see a diagram of how Isogenic Engine is designed to handle multiple client connections here: http://www.isogenicengine.com/documentation/server-layout-diagram/

So far we have tested the connections up to just over 8,000 active players all in the same world, running across between one and three game servers at a time.

We are able to efficiently scale our world by firing up new game server instances that are running in their own thread and communicating with each over over TCP/IP. Persistent world changes are stored in MongoDB so that if a server instance dies or gets killed the game data is restored once the instance is brought back up.

As each new instance is brought up, it announces itself to the others and begins to accept client connections. The main management system can automatically fire up new instances so physical servers can run multiple Node.js game server instances on request from the control server.

The game being tested against is a city-builder demo called IsoCity that has a basic functional capability to allow players to place, move and remove entities from the city world which is kept in sync across all game instances and changes are communicated to connected clients so that they all see the same simulation.

Whilst around 8,000 active players might sound low to you, there is no reason why this cannot scale indefinitely using the current pattern by adding more hardware and firing up more game server instances. Connecting clients are directed to the game server with the least amount of load so that connections are distributed evenly, whilst the control server is kept informed of the number of connections each game server has and can issue fire-up commands to new instances as required.

The whole thing has been a total learning experience for me and although I do not claim to be an expert in all things scale-related, I have single-handedly deployed data analysis systems at my day-job that are used by practically every major bank in the world to do real-time intensive market analysis (not in Node.js for clarity).

Node.js DOES have it's uses and scaling (whilst not entirely automatic) can be achieved with some know-how and fore-thought.

Once Isogenic has been battle-tested with 1 million active players... then I might be telling a very different story but I'm confident at the moment that we're on the right track.

[Malte]

The author just hacking around one thesis made by ryan, that you can
easy build scaling systems. Which is true when you have one server...
and even it is not true the benchmarks reference in the article don't
show that it is slower. Overall I would prefer a system where I don't
have to care about concurrency and in normal web development you don't
want to have to. All of the web frameworks I know from other systems
hide this currency part away from the developer.

There are so many other points he never said a word about... He sound
like the typical java developer, which I was also before and I love
Java! But...

1. You have only one code base. You can share the same code on both
sides and object serialization was never that easy.
2. Javascript is a great dynamic language (and I came from the Python/
Java world) when you have understand the language and don't try to use
the bad parts ;)
3. When you have master Javascript you can easily support your GUI
developers or your GUI developers can support you on the server side
or even better it is the first time that both sides understand each
other that easily. That can make your development process much more
agile. Try this with your java developers ;)
4. The performance/development time ratio I think is currently the
best. I can benefit from reusing code and the specialization in
development of web/network applications and can have a high scalable
system.

On 8 Jan., 18:07, Rob Evans <coolbloke1...@gmail.com> wrote:
> I'd like to jump in and make a few points if I may.
>

> I am the developer over athttp://www.isogenicengine.com, a JavaScript game