Raft - PaaS - Advice from nodester / nodejitsu / haibu

[Tim Dickinson]

Hey all.

So this is not a ANN but more of an request for advice from nodester / nodejitsu / haibu and the community in general. What i have been working on for the past few months is of sort a PaaS. The basic idea behind it is to create a server to can spawn node apps that are pushed out to it with a cli.

I'm calling it Raft as in a boat to float apps on. It has gone through a few iterations since it creation. It started out as a MVC style app container. The basic app structure was you would have your model's, view's and controller's, and raft would load all these into the app, kinda so you didn't have to code and express server or is server or what have you. As i worked on raft and played around with it more i found that the MVC style was much less dynamic then i would have liked. So from that the current version has evolved.

The current version... OK the current version is now very low leave, in fact it does not do much other then load the app and its module in a context with its own process. the only different between the raft context and plain nodejs context is that you get a global called raft. what the raft object does is gives you http, tcp, express server and so on. these servers are just like the native server but for one difference and that been httpserver.listen, the native httpserver.listen take a port and host, but what the raft httpserver.listen take is a string that is a domain that gets routed to the port of that app. 

OK so like i say this is not an announcement but more a request for advice.

Some of the questions:

I know that nodester is hosted on AWS's and with that how many apps are hosted per server? Are the server like a 8GB/RAM 4/core pc running maybe 20 apps per server, or is is a micro server running just one app?

Nodejitsu are the developer of  haibu, but i dont think that is what they are using for the PaaS. Now on that is their backend a custom build of haibu or is it a whole new module in its own?

To nodester. why did you chose git for the pushing of apps to the backend? 

To nodester / nodejitsu. Have you guys thought of a kind of dynos (heroku style)? if so how would you guys go about doing that? like you spawn 2 processes of the same app and just route request to each app like node does with the cluster module?

To nodester. On average what are your costs running 3000+ app on AWS's? 

OK so this is what im going to ask for now. I do have more question but i would like to see if i get any answers for these ones.

Gota love node!

The code for now. Please note that this is not a release but a Q&A

https://npmjs.org/package/raft

https://github.com/FLYBYME/Raft

Thanks all

Tim

[Joshua Holbrook]

Hi. I know nodejitsu. ;)

> Nodejitsu are the developer of haibu, but i dont think that is what they are using for the PaaS.

Actually, haibu is what runs everyone's apps on nodejitsu! We have a
few closed source extensions for haibu, but aside from that it's all
the same. In fact, I encourage you to use haibu as a starting point
for something like raft.

> Have you guys thought of a kind of dynos (heroku style)?

We have a similar concept, called a drone
(https://github.com/nodejitsu/haibu-carapace). Each haibu server
(https://github.com/nodejitsu/haibu) can spawn n drones running
whatever (multiple apps, or multiple copies of the same app). Our
balancers can then proxy to these drones however we please.

> On average what are your costs running 3000+ app on AWS's?

My advice to you here would be to do this:

1) Figure out how many apps you can comfortably run on some AWS
instance. Keep in mind that this is app-dependent. My parents' small
business website doesn't need a whole lot of resources and could share
space with a bunch of similarly quiet apps, but something like
http://now.jit.su can (is) spread out over multiple machines of
roughly the same size.
2) Figure out how much this chosen AWS instance costs. Baby ones cost
less but give you less ponies.
3) ((cost/instance) / (apps/instance)) (number of apps) = (total
cost). The math here is pretty simple; the tough part is estimating
apps/instance.

I can tell you right now that it's non-trivial. I doubt a single
person could realistically afford to run 3000+ apps out-of-pocket, at
least without seriously gimping each app's resources.

> The basic app structure was you would have your model's, view's and controller's, and raft would load all these into the app, kinda so you didn't have to code and express server or is server or what have you.

Reminds me a little of geddy. The one thing I'll say, and many would
disagree with me here (which is fine), but imo you should always be
able to start a single instance of an app without clustering by
running, say, `node app.js`. That said, there's certainly room for,
`require('thing')({ models: modelsPath, views: viewsPath, controllers:
ctrlsPath });` or similar if that's how you like to roll. :) I'd
probably combine an approach like this with haibu were I trying to DIY
a platform.

> why did you chose git for the pushing of apps to the backend?

In my experience: Developers are comfortable with git. Heroku uses git
for deployment iirc, and no.de also used git for deployment. Nodejitsu
uses jitsu+http because it allows us to use npm for app bundling.

Hope this helps,

--Josh

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en



--
Joshua Holbrook
Head of Support
Nodejitsu Inc.
jo...@nodejitsu.com

[dvbportal]

Hi,

I know Nodester, which we are running at Cloudnode:

> I know that nodester is hosted on AWS's and with that how many apps are 

> hosted per server? Are the server like a 8GB/RAM 4/core pc running maybe 

> 20 apps per server, or is is a micro server running just one app?

 

You probably need to be flexible here. Nodester, unlike us, is running on one big instance, which means there is a limit of how much apps can run. Others doing the micro server per app thing are also getting into trouble. They need 3000+ IP addresses which are a limited resource and you have to explain your demand to RIPE.

You need very good monitoring and balancing to dynamically find an optimum in between these models.

> To nodester. why did you chose git for the pushing of apps to the backend? 

Git can do multiple protocols and can go thru client side firewalls. As it is a VCS you never loose track of you apps and see what has changed when. We have also installed Gitweb to allow users direct access to their repositories compare versions and re-download what they need. (http://gitweb.cloudno.de)

Also as Josh said, devs are comfortable with it.

- Hans

[Tim Dickinson]

>Actually, haibu is what runs everyone's apps on nodejitsu! We have a few closed source extensions for haibu, but aside from that it's all the same. In fact, I encourage you to use haibu as a starting point for something like raft. 

Well haibu was the first project that i found to be like mine is.  haibu has heavily influenced raft, so has heroku and nodester.

>We have a similar concept, called a drone (https://github.com/nodejitsu/haibu-carapace). Each haibu server (https://github.com/nodejitsu/haibu) can spawn n drones running whatever (multiple apps, or multiple copies of the same app). Our balancers can then proxy to these drones however we please. 

The drones concept is very slimier to what raft has. I just thought there might have been more to it.

>I can tell you right now that it's non-trivial. I doubt a single person could realistically afford to run 3000+ apps out-of-pocket, at least without seriously gimping each app's resources. 

I dont see myself running 3000+ app out of pocket. You make a good point about how much load each app would have. Looking at nodester they seem to limit each app to 25mb of ram and if it goes over then it kills the app and re-spawns it. I dont really want to go that route, how do you guys monitor each app? Do you do it inside the app or from out-side the app? do you ever kill apps or been resouce hogs?

From the second post

>They need 3000+ IP addresses which are a limited resource and you have to explain your demand to RIPE.

The way raft is setup you only need a few external ip's. These ip's would be pointing at the load-balancer. So 3000+ apps would not be a problem as they would be local ip's

>Nodejitsu uses jitsu+http because it allows us to use npm for app bundling. 

I have followed a very slimier approach to you guys. Do you guys feel that it makes user less likely to use the platform because git is not used?

>but imo you should always be able to start a single instance of an app without clustering by running, say, `node app.js`.

I agree with you here. I have given it some thought and was thinking of doing something like "raft-cli run" and this would allow the user to run the app locally. The whole node app.js this is far fetched for raft as it runs each file.js in its own context. I need to do a lot more thinking about this problem.

[Arnout Kazemier]

>Nodejitsu uses jitsu+http because it allows us to use npm for app bundling. 

I have followed a very slimier approach to you guys. Do you guys feel that it makes user less likely to use the platform because git is not used?

I would rather say it would be more likely that people use the platform. Not everybody uses git, subversion and mercurial are still really big version

control systems. And by starting out with HTTP support you could actually support all these different kind of version control system if you a simple

hook script that deploys the code on push. While this is not the same as having "git" on the server, it comes really close to the functionality and is

much more flexible in my opinion.

I might be biased as I also work for Nodejitsu but I hope my input helps here non the less

[Tim Dickinson]

>I would rather say it would be more likely that people use the platform. Not everybody uses git, subversion and mercurial are still really big version

control systems. And by starting out with HTTP support you could actually support all these different kind of version control system if you a simple

hook script that deploys the code on push. While this is not the same as having "git" on the server, it comes really close to the functionality and is

much more flexible in my opinion.

I do feel the same way about your opinion, i could always just pull a git repo  like haibu does

[chrismatthieu]

Hi Tim,

I'm the founder of Nodester, the open source Node.JS PaaS.  Your project sounds really cool!  Answers to your questions are provided below:

- I know that nodester is hosted on AWS's and with that how many apps are hosted per server? Are the server like a 8GB/RAM 4/core pc running maybe 20 apps per server, or is is a micro server running just one app?

Nodester is currently hosting over 6,000 Node.JS apps on a single Extra Large AWS instance without a reverse proxy!  We have a team currently working on horizontal scaling and monitoring and the ability to spin up additional resources like Heroku's dynos.  We are calling them "Jets" to go along with our rocket theme ;)

- To nodester. why did you chose git for the pushing of apps to the backend? 

We love Git!  It's the modern way for updating code and pushing updates on many services.  Now that Windows users are becoming more familiar with git, our support efforts have decreased with trying to teach people how to use it.  Setting up an RSA key the first time for git has always been our most FAQ.  Over time, this has proven to be the right decision!

- To nodester / nodejitsu. Have you guys thought of a kind of dynos (heroku style)? if so how would you guys go about doing that? like you spawn 2 processes of the same app and just route request to each app like node does with the cluster module?

Jets are coming...

- To nodester. On average what are your costs running 3000+ app on AWS's? 

$500 per month (which is sponsored by @Tropo) :)

Gotta love the Node.JS community and Tropo - http://tropo.com!

Hack the Planet!

@ChrisMatthieu

[Gustavo Machado]

Nodejitsu/Haibu guys,

Do you have more than one Haibu server running? And if so, how would
you go about doing ReverseProxy/LoadBalancing?

Thanks,
Gustavo Machado

[Joshua Holbrook]

> Do you have more than one Haibu server running?

Yes. Dedicated apps each get their own haibu server. Meaning we're
running thousands of 'em.

> how would you go about doing ReverseProxy/LoadBalancing?

We use https://github.com/nodejitsu/node-http-proxy in our balancers.

--Josh

[Tim Dickinson]

How do you guys Load-balance over 2 apps? are you monitoring the apps load? or is it like the cluster module does it with round ribbon or what ever its called?

[José F. Romaniello]

is called round-robin, I dont know if they do that  but i know in the node-http-proxy there is a very nice example:

https://github.com/nodejitsu/node-http-proxy/blob/master/examples/balancer/simple-balancer.js 

btw, I love the level of openess of both companies :)

2012/8/15 Tim Dickinson <price...@gmail.com>

round ribbon

[Tim Dickinson]

That is a good example but what if you have a cluster of http-proxy, you may end up sending 10 request to on app then 10 to the other.

What are peoples take on loadbalanceing over two or more app?

[Bradley Meck]

When using multiple load balancers for multiple domains:

1. SNICallback in https is your friend for figuring out domains.

2. You will want to be grabbing data from a shared storage facility like a Database where you can replicate down to your balancers. This information will hold info on where to route requests, when to remove drones from the tables, and even info such as load averages ideally.

3. RoundRobin works better than you might think #2 is overkill usually.

4. Use IP hashes to determine consistently where your connection/session should go to, sending reconnect and session logic to 2 different drones can lead to bad behavior.

5. Terminate SSL for the love of all that is holy if you have already locked down your workers.

6. Save your routing tables in memory, don't go fetch from DB on every request.

7. Cluster + in memory state for apps, support nightmare

8. Multi-port apps, don't do it unless you are letting people drop their own load balancers.

[José F. Romaniello]

DO you do routing (host header/ snicallback) and load balancing in the
same reverse proxy (mean same process) or you put these two things on
two separated process?

2012/8/16, Bradley Meck <bradle...@gmail.com>:

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>

--

Enviado desde mi dispositivo móvil

[Bradley Meck]

Yes, same process generally. It replicates a large amount of data into an in memory routing table etc. the cost of actually piping data around once the connections have been established isn't bad.

[Tim Dickinson]

>6. Save your routing tables in memory, don't go fetch from DB on every request.

For now I'm query the db every time a request comes in. I tried keeping it in memory but i need to find a better way to push to all nodes in the cluster.

>7. Cluster + in memory state for apps, support nightmare

To scale out this is needed. In memory state for apps is something the developer would have to consider.

>8. Multi-port apps, don't do it unless you are letting people drop their own load balancers.

Drop their own load balancers is far fetch as the load balancing is a core part of raft.

Raft is very slimier to how heroku works in fact heroku has been a meager influence to raft. I need to do more reading about how heroku does scaling with their dynos.

I was thinking about multi request from the same user to different app. Maybe something like a session or by ip to route the same user to the same app every time. This would stop one request to one app and another request to the second app from the same user. This would help the developer to keep an in app state.