Scale up or out?

Showing 1-4 of 4 messages
Scale up or out? Eric Anderson 6/26/12 7:24 AM
Hey all,

Question about EC2 (or scale in general): i'm building a decent cluster, to handle 15-20k inserts/s and 5-10k gets per second.  (for a rough idea of what I'm doing).  I've been playing with a 15-node cluster of m2.xlarge systems, but I am wondering what is better: more small systems or less larger systems?  

Any recommendations/hints/tricks would be a huge help!

Eric

Re: Scale up or out? Jeremiah Peschka 6/26/12 7:37 AM
Scale out - zerg rush your data.

Many smaller systems means that, in theory, you're going to be affected less by poor performance of any single instance. 

When you're building out instances in EC2 always remember that the largest instance in a class of instances is most likely to be the only VM on that server. So, an m2.4xlarge will probably be the only instance on that VM host. The biggest downside to shared VMs is that you're also sharing a 1gigabit ethernet connection. 

If you're okay with the possibility of sharing your connectivity with other people, then don't worry about instance sizes and just keep scaling out.

If you do worry about what other jerks are doing, then try to size your instances so that you're the only person on that piece of hardware. Basically, the instances you're looking for are m1.xlarge, m2.4xlarge, c1.xlarge, cc1.4xlarge, cc2.8xlarge. The upside is that you can also hog all of the instance storage and take advantage of local storage AND get the redundancy that Riak brings to the table - EBS volumes are RAIDed behind the scenes.

This handy chart will help: http://www.ec2instances.info/

---
Jeremiah Peschka
Managing Director, Brent Ozar PLF, LLC


_______________________________________________
riak-users mailing list
riak-...@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


Re: Scale up or out? Aphyr 6/26/12 9:34 AM
Your cheapest/fastest option is probably physical HW with SSDs.

--Kyle
Re: Scale up or out? Eric Anderson 6/27/12 7:14 AM
On Jun 26, 2012, at 9:37 AM, Jeremiah Peschka <jeremiah...@gmail.com> wrote:

Scale out - zerg rush your data.

Many smaller systems means that, in theory, you're going to be affected less by poor performance of any single instance. 

When you're building out instances in EC2 always remember that the largest instance in a class of instances is most likely to be the only VM on that server. So, an m2.4xlarge will probably be the only instance on that VM host. The biggest downside to shared VMs is that you're also sharing a 1gigabit ethernet connection. 

If you're okay with the possibility of sharing your connectivity with other people, then don't worry about instance sizes and just keep scaling out.

If you do worry about what other jerks are doing, then try to size your instances so that you're the only person on that piece of hardware. Basically, the instances you're looking for are m1.xlarge, m2.4xlarge, c1.xlarge, cc1.4xlarge, cc2.8xlarge. The upside is that you can also hog all of the instance storage and take advantage of local storage AND get the redundancy that Riak brings to the table - EBS volumes are RAIDed behind the scenes.

Thanks!

That helps a lot!

Eric Anderson
Co-Founder
http://CopperEgg.com




On Tue, Jun 26, 2012 at 7:24 AM, Eric Anderson <ande...@copperegg.com> wrote:
Hey all,

Question about EC2 (or scale in general): i'm building a decent cluster, to handle 15-20k inserts/s and 5-10k gets per second.  (for a rough idea of what I'm doing).  I've been playing with a 15-node cluster of m2.xlarge systems, but I am wondering what is better: more small systems or less larger systems?  

Any recommendations/hints/tricks would be a huge help!

Eric


_______________________________________________
riak-users mailing list
riak-...@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com