Distributed systems at Amazon

Eric Anderson

unread,

Jul 17, 2012, 3:27:42 PM7/17/12

to distsys...@googlegroups.com

At CopperEgg we're a pretty big user of Amazon's EC2 (and lots of other services of Amazon). We've had some interesting problems to solve regarding some software and having it distributed across zones.

Are there any favorite software tools/etc that you use that work particularly well in a distributed cloud environment?

Eric Anderson

CopperEgg

Cliff Moon

unread,

Jul 17, 2012, 3:33:17 PM7/17/12

to distsys...@googlegroups.com

Not to toot our own horn (and let me know if it's inappropriate for this forum), but this is basically the problem that Boundary is built to solve: visualizing and diagnosing network performance issues, especially involving cloud and multi-az deployments. Signup is free https://boundary.com/signup and feedback is always welcome. Thanks.

Eric Anderson
CopperEgg

--
You received this message because you are subscribed to the Google Groups "Distributed Systems" group.
To post to this group, send email to distsys...@googlegroups.com.
To unsubscribe from this group, send email to distsys-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msg/distsys-discuss/-/0gyFHAOgHLUJ.
For more options, visit https://groups.google.com/groups/opt_out.

Kelly Sommers

unread,

Jul 17, 2012, 3:42:05 PM7/17/12

to distsys...@googlegroups.com

Eric,

Would you be able to describe what kinds of challenges you faced? We are getting close to having to ramp up on some new deployment configurations that we didn't have before so I would be keen to learn what types of things to look out for.

Kelly

Eric Anderson

CopperEgg

Michael Rose

unread,

Jul 17, 2012, 5:29:12 PM7/17/12

to distsys...@googlegroups.com

What kind of major issues are you having distributing things across AZs?

The first step is identifying what's not scaling. Is it your process or your application?

Personally one of our favorite pieces of software is Zookeeper -- it makes everything stupid easy to keep some shared metadata and do service autodiscovery / autoconfig.

Another good tool we use is Storm for writing real-time systems, it becomes a simple matter to deploy a large real-time system on top of any number of EC2 instances.