RE: Invent

[Steven Hansen]

Is anyone going to Amazon's RE: Invent conference next month?

[Jonathan Duncan]

On 30 Oct 2012, at 14:15, Steven Hansen wrote:

> Is anyone going to Amazon's RE: Invent conference next month?

I wish. I was really wanting to go. Just don't have that cash on hand. :)

[Marc Carmen]

Yes I will be there

On Tue, Oct 30, 2012 at 2:15 PM, Steven Hansen <ste...@mediarain.com> wrote:

Is anyone going to Amazon's RE: Invent conference next month?

--
Thanks,
Marc Carmen
marc....@gmail.com

[Steven Hansen]

Marc,

You are now in charge of coming back with a full report!

..............................................

STEVEN HANSEN

..............................................

director of technology

..............................................

RAIN / mediarain.com

..............................................

P / 801.802.6464

..............................................

NEW YORK / UTAH

..............................................

+ small agency of the year
+ inc. 500/5000
+ webby mobile game of the year

[Jonathan Duncan]

I would love to hear how it goes and what you learn.

On 30 Oct 2012, at 16:58, Steven Hansen wrote:

> Marc,
>
> You are now in charge of coming back with a full report!
>

> STEVEN HANSEN

[Marc Carmen]

Well I will bring back the best notes I can.  I will be primarily attending the classes on scalability using AWS but I may catch some on Startups as well.

[Jonathan Duncan]

Scalability is one if the topics I am interested in as well.

[tertullian01]

I apologize it has taking me so long to post a report on this.  As you probably know by now you can view the slides from the presentations AWS Slides and the videos are posted here AWS Videos 

I just wanted to write up the overall experience and summary of what I learned there.  I would have done it sooner but I have been very busy moving from standard ec2 instances to Beanstalk and RDS.

So here are some of the important points that I learned.  I am happy to go into a discussion of any one of them if you have questions:

1. Elastic Beanstalk and RDS are perfect for vanilla systems that don't need a lot of custom configuration.  They has been put together for teams that don't want to have to do a lot of server administration.
1. That being said there are some things you can do to do custom configuration on these systems but it is limited
2. Most of the really big services that you hear about using AWS for scalability are doing the own server administration and are not using Beanstalk.  Some are using RDS.  However, this may be due to the fact that they rolled out their own custom system prior to the availability of Elastic Beanstalk for their backend language.
1. For example, Netflix has custom built all of their own management and testing tools to handle their load.  So they use EC2 instances, load balancers, etc... rather than something like Elastic Beanstalk.  Netflix has released some of their testing tools as open source.  They are known as the Simian Army: https://github.com/Netflix/simianarmy  As a side note, they were there recruiting and are always looking for engineers.  So if you are interested I can privately pass on the name of the recruiter that I met there.
3. The transition from a standard server backend or even an EC2 backend to Elastic Beanstalk is designed to be nearly seamless. I found this was true with a Java backend using tomcat.  PHP, Python, Ruby, and .Net are supported as well but I have not worked with transitioning those backends.  One thing to note on that is that Java was the first and most mature backend for Elastic Beanstalk.  
4. The transition from a standard database server to RDS is nearly seamless although if your service is already up and running it can be a pain to do.  You will incur some downtime while you do the switch over.  If you have a large database it will take a while.  I recommend running at least one test to get all of the commands to work correctly and you can get an idea of the time it will take to complete.
5. Reserved instances are your friend.  Once you determine what instance size you need you will definitely want to purchase reserved instances if you can.  Once they are purchased you have them and you cannot switch to a different instance size.  But they will significantly reduce your cost.
1. Determining what instance size you need with Elastic Beanstalk is a little bit of a process.  Sometimes you will want to go with a small instance but have beanstalk load a lot of those or you may need to go with a large instance and only a few of those.  You will have to play with this to figure out what works for you.
6. With any AWS system you are running you want to make sure that you are running in at least two availability zones.  This is true for RDS, Elastic Beanstalk, and standard EC2 instances.  Most of the outages that have occurred have affected one single availability zone.  However, once the outage occurs there is a run by all of the users that are only in one availability zone to get up and running in a second which means that it will take everyone longer.  If you are already up in a second or third availability zone and you have a load balancer that automatically redirects your traffic then you should be fine.
7. Choose your region wisely.  The East region is by far the most popular and most utilized.  It gets all of the latest updates first.  However, this also means it tends to have more issues than other regions.  As I have made the switch from standard EC2 instances to RDS and Elastic Beanstalk I have moved from the East to the West region.  

The number one rule that I have found to be true with any of this is test.  As I prepared to make the transition I would fire up an RDS instance and Elastic Beanstalk.  I would point my test environment there and run scalability and fault tolerance testing.  By the end of the day I could shut the system down and incur very small charges. 

As I mentioned I am happy to answer any questions or have longer discussions on any of these points.  Just let me know.

[Jonathan Duncan]

Thank you for reporting back!

On 24 Jan 2013, at 06:54, tertullian01 wrote:

> I apologize it has taking me so long to post a report on this. As you
> probably know by now you can view the slides from the presentations AWS

> Slides <http://www.slideshare.net/AmazonWebServices> and the videos are
> posted here AWS Videos <http://www.youtube.com/user/AmazonWebServices>

>
> I just wanted to write up the overall experience and summary of what I
> learned there. I would have done it sooner but I have been very busy
> moving from standard ec2 instances to Beanstalk and RDS.
>
> So here are some of the important points that I learned. I am happy to go
> into a discussion of any one of them if you have questions:
>

> 1. Elastic Beanstalk and RDS are perfect for vanilla systems that don't

> need a lot of custom configuration. They has been put together for teams
> that don't want to have to do a lot of server administration.

> 1. That being said there are some things you can do to do custom

> configuration on these systems but it is limited

> 2. Most of the really big services that you hear about using AWS for

> scalability are doing the own server administration and are not using
> Beanstalk. Some are using RDS. However, this may be due to the fact that
> they rolled out their own custom system prior to the availability of
> Elastic Beanstalk for their backend language.

> 1. For example, Netflix has custom built all of their own management

> and testing tools to handle their load. So they use EC2 instances, load
> balancers, etc... rather than something like Elastic Beanstalk. Netflix
> has released some of their testing tools as open source. They are known as
> the Simian Army: https://github.com/Netflix/simianarmy As a side note,
> they were there recruiting and are always looking for engineers. So if you
> are interested I can privately pass on the name of the recruiter that I met
> there.

> 3. The transition from a standard server backend or even an EC2 backend

> to Elastic Beanstalk is designed to be nearly seamless. I found this was
> true with a Java backend using tomcat. PHP, Python, Ruby, and .Net are
> supported as well but I have not worked with transitioning those backends.
> One thing to note on that is that Java was the first and most mature
> backend for Elastic Beanstalk.

> 4. The transition from a standard database server to RDS is nearly

> seamless although if your service is already up and running it can be a
> pain to do. You will incur some downtime while you do the switch over. If
> you have a large database it will take a while. I recommend running at
> least one test to get all of the commands to work correctly and you can get
> an idea of the time it will take to complete.

> 5. Reserved instances are your friend. Once you determine what instance

> size you need you will definitely want to purchase reserved instances if
> you can. Once they are purchased you have them and you cannot switch to a
> different instance size. But they will significantly reduce your cost.

> 1. Determining what instance size you need with Elastic Beanstalk is

> a little bit of a process. Sometimes you will want to go with a small
> instance but have beanstalk load a lot of those or you may need to go with
> a large instance and only a few of those. You will have to play with this
> to figure out what works for you.

> 6. With any AWS system you are running you want to make sure that you

> are running in at least two availability zones. This is true for RDS,
> Elastic Beanstalk, and standard EC2 instances. Most of the outages that
> have occurred have affected one single availability zone. However, once
> the outage occurs there is a run by all of the users that are only in one
> availability zone to get up and running in a second which means that it
> will take everyone longer. If you are already up in a second or third
> availability zone and you have a load balancer that automatically redirects
> your traffic then you should be fine.

> 7. Choose your region wisely. The East region is by far the most