Using deb package management for gems (was Re: NYC Puppet Meetup) - Puppet Users

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion Using deb package management for gems (was Re: NYC Puppet Meetup)
Received-SPF: pass (google.com: domain of larry...@gmail.com designates 209.85.200.171 as permitted sender) client-ip=209.85.200.171;
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:user-agent:mime-version:to:cc:subject
         :references:in-reply-to:content-type:content-transfer-encoding;
        b=e5ZaEe7DAl/0ld4HAXPadJ1t5Ifssdp3hU7aZ7kA039BZKw0prd7knFz8GwXez1+NT
         FpSRbS/zdqcolH8b2JGrEocaif7HWC271pgWWwO3Rnb/xE2Dr8229sfXSjZO6yoprzqX
         7K6MgdH2N3rdYPDJNA7dsFTPyIbBUWpKECmFY=
Message-ID: <49749716.40...@gmail.com>
Date: Mon, 19 Jan 2009 10:07:02 -0500
From: Larry Ludwig <larry...@gmail.com>
User-Agent: Thunderbird 2.0.0.19 (Macintosh/20081209)
MIME-Version: 1.0
To: Brian Gupta <brian.gu...@brandorr.com>
CC: Puppet Users <puppet-users@googlegroups.com>
Subject: Re: Using deb package management for gems (was Re: NYC Puppet Meetup)
References: <e387b84f-5489-4ea1-856e-382cc5f59...@e1g2000pra.googlegroups.com>	 <5b5090780901180939v73eaa9aem40b98c6b3ad2...@mail.gmail.com>	 <5b5090780901180946n53304b92ma954035203604...@mail.gmail.com>	 <5b5090780901180948t27f3cf82mcc4c0a8560c31...@mail.gmail.com>	 <5b5090780901181002o35c9170bj67b65d84bc2af...@mail.gmail.com>	 <f668c080-b06b-4e0e-ac50-f027d1593...@w39g2000prb.googlegroups.com>	 <54c4e4f40901182154t6b8fe537ubd1a825bf4e6c...@mail.gmail.com>	 <49747CDA.3010...@gmail.com>	 <54c4e4f40901190536h736b1f03l8e8a8fe51d8ba...@mail.gmail.com>	 <497484C6.7030...@gmail.com> <54c4e4f40901190625o2239a809pa87b13f4a65e4...@mail.gmail.com>
In-Reply-To: <54c4e4f40901190625o2239a809pa87b13f4a65e4...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Brian Gupta wrote:
>
> Mostly we want the ability, to leverage the micro-accounting EC2
> offers us. (To lower operational costs).
>
> Since EC2 bills by the hour, wouldn't it be prudent to be able to spin
> down idling webservers during the evening, and spin up extra ones when
> you know there are upcoming high traffic website events? Currently we
> have this ability. For now it is still a manual process, in that we
> have to say "please spin up X webserver nodes now in the xyz
> environment", but they do autoconfigure themselves, update DNS, and
> even adjust the load balancer config. We are still working on the
> scheduling ability, but our puppet code and ec2 glue are all complete,
> if should be fairly straightforward to throw something together. (We
> might even leverage the OS scheduler.)
>   
Hmm this brings up an interesting discussion.

It depends... if a no-op CPU command is billable to EC2 and they are 
truly sitting idle does Amazon charge for it?  Obliviously a running 
instance will have CPU cycles, but not much if idle.

Also the build process of a new instance I suspect has a decent amount 
of CPU time (obviously less if you are using binaries)

I'm sure there is some point where it makes more sense to leave the 
instance running for X amount of hours, instead of regening it. (meaning 
it costs more to build a new EC2 instance, than just leave an existing 
one active)  The other issues related to re-gening is new IP address and 
the time for EC2 to have your new instance ready (ie 10 min)

Have you done any investigation on the cost benefits?  If so what were 
the results?

I'm obviously all for the automating the build of a new instance, and 
also automatically adding it to the pool of resources.  I'm not so sure 
the next level of automation.  Auto-scaling without any human interaction.

The issue I have with most auto-scaling is the very basic metrics used 
to measure when to add/remove resources.    I believe the issues are 
very customer/app specific and while with one customer X metric might be 
valid, with another you need X, Y and Z in some formula to determine 
when to go to the next level.  The other issue related to this is the 
application itself.  What may have allowed a customer app to scale to 
one level, may require code changes to scale effectively at a much 
higher level.  This is something auto-scaling could never do.  Vertical 
scaling is ALWAYS easier than going horizontal. 

Puppet is great at the automation of administration. To me auto-scaling 
could be a different tool and write high level rules on when to scale 
and what part to scale.  Do I hear another tool in the making? :-)

-L


> Our eventual goal has always been *being able* to autoscale.
> Realistically though, since unexpected load is fairly uncommon, we
> still want a human in the scaling feedback loop. (So basically we want
> autoscaling with a "Human! Press this button!" step in the autoscaling
> process.)
>
> Cheers,
> Brian
>
>   
>>>> -L
>>>>
>>>> --
>>>> Larry Ludwig
>>>> Empowering Media
>>>> 1-866-792-0489 x600
>>>> Managed and Unmanaged Xen VPSes
>>>> http://www.hostcube.com/
>>>>
>>>>
>>>>
>>>>         
>> -L
>>
>> --
>> Larry Ludwig
>> Empowering Media
>> 1-866-792-0489 x600
>> Managed and Unmanaged Xen VPSes
>> http://www.hostcube.com/
>>
>>
>>
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
