I'm trying to do a test rebundle of a Karmic Beta image, but I get:
ec2-bundle-vol: command not found
Are the EC2 tools not installed in the Karmic Beta AMI (ami-52be5d3b)?
Or am I doing something wrong/stupid (though I'm using the same
command that has worked OK in the past with other Ubuntu AMIs) ?
> I'm trying to do a test rebundle of a Karmic Beta image, but I get:
> ec2-bundle-vol: command not found
> Are the EC2 tools not installed in the Karmic Beta AMI (ami-52be5d3b)? > Or am I doing something wrong/stupid (though I'm using the same > command that has worked OK in the past with other Ubuntu AMIs) ?
> Any ideas / solutions?
We've made a change to only put items in the base images that are in the ubuntu 'main' component [1]. The license on the ec2-ami-tools keeps them from being included in main.
That said, you have 2 options: 1. use the euca2ools package, which is already installed. These are ec2-ami-tools and ec2-api-tools workalikes. They're not 100% command line compatible at the moment, but they are close. There is more information for those at [2] and [3]. 2. install and use the ec2-ami-tools: mirror=http://us.ec2.archive.ubuntu.com/ubuntu/ printf "%s\n%s\n" "deb ${mirror} karmic multiverse" \ "deb-src ${mirror} karmic main" | sudo tee /etc/apt/sources.list.d/multiverse.list sudo apt-get update && sudo apt-get install ec2-ami-tools
Sorry for the inconvenience of not having those tools there out of the box. But either of the two should work for you. And, fwiw, the euca2ools are significantly faster than their ec2 counterparts.
> > I'm trying to do a test rebundle of a Karmic Beta image, but I get:
> > ec2-bundle-vol: command not found
> > Are the EC2 tools not installed in the Karmic Beta AMI (ami-52be5d3b)?
> > Or am I doing something wrong/stupid (though I'm using the same
> > command that has worked OK in the past with other Ubuntu AMIs) ?
> > Any ideas / solutions?
> We've made a change to only put items in the base images that are in the
> ubuntu 'main' component [1]. The license on the ec2-ami-tools keeps them
> from being included in main.
> That said, you have 2 options:
> 1. use the euca2ools package, which is already installed. These are
> ec2-ami-tools and ec2-api-tools workalikes. They're not 100% command line
> compatible at the moment, but they are close. There is more information
> for those at [2] and [3].
> 2. install and use the ec2-ami-tools:
> mirror=http://us.ec2.archive.ubuntu.com/ubuntu/ > printf "%s\n%s\n" "deb ${mirror} karmic multiverse" \
> "deb-src ${mirror} karmic main" |
> sudo tee /etc/apt/sources.list.d/multiverse.list
> sudo apt-get update && sudo apt-get install ec2-ami-tools
> Sorry for the inconvenience of not having those tools there out of the
> box. But either of the two should work for you. And, fwiw, the euca2ools
> are significantly faster than their ec2 counterparts.
> > I'm trying to do a test rebundle of a Karmic Beta image, but I get:
> > ec2-bundle-vol: command not found
> > Are the EC2 tools not installed in the Karmic Beta AMI (ami-52be5d3b)?
> > Or am I doing something wrong/stupid (though I'm using the same
> > command that has worked OK in the past with other Ubuntu AMIs) ?
> > Any ideas / solutions?
> We've made a change to only put items in the base images that are in the
> ubuntu 'main' component [1]. The license on the ec2-ami-tools keeps them
> from being included in main.
> That said, you have 2 options:
> 1. use the euca2ools package, which is already installed. These are
> ec2-ami-tools and ec2-api-tools workalikes. They're not 100% command line
> compatible at the moment, but they are close. There is more information
> for those at [2] and [3].
> 2. install and use the ec2-ami-tools:
> mirror=http://us.ec2.archive.ubuntu.com/ubuntu/ > printf "%s\n%s\n" "deb ${mirror} karmic multiverse" \
> "deb-src ${mirror} karmic main" |
> sudo tee /etc/apt/sources.list.d/multiverse.list
> sudo apt-get update && sudo apt-get install ec2-ami-tools
> Sorry for the inconvenience of not having those tools there out of the
> box. But either of the two should work for you. And, fwiw, the euca2ools
> are significantly faster than their ec2 counterparts.
The us.ec2.archive.ubuntu.com mirror (and corresponding "eu" mirror)
are only accessible from within EC2 to avoid extra network charges at
AWS rates.
If you want to follow the instructions for installing the tools on a
system outside of EC2, you can substitute your standard Ubuntu mirror
(see /etc/apt/sources.list) or us.archive.ubuntu.com, which
humorously, is located in the UK.
--
On Nov 5, 6:02 pm, Sanjaya Ratnaweera <sanja...@gmail.com> wrote:
> On Oct 5, 7:22 pm, Scott Moser <smo...@ubuntu.com> wrote:
> > On Mon, 5 Oct 2009, rtweed wrote:
> > > I'm trying to do a test rebundle of a Karmic Beta image, but I get:
> > > ec2-bundle-vol: command not found
> > > Are the EC2 tools not installed in the Karmic Beta AMI (ami-52be5d3b)?
> > > Or am I doing something wrong/stupid (though I'm using the same
> > > command that has worked OK in the past with other Ubuntu AMIs) ?
> > > Any ideas / solutions?
> > We've made a change to only put items in the base images that are in the
> > ubuntu 'main' component [1]. The license on the ec2-ami-tools keeps them
> > from being included in main.
> > That said, you have 2 options:
> > 1. use the euca2ools package, which is already installed. These are
> > ec2-ami-tools and ec2-api-tools workalikes. They're not 100% command line
> > compatible at the moment, but they are close. There is more information
> > for those at [2] and [3].
> > 2. install and use the ec2-ami-tools:
> > mirror=http://us.ec2.archive.ubuntu.com/ubuntu/ > > printf "%s\n%s\n" "deb ${mirror} karmic multiverse" \
> > "deb-src ${mirror} karmic main" |
> > sudo tee /etc/apt/sources.list.d/multiverse.list
> > sudo apt-get update && sudo apt-get install ec2-ami-tools
> > Sorry for the inconvenience of not having those tools there out of the
> > box. But either of the two should work for you. And, fwiw, the euca2ools
> > are significantly faster than their ec2 counterparts.
On Oct 5, 6:22 am, Scott Moser <smo...@ubuntu.com> wrote:
> And, fwiw, the euca2ools are significantly faster than their ec2 counterparts.
I can believe that's true comparing euca2ools to the equivalent
Amazon's EC2 API tools (written in Java), but in my tests so far,
euca2ools is 50% slower than the equivalent Amazon's EC2 AMI tools
(written in Ruby).
With a base Karmic instance, bundling and uploading to S3 takes over
8min with euca2ools, and only 5.5min with Amazon's EC2 AMI tools.
> On Oct 5, 6:22 am, Scott Moser <smo...@ubuntu.com> wrote: >> And, fwiw, the euca2ools are significantly faster than their ec2 counterparts.
> I can believe that's true comparing euca2ools to the equivalent > Amazon's EC2 API tools (written in Java), but in my tests so far, > euca2ools is 50% slower than the equivalent Amazon's EC2 AMI tools > (written in Ruby).
> With a base Karmic instance, bundling and uploading to S3 takes over > 8min with euca2ools, and only 5.5min with Amazon's EC2 AMI tools.
I haven't tried the euca2ools, but I would not be surprised if the "significantly faster" statement has to do with startup time of the tool, so e.g. the ec2-describe-blah Ruby tool would be faster than the Java version. I can believe that a long running command like bundling and uploading would be faster with the Java version.
> On Oct 5, 6:22 am, Scott Moser <smo...@ubuntu.com> wrote: > > And, fwiw, the euca2ools are significantly faster than their ec2 counterparts.
> I can believe that's true comparing euca2ools to the equivalent > Amazon's EC2 API tools (written in Java), but in my tests so far, > euca2ools is 50% slower than the equivalent Amazon's EC2 AMI tools > (written in Ruby).
> With a base Karmic instance, bundling and uploading to S3 takes over > 8min with euca2ools, and only 5.5min with Amazon's EC2 AMI tools.
Interesting. Not something I would have expected. I really would have thought that most of the bottleneck of either tool would have been in external forks (ie, calling 'tar' or 'gzip'), or in the upload itself.
On Tue, 10 Nov 2009, Michael Wood wrote: > I haven't tried the euca2ools, but I would not be surprised if the > "significantly faster" statement has to do with startup time of the
Absolutely true. but on my laptop:
$ time ec2-describe-images -a >/dev/null real:0m7.703s user:0m6.560s sys:0m0.270s $ time euca-describe-images -a >/dev/null real:0m4.473s user:0m0.680s sys:0m0.040s
$ time ec2-describe-instances >/dev/null real:0m4.135s user:0m4.840s sys:0m0.170s $ time euca-describe-instances >/dev/null real:0m0.567s user:0m0.120s sys:0m0.040s
The net, is that for those frequently run commands, the euca2ools save me considerable real (wall) and cpu time.
> tool, so e.g. the ec2-describe-blah Ruby tool would be faster than the > Java version. I can believe that a long running command like bundling > and uploading would be faster with the Java version.
Maybe... I'm not exactly sure what all the java tools do, and when then invoke system commands. As Eric pointed out, the bundle and uploading from ec2-ami-tools are ruby, and I do know that they do much of their work with system commands (tar/gzip)
I wouldn't expect that uploading would be significantly different on any of these. In most cases I'd expect network bandwidth to bottleneck.
> > And, fwiw, the euca2ools are significantly faster than their ec2 counterparts.
> I can believe that's true comparing euca2ools to the equivalent
> Amazon's EC2 API tools (written in Java), but in my tests so far,
> euca2ools is 50% slower than the equivalent Amazon's EC2 AMI tools
> (written in Ruby).
This is indeed true, at least for Amazon's original API tools (java)
and parts of euca2ools that correspond to those calls (boto, python).
I haven't checked AMI tools (those used for bundling AMIs).
The former uses SOAP API (and works off of X509 cert/key), the latter
uses query API (and works off of access key id and secret access key).
This was the first time I used the euca2ools, and I might not have set
things up to give them the best chance of performing against the EC2 AMI
tools.
I had set EC2_URL and S3_URL to "https://..." instead of "http://...".
I'm not sure which URL the AMI tools default to, but this could affect
upload performance.
I don't know why the bundle phase was also slower, but more testing
seems in order before I make blanket statements.
In case it wasn't clear, I do completely believe that euca2ools would be
faster than the EC2 API tools (note the difference between "AMI" and
"API"). The EC2 API tools are horribly slow and every alternative from
euca2ools to Tim Kay's "aws" command to boto to typica to my own Perl
commands accessing the API should be blazing fast in comparison.
Scott Moser wrote:
> On Tue, 10 Nov 2009, Michael Wood wrote:
>> I haven't tried the euca2ools, but I would not be surprised if the
>> "significantly faster" statement has to do with startup time of the
> Absolutely true. but on my laptop:
> $ time ec2-describe-images -a >/dev/null
> real:0m7.703s user:0m6.560s sys:0m0.270s
> $ time euca-describe-images -a >/dev/null
> real:0m4.473s user:0m0.680s sys:0m0.040s
> $ time ec2-describe-instances >/dev/null
> real:0m4.135s user:0m4.840s sys:0m0.170s
> $ time euca-describe-instances >/dev/null
> real:0m0.567s user:0m0.120s sys:0m0.040s
> The net, is that for those frequently run commands, the euca2ools save me
> considerable real (wall) and cpu time.
>> tool, so e.g. the ec2-describe-blah Ruby tool would be faster than the
>> Java version. I can believe that a long running command like bundling
>> and uploading would be faster with the Java version.
> Maybe... I'm not exactly sure what all the java tools do, and when then
> invoke system commands. As Eric pointed out, the bundle and uploading
> from ec2-ami-tools are ruby, and I do know that they do much of their work
> with system commands (tar/gzip)
> I wouldn't expect that uploading would be significantly different on any
> of these. In most cases I'd expect network bandwidth to bottleneck.
|
