Building a vagrant image with Packer on a CI environment

[Michael McCarthy]

Hi all,

Apologies if this has been answered before, I couldn't find any similar posts. We're using a virtualbox-iso image and the vagrant post-processor and it all works really nicely. Massively impressed so far.

Now we're at the stage where we want to build on a build server so we can just pop the vbox out as an artifact at the end of a build and a developer can vagrant box add/init/up this. What we're finding is that virtualbox itself won't run on a virtualised environment such as our elastic bamboo slaves in AWS. We haven't tried yet but we suspect the same will be true of the Jenkins slaves we run on OpenStack.

Has anyone else encountered this and what was the workaround? Could we just use a different builder?

Thanks!

[Alvaro Miranda Aguilera]

packer have providers, one of them is virtualbox.

virtualbox + vagrant = vagrant box template

pretty much that cover the out of the box packer + vagrant use case.

on top of that you have several providers on packer and serveral providers on vagrant.

it seems to me, that you want something without virtualbox, BUT:

your life cycle can be like this.

iso
scripts
virtualbox

ami
scripts
aws

in that way, you can test locally, and you are happy with the scripts, etc, and then move to aws.

or just use aws difrectly.

vagrant + aws pluging won't use virtualbox boxes, you can reuse the amis/volumes created on packer in the first step

make sense?

--
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael McCarthy]

Thanks, I think I get you there but I'm not sure if it will solve my use case. Here's hopefully a better explanation!

My required output is a Vagrant box that can be used with Virtualbox. Working backwards I presume from this that the last step in my build is going to be a Vagrant post-processor. I understand that Vagrant now has multiple providers so you can build Vagrant boxes that are targeted at different providers such as VMWare etc. What I think you are saying is that on the build server I can use an AMI builder (if we are running on AWS) and I can test everything out locally using an ISO builder and the same set of scripts. The final output may vary slightly depending on how similar the AMI I use is compared to the installation I do with the ISO.

I can always test this but do you now off hand if the Vagrant post-processor will require Virtulabox to run if it is building a box that is targeted at the Virtualbox provider? It isn't the end of the world if this is the case - end users could always run the necessary post processing scripts locally - it would still be shorter than an entire build from scratch.

Many thanks

Michael

[Alvaro Miranda Aguilera]

if i get you , you are asking if you can create a virtualbox box without virtualbox ?

outside vagrant it may be possible as virtualbox will import an ovf or ova that can be created with other systems like vmware

but in packer, I am afraid that the providers will require the specific one.. in this case virtualbox wil use VboxManage, Vbox.....etc.

if you any reason you don't have how to create those boxes, let me know and I may create one for you and put on vagrantcloud.com

[ch...@liatrio.com]

I have a very similar use case and am bumping into the same issue.

We run our Jenkins server on AWS EC2 instance. We are creating our own CI pipeline to maintain local environments (for software developers to use) created with packer on VirtualBox.

We source control the packer templates and puppet modules used to create these images. When someone changes the packer template or any of the puppet modules, we want a jenkins job - that executes packer to run.

However, as you are finding - we cant run Virtualbox on the EC2 instances. 

Is our only option physical machines (even as a jenkins slave) to do the packer/virtualbox work?

Chris

[Alvaro Miranda Aguilera]

if the machine is simple, wouldn't docker fit the bill?

[ch...@liatrio.com]

Yeah. If it was simple, docker would work great.

Unfortunately it is not simple and I think the best approach currently is to create VM's. This solution works great in a proof of concept, now just struggling to fully implement it in an all AWS world.

[Alvaro Miranda Aguilera]

let me expand my suggestion.

If you don't require block devices, or complex network setup, docker should be able to help you.

What requirement you have that you can't use docker today?

in terms of light -> heavy I see virtualization in this way today:

docker -> lxc -> xen/kvm -> vmware/virtualbox/hyper-v

[ch...@liatrio.com]

So, I appreciate the input and suggestions.

I am working with a very large organization - and basically docker is NOT an option right now. :)

Instead of going further down - I would just love to focus on the original ask - can I run Packer, and create Virtualbox VM's while using AWS/Xen? Or is there no solution, and I need to have a physical server that would be responsible for the building of the packer images?

Thanks again!

[Alvaro Miranda Aguilera]

Virtualbox requires physical VT extensions for 64 bits OS

so if you get a physical box you may have luck, or you can test 32 bit OS.

seems most of amazon instances run on XEN, so you run out of luck at all. Virtualbox doesn't run inside xen.

another option is import a virtualbox vm in aws if you can generate those outside:

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/VMImportPrerequisites.html

[Michael Crawford]

I've been working on something similar, but to deploy on either physical blades or on top of VMware vCenter VMs in-house. I've been able to get this to work, but it's much slower when running hypervisor on hypervisor than hypervisor on physical hardware.

So, even if you are able to get this to work, you might find it's so slow it's not something you can really use in a CI pipeline where there's some expectation of quick results.

Alvaro above suggested using two pipelines:

Physical:

1. iso

2. scripts
3. VirtualBox

AWS:
1. ami (original)
2. scripts
3. aws (new ami)

I'm trying to do something similar, but there's a little more setup:

Phase 1: Create initial internal ami-equivalent base boxes (very static once created)

1. iso

2. kickstart + scripts which replicate existing AWS AMIs you would start with as exactly as possible
3. VMware/VirtualBox Vagrant base box which is equivalent (same RPMs, same initial config) to the AMI you will start with below.
4. we're planning on uploading these into vCenter as VM templates, also looked into creating them as OpenStack images.

Now we have the same behavior across:

1. Chef test-kitchen (or equivalent) when starting with vagrant base boxes on our mirror

2. Chef runs (or equivalent) or additional packer builds intended for vCenter deployment

3. Chef runs (or equivalent) or additional packer builds intended for AWS deployment

Phase 2: In-House CI deployment pipeline:

1. Vagrant base box which is equivalent to AMI in AWS pipeline (out of phase 1)

2. scripts / cookbooks
3. customized base box

4. new template uploaded into vCenter

5. VMs created from template, at some point via VCAC automation

Phase 2: AWS CI deployment pipeline:
1. ami which is equivalent to Vagrant base box in In-House pipeline

2. scripts / cookbooks
3. customized ami

4. new instances created from AMI via CloudFormation

Still working through the implementation of this, but that's the approach...

Comments appreciated!

Mike

[Alvaro Miranda Aguilera]

Hello,

I am not sure of this as I haven't tested recently, but you should be able to simplify the numbers of packer build and master gold images like this

1 iso -> virtualbox without virtualbox tools

2  virtualbox without virtualbox tools ->  virtualbox wit virtualbox tools

3  virtualbox without virtualbox tools -> qemu/kvm with virtualbox tool converter (for openstack)

4  virtualbox without virtualbox tools -> Vmware, should be able to read and use this

5  virtualbox without virtualbox tools > amazon should be able to import this box

6  virtualbox without virtualbox tools > should be ok to mount/copy for docker and lxc

so basically the point I want to make, is you can convert between formats, so whatever is easier for you, you should be able to re-use some base image..

Alvaro,