`vagrant init $box` creates default Vagrantfile, not my custom as described in my packer file

[john napiorkowski]

Searching around seems to suggest other people have this issue but none of the part solutions worked for me.  What I am trying to do is use packer to build on Atlas a vagrant box with two providers: virtual box and aws.  Here's my JSON file:

{

    "variables": {

        "aws_access_key": "{{env `AWS_ACCESS_KEU`}}",

        "aws_secret_key": "{{env `AWS_SECRET_KEY`}}",

        "atlas_token": "{{env `ATLAS_TOKEN`}}"

    },

    "provisioners": [

        {

            "type": "shell",

            "execute_command": "echo '{{user `ssh_pass`}}' | {{ .Vars }} sudo -E -S sh '{{ .Path }}'",

            "scripts": [

                "scripts/base.sh",

                "scripts/vagrant.sh",

                "scripts/virtualbox.sh"

            ]

        },

        {

          "type": "file",

          "source": "files/perl.conf",

          "destination": "/tmp/perl.conf"

        },

        {

            "type": "shell",

            "execute_command": "echo '{{user `ssh_pass`}}' | {{ .Vars }} sudo -E -S sh '{{ .Path }}'",

            "scripts": [

                "scripts/cleanup.sh"

            ]

        }

    ],

   "push": {

        "name": "jjn1056/catme",

        "token": "{{user `atlas_token`}}"

    },

    "builders": [

        {

            "type": "virtualbox-iso",

            "boot_command": [

                "<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg<enter><wait>"

            ],

            "boot_wait": "10s",

            "disk_size": 81920,

            "guest_os_type": "RedHat_64",

            "headless": false,

            "http_directory": "http",

            "iso_urls": [

                "http://centos.eecs.wsu.edu/6.7/isos/x86_64/CentOS-6.7-x86_64-minimal.iso"

            ],

            "iso_checksum_type": "md5",

            "iso_checksum": "9381a24b8bee2fed0c26896141a64b69",

            "ssh_username": "vagrant",

            "ssh_password": "vagrant",

            "ssh_port": 22,

            "ssh_wait_timeout": "10000s",

            "shutdown_command": "echo 'vagrant'|sudo -S /sbin/halt -h -p",

            "guest_additions_path": "VBoxGuestAdditions_{{.Version}}.iso",

            "virtualbox_version_file": ".vbox_version",

            "vm_name": "packer-centos-6.7-x86_64"

        }

    ],

    "post-processors": [

      [

        {

          "type": "vagrant",

          "keep_input_artifact": false,

          "output": "builds/{{.Provider}}-centos67.box",

          "vagrantfile_template": "files/Vagrantfile.template",

        },

        {

          "type": "atlas",

          "token": "{{user `atlas_token`}}",

          "artifact": "jjn1056/catme",

          "artifact_type": "vagrant.box",

          "metadata": {

            "provider": "virtualbox",

            "version": "2.0.2",

            "created_at": "{{timestamp}}",

            "description": "Catme on the desktop"

          }

        },

        {

          "type": "atlas",

          "token": "{{user `atlas_token`}}",

          "artifact": "jjn1056/catme",

          "artifact_type": "vagrant.box",

          "metadata": {

            "provider": "aws",

            "version": "2.0.2",

            "created_at": "{{timestamp}}",

            "description": "Catme on the cloud"

          }

        }

      ]

    ]

}

And the file 'files/Vagrantfile.template'

Vagrant.configure(2) do |config|

  config.name = "catme"

  config.vm.box = "jjn1056/catme"

  config.vm.provider :virtualbox do |override| 

    override.vm.box = "jjn1056/catme"

    override.vm.network "forwarded_port", guest: 80, host: 8080

    override.vm.network "forwarded_port", guest: 443, host: 8443

    override.vm.network "forwarded_port", guest: 5432, host: 5432

    override.vm.network "private_network", ip: "192.168.33.10"

  end

  config.vm.provider :aws do |aws, override|

    override.vm.box = "aws_dummy_box"

    override.vm.box_url = "https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box"

    override.ssh.username = "ubuntu"

    override.ssh.private_key_path = "#{ENV['HOME']}/.ssh/id_rsa"

    aws.access_key_id = "#{ENV['AWS_ACCESS_KEY']}"

    aws.secret_access_key = "#{ENV['AWS_SECRET_KEY']}"

    aws.ami = "ami-524e4726"

    aws.region = "eu-west-1"

    aws.availability_zone = "eu-west-1c"

    aws.instance_type = "m1.small"

    aws.security_groups = [ "" ]

    aws.keypair_name = "ZZZZZ"

  end

  config.vm.provision "shell", inline: <<-SHELL

    sudo -E su vagrant -c 'ssh-keyscan -t rsa catme02.catme.org > ~/.ssh/known_hosts'

    sudo -E su vagrant -c 'GIT_SSH_COMMAND="ssh -v" git clone gitosis@XXXX:catme.git /vagrant/catme-git'

    sudo -E ln -s /vagrant/catme-git/catme /var/www/catme

    sudo -E ln -s /vagrant/catme-git/catme-lib /var/www/catme-lib

    sudo -E /vagrant/catme-git/catme-lib/master-pages/makelinks-soft.sh

    mkdir -p /var/www/catme-lib/tmp

    chmod 775 /var/www/catme-lib/tmp

    sudo perl -p -i -e "s/User apache/User vagrant/" /etc/httpd/conf/httpd.conf

    sudo perl -p -i -e "s/Group apache/Group vagrant/" /etc/httpd/conf/httpd.conf

    sudo /etc/init.d/httpd start

    PGPASSWORD=XXXXXXX psql catme catmerw -f /vagrant/catme-git/catme-lib/config/dbbackups/hals_dump.sql 

  SHELL

end

Would appreciate any sort of hint.  I figure I must  have a number of things wrong, but this one is really stopping me.  I do 'packer push template.json' it goes out to atlas, builds with no errors , I see a vagrant box is there two both providers but I never get the expected Vagrantfile when I do init.  Its really getting me :) 

Thanks, I'll be sure to blog about whatever help I get and the outcome of my experiments.

John

[john napiorkowski]

So I was thinking maybe this is a rather complex question to ask, and maybe not everyone can help.  So here's a more simple question that would help me a lot in trying to figure this out, and possibly is one more people are able to answer:

If I define a vagrantfile_template in my packer JSON template for a given $box, is that supposed to be the Vagrantfile that gets generated when I do something like `vagrant init $box`?  If so, does anyone have a straightforward example of this in action?  I think if I had a working example I could probably trace back in the code how it works and then figure out what I am doing wrong.  Thanks tremendously,  -john

[Alvaro Miranda Aguilera]

Hello.

# vagrant init -m user/box
# vagrant init -m box

will only create a simple, Vagrantfile witht that name for
config.vm.box = "name"

since Vagrant 1.5+ user/box mean Vagrant will try to download the box
from Atlas (former Vagrantcloud)

So, locally the process goes like this.

Packer + template.json = file.box

vagrant box add -name name_of_box --provider=virtualbox file.box

That will add that box, locally to Vagrant

you can see/confirm the box you got with

vagrant box list

Then

vagrant init name_of_box

create a Vagrantfile and Vagrant on first run will try to use that box.

IF the box exists, will work.
If the box doesn't exists, will fail.


If you put the box in a local network in a known url, don't share the
commands, share a Vagrantfile

vagrant init -m name_of_box
edit Vagrantfile
add config.vm.box_url = "http://server/url/awesome/file.box"

If you can have the box in Atlas, you can have it there, public or
private, and just tell the users

vagrant init -m user/box

or share a Vagrantfile.

====

That is the workflow for boxes more or less.

If there is anything else, please let me know. You can include a
Vagrantfile with the box, but that works like this.

vagrant up will read the Vagrantfile in the current directoy, in the
~/.vagrant.d/Vagrantfile if present and in the directory of the box.

Will crate a new Vagrantfile will that, and then run.

To read about the Lookout, check this:
https://docs.vagrantup.com/v2/vagrantfile/index.html

If you want to include a file or Vagrantfile when packaging the box
with Vagrant:
https://docs.vagrantup.com/v2/cli/package.html

If you use packer:

https://packer.io/docs/post-processors/vagrant.html
include (array of strings) - Paths to files to include in the Vagrant
box. These files will each be copied into the top level directory of
the Vagrant box (regardless of their paths). They can then be used
from the Vagrantfile.

Thanks!
Alvaro

> --
> This mailing list is governed under the HashiCorp Community Guidelines -
> https://www.hashicorp.com/community-guidelines.html. Behavior in violation
> of those guidelines may result in your removal from this mailing list.
>
> GitHub Issues: https://github.com/mitchellh/packer/issues
> IRC: #packer-tool on Freenode
> ---
> You received this message because you are subscribed to the Google Groups
> "Packer" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to packer-tool...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/packer-tool/d4ac7a91-ecd7-4d72-9360-e6c367e8861a%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

[john napiorkowski]

Hi Alvaro, and thanks for responding:

This doesn't seem to be working for me, so let me review what I have and did. Its very possible since I'm new to this my expectations are just not right.  But here's what happened when I tried what you said and the results I got. 

So remember first I created a vagrant box on Altas using packer with the atlas provider.  As part of my packer template I used 'vagrantfile_template' to place a default Vagrantfile for this box, since in order to use the box properly there's some startup provisioning stuff you need have have run.  That box ran and was created under https://atlas.hashicorp.com/jjn1056/boxes/catme (which is private, sorry this is for a client so has to be, but I'll try a new simple box that is public if I continue to be stuck for all of us to look at).  I installed the box locally (after doing 'vagrant login') with:

$ vagrant box add jjn1056/catme

Which downloaded and installed it locally.  I can see that box has my custom vagrant file in it because if I do:

$ cat ~/.vagrant.d/boxes/jjn1056-VAGRANTSLASH-catme/2.0.2/virtualbox/Vagrantfile

the output is the custom file.  And that is the correct box, version and all.  But when I do

$ vagrant init -m jj1n056/catme

the Vagrantfile that gets created is the default one, not the custom one I see at "~/.vagrant.d/boxes/jjn1056-VAGRANTSLASH-catme/2.0.2/virtualbox/Vagrantfile"

So I'm trying to figure out should that work?  My reading of the docs would lead me to think that, and I can't think of a value of letting you have a custom vagrant file if its not allowed.  So does anyone know if this should work?  Or am I just not reading the docs right and not getting it?  If you can't generate a vagrant file like this, what would be the purpose of having that option?

Thanks!

john

You received this message because you are subscribed to a topic in the Google Groups "Packer" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/packer-tool/Iu1jSRgbWOk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to packer-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/CAHqq0exKAi078%2BoWScFGouBvmcXOtux4CDbMSynX3y64b9E-Gg%40mail.gmail.com.

[john napiorkowski]

Ah so now I see the docs:

"A common misconception is that the --vagrantfile option will package a Vagrantfile that is used when vagrant init is used with this box. This is not the case. Instead, a Vagrantfile is loaded and read as part of the Vagrant load process when the box is used. For more information, read about the Vagrantfile load order."

 and have to think about this some more.   I can't for the life of me understand the point of letting you associate a vagrant file like this and yet it doesn't get used for the init... if this is such a common misunderstanding it feels like an interface failure to me.  Let me read it and I'll look again at the Golang code.

Thanks!

john

[john napiorkowski]

So wanted to say thanks again Alvaro, since its now clear that the answer to my question is "no, it doesn't work that way".  I guess when you include the vagrant file as part of the box it goes into the search path used for 'vagrant up'.  For the life of me I can't see the value of that, or the use case, since you can't do 'vagrant up $box..' without having done a vagrant init first.  And if you do 'init' that builds a default Vagrantfile, so your custom one never gets found.  Maybe you can delete the default one, but that seems silly.   I guess there's a use case here but I can't figure out what.  And I'd recommend different docs on this in packer, because if you google this problem you see a ton of people think exactly what I thought.  So it seems like its the kind of thing people seem to want to have (or at least we all think we want it, at some point in our learning process).

Its not terrible to have to distribute the Vagrantfile as part of the devops repo and I see that appears to be the common case.  So I'll just stop worrying about it and start working on the AWS provider :)

Thanks all!  Its pretty great tech, and making immutable architectures is a tremendous win.

John

[Alvaro Miranda Aguilera]

Hello,

It should work, what is happening is  you may need to play with that and see what it does

Check this:

vagrant init -m hashicorp/precise64

now I edit that Vagrantfile and I add this:

puts "from the local folder”

I edit ~/.vagrant.d/boxes/hashicorp-VAGRANTSLASH-precise64/1.1.0/virtualbox/Vagrantfile and add

puts "from the virtualbox folder”

Then edit ~/.vagrant.d/Vagrantfile and add:

puts "this is from the Global Vagrantfile"

Alvaros-Mini:precise64 kikitux$ vagrant status

this is from the Global Vagrantfile

from the local folder

from the virtualbox folder

Current machine states:

default                   not created 

The VMware machine has not yet been created. Run `vagrant up`

to create the machine. If a machine is not created, only the

default provider will be shown. Therefore, if a provider is not listed,

then the machine is not created for that provider.

Alvaros-Mini:precise64 kikitux$ 

What the documentation is telling you is.

vagrant init Won’t create a Vagrantfile with that content, since is not required.

If you re-read my first email, I tried to explain that vagrant will read all the Vagrantfiles and crate one Vagrantfile in memory and will run it.

So in this way, you have

~/.vagrant.d/Vagrantfile

this is global and will be included/readed in all the actions of vagrant in all the vms.

~/.vagrant.d/boxes/hashicorp-VAGRANTSLASH-precise64/1.1.0/virtualbox/Vagrantfile 

this is local to the vm, and will be used in all the VMS that use this box

then

Vagrantfile in the local folder have customizations that will ne used in the local project.

make sense?

Alvaro.

[john napiorkowski]

Wow...

Ok, so it doing something just the opposite of what I'd expect, at first thought but then it does sorta make sense.  What seems to be happening the actually does make sense is that the configurations are combined.  That does actually work for me, mental model wise, and I'll be discussing it with my coworkers.  I wonder if there's a documentation patch that would be useful.

Thanks again!

john

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to a topic in the Google Groups "Packer" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/packer-tool/Iu1jSRgbWOk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to packer-tool...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/79DE388F-F013-4159-B0C3-F3C4AAC13990%40gmail.com.

[Alvaro Miranda Aguilera]

happy to help.
Alvaro

> You received this message because you are subscribed to the Google Groups
> "Packer" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to packer-tool...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/packer-tool/CAMF3VnhyLDFJYKD4eGgN2N_rv_WFAEGi9rUXq7hOtBsNUtndow%40mail.gmail.com.