Network file system in packer Shell provisioner QEMU-KVM

[tanner...@gmail.com]

Hey everyone,

I have been trying to setup a database in packer which grabs an sql dump file from a NFS folder then imports it.
This is achieved with a file provisioned script that I upload to "/importBackup.sh"
The issue is that it seems the shell provisioner can't use the NFS.
Is this a side effect of how packer functions or a bug?

I have the mount configuration set up in the fstab done by the Kickstart file.
I have verified that the configuration functions by loading the generated image into Virtual Machine Manager and inspecting it.

Oddly enough when I tried running the backup script at the post process step by SSHing into the machine after setting it up in Virtual Machine Manager, the script file appears to be missing.
That script also appears just fine when I manually inspect the VM.

I am running CentOS 7 on QEMU-KVM.

Let me know if additional info is needed.

Regards,

Tanner Posada

[tanner...@gmail.com]

An Additional note is I have gone to the permissions of the source/storage machine and set the connection access to open for all IPs and still could not connect to the mounted file system.

[Alvaro Miranda Aguilera]

if you could share the packer template for the file transfer + provisioner when you running it and a log of the run

will help.

alvaro

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/d1b013c2-68c5-4bae-8395-1dfed2140e61%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Alvaro

[tanner...@gmail.com]

Hey Alvaro,

Thank you for responding here is the builder json contents

{
  "builders":
  [
    {
      "type": "qemu",
      "qemu_binary": "/usr/libexec/qemu-kvm",
      "accelerator": "kvm",
      "headless": false,
      "qemuargs": [
        [ "-m", "6000M" ],
        [ "-smp", "cpus=6,maxcpus=6,cores=1" ],
        [ "-netdev", "user,hostfwd=tcp::{{ .SSHHostPort }}-:22,id=forward"],
        [ "-device", "virtio-net,netdev=forward,id=net0"]
      ],
      "disk_interface": "virtio",
      "disk_size": 42000,
      "format": "qcow2",
      "net_device": "virtio-net",
      "iso_url": "/root/Downloads/CentOS-7-x86_64-Minimal-1708.iso",
      "iso_checksum": "aae20c8052a55cf179af88d9dd35f1a889cd5773",
      "iso_checksum_type": "sha1",
      "vm_name": "db-node",
      "output_directory": "img-output",
      "http_directory": "docroot",
      "http_port_min": 10082,
      "http_port_max": 10089,
      "ssh_host_port_min": 2222,
      "ssh_host_port_max": 2229,
      "ssh_username": "root",
      "ssh_private_key_file": "/root/.ssh/id_rsa-1",
      "ssh_port": 22,
      "ssh_wait_timeout": "3600s",
      "boot_wait": "40s",
      "boot_command": [
        "<up><wait><tab><wait>text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/db-ks.cfg<enter><wait>"
      ],
      "shutdown_command": "shutdown -P now"
    }
  ],
  "provisioners": [
    {
      "type" : "file",
      "source" : "ifcfg-eth0",
      "destination": "/etc/sysconfig/network-scripts/ifcfg-eth0"
    },
    {
      "type" : "file",
      "source" : "/home/packer/privateScripts/importBackup.sh",
      "destination": "/importBackup.sh"
    },
    {
      "type" : "file",
      "source" : "/home/packer/privateScripts/doSqlBackup.sh",
      "destination": "/doSqlBackup.sh"
    },
    {
      "type" : "file",
      "source" : "./crontask.txt",
      "destination": "/crontask.txt"
    },
    {
      "type": "shell",
      "script": "/home/packer/privateScripts/setRoot.sh"
    },
    {
      "type": "shell",
      "expect_disconnect": true,
      "inline_shebang": "/bin/sh -ex",
      "inline": [
        "curl -k https://puppetmaster.service.smc:8140/packages/current/install.bash | bash",
        "printf '[Unit]\nAfter=mysqld.service\n\n[Service]\nExecStart=/importBackup.sh\nType=idle\n\n[Install]\nWantedBy=default.target' > /etc/systemd/system/importBackup.service && chmod 664 /etc/systemd/system/importBackup.service",
        "systemctl daemon-reload && systemctl enable importBackup.service",
        "echo 'Import will be performed on next system boot ONCE'",
        "firewall-cmd --set-default-zone=public",
        "firewall-cmd --permanent --zone=public --add-port=5666/tcp",
        "firewall-cmd --permanent --zone=public --add-port=5667/tcp",
        "firewall-cmd --permanent --zone=public --add-port=3306/tcp",
        "firewall-cmd --reload",
        "crontab /crontask.txt",
        ". /importBackup.sh",
        "ip link set dev eth0 down && ip link set dev eth0 up",
        "echo 'network restart succeeded'"
      ]
    }
  ],
  "post-processors":
  [
    {
      "type": "shell-local",
      "inline": [
                  "mv img-output/* /home/packer/img-container -f && rmdir img-output",
                  "echo 'successfully moved file\nMaking VM...'",
                  "virt-install -n databaseClone --vcpus=1 -r 2048 --os-type=linux --os-variant=centos7.0 --disk /home/packer/img-container/db-node,device=disk,bus=virtio -w bridge=virbr1,model=virtio --vnc --noautoconsole --import && echo 'VM created!'",
                  "echo 'giving system time to safely load up'",
                  "sleep 1m",
                  "virsh shutdown databaseClone",
                  "scp /root/.ssh/id_rsa-1 /root/.ssh/id_rsa-1.pub root@admin:/root/.ssh/",
                  "ssh admin \"printf '\n\nHost db2 db2.service.smc 172.31.1.145\nHostname 172.31.1.145\nIdentityFile ~/.ssh/id_rsa-1\nUser root' >> /root/.ssh/config\""
                ]
    }
  ]
}

Attached is the output of the build attempt.

The system is mounted to that folder via fstab configuration made in the KickStart file.

If you need anything else let me know.

Tanner Posada

On Wednesday, May 9, 2018 at 9:19:15 AM UTC-7, Alvaro Miranda Aguilera wrote:

if you could share the packer template for the file transfer + provisioner when you running it and a log of the run

will help.

alvaro

On Tue, May 8, 2018 at 12:08 AM, <tanner...@gmail.com> wrote:

Hey everyone,

I have been trying to setup a database in packer which grabs an sql dump file from a NFS folder then imports it.
This is achieved with a file provisioned script that I upload to "/importBackup.sh"
The issue is that it seems the shell provisioner can't use the NFS.
Is this a side effect of how packer functions or a bug?

I have the mount configuration set up in the fstab done by the Kickstart file.
I have verified that the configuration functions by loading the generated image into Virtual Machine Manager and inspecting it.

Oddly enough when I tried running the backup script at the post process step by SSHing into the machine after setting it up in Virtual Machine Manager, the script file appears to be missing.
That script also appears just fine when I manually inspect the VM.

I am running CentOS 7 on QEMU-KVM.

Let me know if additional info is needed.

Regards,

Tanner Posada

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.

To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/d1b013c2-68c5-4bae-8395-1dfed2140e61%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Alvaro

[Rickard von Essen]

I'm pretty sure this is an error in your scripts. Packer doesn't care/know if you are using NFS or something else. Try to step back a bit and start with something simpler and then you add steps until you find the one that is wrong. Be sure to verify each one.

To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/0a8c7867-e4b7-4bb9-b78a-e10969421b09%40googlegroups.com.

[tanner...@gmail.com]

Hey Rickard,

I have had the script work just fine after running it on the image I generate with packer.
What I noticed is (based on the error output) the file system isn't mounted on the provisioner step despite having rebooted the VM after the kickstart installation.
When attempting to manually mount it in the shell provisioner I receive this error: "mount.nfs: access denied by server while mounting xxx.xxx.xxx.xxx:/data/backup"
The storage server has that directory open to any IP that tries to connect to it so I am unsure why its denying access.

Let me know your thoughts on this info.

Regards

Tanner Posada

[Rickard von Essen]

It's hard to say without knowing your complete setup. But it's not packer causing the problem. My first hunch is that you try to bind a nfs mount where the client is behind nat. That will not work. Try to do the procedure by hand first.

To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/3d6144f2-c957-4772-90ea-5cd391ed612f%40googlegroups.com.

[tanner...@gmail.com]

Hey Rickard,

Ah I see what you mean, I'll have to change the -netdev and -device in the qemuargs on this one for it to work correctly.
Thanks for pointing that out.

Tanner Posada