Packer qemu with macvtap

[tanner...@gmail.com]

Hey Everyone,

I am currently attempting to generate a virtual machine image with packer to help automate our node creation process.
Every node we've made thus far with virtual machine manager has been copied off a base node and connects to the network via macvtap.
These macvtaps appear to be automatically generated after you select the configuration in the virtual machine manager then boot up the VM.
They can also manually be configured via the XML that is passed to Virt-manager

Auto-generated XML example of macvtap(redacted mac):

<interface type='direct'>
      <mac address='XX:XX:XX:YY:YY:YY'/>
      <source dev='enp1s0f0' mode='bridge'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

The trouble I'm having is figuring out how it goes about generating these macvtaps and successfully connecting to the intranet by passing the device info via qemuargs.
I've tried a couple of options; namely here and here and usually receive an error like: "TUNSETOFFLOAD ioctl() failed: Inappropriate ioctl for device"

The use of a bridge or user generated network is unavailable to me and results in ssh timeouts whether it cant establish a handshake or it timeouts on the loopback lookup.

Packer json (redacted mac):
 

 "builders":
  [
    {
      "type": "qemu",
      "name": "packer-base",
      "qemu_binary": "/usr/libexec/qemu-kvm",
      "vm_name": "packer-base",
      "iso_url": "/root/Downloads/CentOS-7-x86_64-Minimal-1708.iso",
      "iso_checksum": "aae20c8052a55cf179af88d9dd35f1a889cd5773",
      "iso_checksum_type": "sha1",
      "output_directory": "./centos-built",
      "shutdown_command": "shutdown -P now",
      "disk_size": 5000,
      "format": "qcow2",
      "headless" : false,
      "accelerator": "kvm",
      "http_directory": "httpdir",
      "http_port_min": 10082,
      "http_port_max": 10089,
      "ssh_host_port_min": 2222,
      "ssh_host_port_max": 2229,
      "ssh_username": "root",
      "ssh_password": "somepass",
      "ssh_port": 22,
      "ssh_timeout": "30s",
      "ssh_wait_timeout": "2m",
      "net_device": "virtio-net",
      "disk_interface": "virtio",
      "qemuargs": [
        [ "-netdev", "tap,fd=3,id=hostnet0,vhost=on,vhostfd=4" ],
        [ "-device", "virtio-net-pci,netdev=hostnet0,id=net0,mac=XX:XX:XX:YY:YY:YY" ]
      ],
      "boot_wait": "7s",
      "boot_command": [
        "<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/centos7_2-ks.cfg<enter><wait>"
      ]
    }
  ]
}

ks file

#platform=x86, AMD64, or Intel EM64T
#version=DEVEL
# Install OS instead of upgrade

skipx
install
# Keyboard layouts
# old format: keyboard us
# new format:
keyboard --vckeymap=us --xlayouts='us'
# Root password
rootpw --iscrypted somepassbutencrypted
# System language
lang en_US.UTF-8

# Firewall configuration
firewall --disabled
# System authorization information
auth  --useshadow  --passalgo=sha512
# Use text mode install
# Run the Setup Agent on first boot
firstboot --enable
# Use http
url --url http://mirror.raystedman.net/centos/7/os/x86_64/
repo --name=updates --baseurl=http://mirror.raystedman.net/centos/7/updates/x86_64/
# SELinux configuration
selinux --disabled

# System services
services --disabled="chronyd"
services --enabled=NetworkManager,sshd
user --name=hsimage --plaintext --password hsimage
ignoredisk --only-use=vda
# Network information
network  --bootproto=static --device=eth0 --gateway=172.31.0.1 --ip=172.31.1.152 --netmask=255.255.0.0 --nameserver=10.2.1.205,10.2.1.225 --onboot=yes --activate

# System timezone
timezone America/Los_Angeles --nontp
# System bootloader configuration
bootloader --append=" crashkernel=auto" --location=none --boot-drive=vda
autopart --type=lvm
# Partition clearing information
clearpart --all --initlabel
# restart after installation
reboot

%packages
@^minimal
@core
kexec-tools
git
%end

%post

sed -i "s/#PermitRootLogin yes/PermitRootLogin yes/g" /etc/ssh/sshd_config



%end

Qemu commandline documentation

example configuration from existing VM retrieved via 'ps' command(redacted mac):

ps -ef | grep qemu-kvm

-netdev tap,fd=29,id=hostnet0,vhost=on,vhostfd=32
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=XX:XX:XX:YY:YY:YY,bus=pci.0,addr=0x3

There is just something I'm not quite getting with preliminary vm network configuration that is preventing me from making a tap connection work. 

Let me know if there is any additional info that is needed and thanks in advance for any help!

Regards,
Tanner

[Alvaro Miranda Aguilera]

Hello

I think you are hitting a chicken egg issue.

You cna't download the KS since there is NO IP setup on the guest, and the guest IP setup is in the KS.

Not sure in QEMU but for other providers you can use floppy, that doesn't require networking, so that does the trick.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/554875f5-2565-44e5-92e5-c0de72c815e3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Alvaro

[tanner...@gmail.com]

Hey Alvaro,

Floppy does work it just requires different syntax specifically for RHEL/CENTOS 7 since they deprecated "ks=floppy"

Someone found a workaround here: https://github.com/hashicorp/packer/issues/2920

I'll try that out and let you know my findings, so far -netdev "user" still doesn't work but I suspect a bridge setup ought to work now.

Thanks for the response!

Tanner

[tanner...@gmail.com]

Hey Alvaro,

Seems like I'm still hitting into this issue. I tried from scratch with a new network and KVM setup but still hit the same problems.
I'm going to look at a few other options. I couldn't help but notice a majority of other help places recommended a DHCP server to help serve IP info the the VM.

I'm also considering using the example given for QEMU using exactly CENTOS 6 just to see if there is some issue outside of packer or the kick start file that is preventing me from succeeding.
 I shall update this thread with any findings.

Regards,

Tanner

[tanner...@gmail.com]

Hey Everyone,

So far nothing has been particularly successful, I suspect my kickstart file is off somehow but I'm I need to look into how to test one in the first place.

The problem has more or less changed from using a macvtap network to: "getting anything to work in a qemu environment."

If there is any additional info I ought to post for my runs let me know.

I'm going to look at VM alternatives and see if I can get any image built at all.

Regards,

Tanner

[Alvaro Miranda Aguilera]

Hello

Can you try some other project to see if works?

ie try to build this:

https://github.com/stardata/packer-centos7-kvm-example

Then you can modify to suit

--

This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/13110f9b-229c-4c4d-8901-64fd1b685aa7%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Alvaro

[tanner...@gmail.com]

Hey Alvaro,

A nice find! I didn't stumble across that while looking around for solutions, (I was probably too deep in my rabbit hole to notice). That did end up working!
I shall modify off this to get the setups I require.
Thanks again for the help!
Packer ought to consider updating the QEMU section to use this setup as it worked first try.

Regards,

Tanner