Vagrantfile with complex network topology?

[Luke Gorrie]

Howdy,

Is anybody building complex network topologies for Vagrant VMs?

I'm having fun now seeing if I can write a Vagrant file that brings up about 10 machines (hosts, switches, routers) and wire them up using extra ethernet ports and virtual patch cables between them. In my particular use case today it's important that each switch/router be a full VM and so I'm not tempted to using the Linux toolchest (brctl, etc) on the hypervisor host -- so it would be neat to handle it all with Vagrant somehow.

The most straightforward approach I've found is to use VirtualBox "internal networks" to add additional NICs to VMs and wire them together with virtual patch cables:

     vb.customize ["modifyvm", :id, "--nic2", "intnet"]
     vb.customize ["modifyvm", :id, "--intnet2", "foo"]

Seems like an internal network with only two attached ports is near enough to being a patch cable.

If anybody else is working with this kind of thing I'd love to hear tips. I'm not married to VirtualBox and could easily using another provider if that were better.

Cheers,
-Luke

[Steve Witt]

I'm doing something like that to test some networking applications in a
virtual network environment. I have 4 and 6 router networks with an
additional 4 VMs acting as hosts, so 8 or 10 VMs total using Virtualbox
private networking. I define the VMs in the network using a hash at the
beginning of the Vagrantfile:

nodes = {
:node1 => ['192.168.201.1', '192.168.202.1', '172.16.1.1'],
:node2 => ['192.168.201.2', '192.168.203.1', '172.16.2.1'],
:node3 => ['192.168.202.2', '192.168.204.1', '172.16.3.1'],
:node4 => ['192.168.203.2', '192.168.204.2', '172.16.4.1'],
:host1 => ['172.16.1.2'],
:host2 => ['172.16.2.2'],
:host3 => ['172.16.3.2'],
:host4 => ['172.16.4.2'],
}

Then further down loop through this hash setting up all the VMs.


nodes.keys.each do |node|

# 'config.vm.define' is used in a multi-vm case to create each vm
config.vm.define "#{node}" do |box|
box.vm.hostname = "#{node}"

# loop through IP addresses assigned to the node.
nodes[node].each do |ipAddr|
box.vm.network :private_network, ip: "#{ipAddr}"
end

config.vm.provider :virtualbox do |vb|
vb.name = "#{node}"
end
end
end


There's more to the Vagrantfile such as setting up synced directories and
provisioning (I use puppet because I'm familiar with it).

So the IP addresses assigned to the 'node' VMs (the routers) are picked so
that the network in this case is a 4 node mesh and they are connected
together on the common networks. I'm sending data between the 'host' VMs
to test the networking applications (routing protocols) on the 'node' VMs.
When these Virtualbox VMs are started, a 'vboxnet' network interface is
created on the physical computer running all the VMs and I had to set
these to down to get traffic to route correctly between VMs and not just
directly from host to host through the physical computer. I couldn't
figure out how to do this in vagrant and so wrote a short script that
brings up the VMs and then sets the vboxnet interfaces down.

#!/usr/bin/env ruby
# -*- mode: ruby -*-

system("vagrant up")

cmd = "ip link show"
output = `#{cmd}`

output.each_line do |line|
if line =~ /(vboxnet[0-9]+)/ then
puts "Disable vbox interface #{$1}"
system("sudo ip link set #{$1} down")
end
end



This has been working well for my purposes.