Creating an OpenWrt netvm

[goo...@mamawe.net]

Hi all,

recently I've set up a laptop with Qubes OS and so far have been able to get most of my environment working within it. One thing is causing me some headaches, maybe I can get some hints here.

I'm trying to setup a DomU with OpenWrt to make some experiments. Since this is not Fedora, I first tried to setup a HVM DomU using the following command from within Dom0

qvm-create --label=orange --hvm --standalone \
--root-copy-from=openwrt-x86-generic-combined-ext4.img \
--mem=1 --vcpus=1

Starting this VM from Qubes VM Manager brought the laptop nearly to the knees till I managed to shutdown this VM.

Obiously I'm doing something wrong here, but I'm unable to see what.

Another point is: how do I get a console from this VM? All the other VMs have X11 and gnome-terminal but OpenWrt has only just a text console.

Thanks,

Mathias

[Marek Marczykowski-Górecki]

On Fri, Feb 06, 2015 at 09:14:45AM -0800, goo...@mamawe.net wrote:
> Hi all,
>
> recently I've set up a laptop with Qubes OS and so far have been able to get most of my environment working within it. One thing is causing me some headaches, maybe I can get some hints here.
>
> I'm trying to setup a DomU with OpenWrt to make some experiments. Since this is not Fedora, I first tried to setup a HVM DomU using the following command from within Dom0
>
> qvm-create --label=orange --hvm --standalone \
> --root-copy-from=openwrt-x86-generic-combined-ext4.img \
> --mem=1 --vcpus=1

1 (one) MB of memory is surely not enough, even for openwrt...

> Starting this VM from Qubes VM Manager brought the laptop nearly to the knees till I managed to shutdown this VM.
>
> Obiously I'm doing something wrong here, but I'm unable to see what.
>
> Another point is: how do I get a console from this VM? All the other
> VMs have X11 and gnome-terminal but OpenWrt has only just a text
> console.

If you successfully start that VM, there should be a window with VM
console (actually output of emulated VGA).


--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[Marek Marczykowski-Górecki]

On Fri, Feb 06, 2015 at 07:17:45PM +0100, Marek Marczykowski-Górecki wrote:
> On Fri, Feb 06, 2015 at 09:14:45AM -0800, goo...@mamawe.net wrote:
> > Hi all,
> >
> > recently I've set up a laptop with Qubes OS and so far have been able to get most of my environment working within it. One thing is causing me some headaches, maybe I can get some hints here.
> >
> > I'm trying to setup a DomU with OpenWrt to make some experiments. Since this is not Fedora, I first tried to setup a HVM DomU using the following command from within Dom0
> >
> > qvm-create --label=orange --hvm --standalone \
> > --root-copy-from=openwrt-x86-generic-combined-ext4.img \
> > --mem=1 --vcpus=1
>
> 1 (one) MB of memory is surely not enough, even for openwrt...
>
> > Starting this VM from Qubes VM Manager brought the laptop nearly to the knees till I managed to shutdown this VM.
> >
> > Obiously I'm doing something wrong here, but I'm unable to see what.
> >
> > Another point is: how do I get a console from this VM? All the other
> > VMs have X11 and gnome-terminal but OpenWrt has only just a text
> > console.
>
> If you successfully start that VM, there should be a window with VM
> console (actually output of emulated VGA).

BTW Currently Qubes does not support HVM as a netvm (you wont be able to
connect other VMs to it). This may be available somewhere around R3.2,
but the roadmap isn't set for it yet.

[goo...@mamawe.net]

Am Freitag, 6. Februar 2015 19:17:53 UTC+1 schrieb Marek Marczykowski-Górecki:

[...]

> > qvm-create --label=orange --hvm --standalone \
> > --root-copy-from=openwrt-x86-generic-combined-ext4.img \
> > --mem=1 --vcpus=1
>
> 1 (one) MB of memory is surely not enough, even for openwrt...

That was the fault. Using '--mem=512' proved to be enough RAM to start the VM.

[...]

> If you successfully start that VM, there should be a window with VM
> console (actually output of emulated VGA).

I get the console after starting the VM with qvm-start.

Thanks for your help,

Mathias

[goo...@mamawe.net]

Am Freitag, 6. Februar 2015 19:19:48 UTC+1 schrieb Marek Marczykowski-Górecki:

[...]

> BTW Currently Qubes does not support HVM as a netvm (you wont be able to
> connect other VMs to it). This may be available somewhere around R3.2,
> but the roadmap isn't set for it yet.

Then I'll try to setup a paravirtual VM. At least I have something to start with.

Thanks,

Mathias

[Marek Marczykowski-Górecki]

Currently there is no easy way to do that, but you can create some
standalone VM based on existing template, then replace its root.img with
openwrt. As there is no qrexec agent nor gui agent installed (or there
is?), the only way to access the console is "sudo xl console VMNAME".

[goo...@mamawe.net]

Am Freitag, 6. Februar 2015 20:00:10 UTC+1 schrieb Marek Marczykowski-Górecki:
> On Fri, Feb 06, 2015 at 10:54:00AM -0800, I wrote:
[...]

> > Then I'll try to setup a paravirtual VM. At least I have something to start with.
>
> Currently there is no easy way to do that, but you can create some
> standalone VM based on existing template, then replace its root.img with
> openwrt. As there is no qrexec agent nor gui agent installed (or there
> is?), the only way to access the console is "sudo xl console VMNAME".

Thanks for the hints, I'll try that.

If I can access the VM via network I won't need the console that much.

Regards,

Mathias

[goo...@mamawe.net]

Hi Marek,

Am Freitag, 6. Februar 2015 20:11:21 UTC+1 schrieb ich:
> Am Freitag, 6. Februar 2015 20:00:10 UTC+1 schrieb Marek Marczykowski-Górecki:
> > On Fri, Feb 06, 2015 at 10:54:00AM -0800, I wrote:
> [...]
> > > Then I'll try to setup a paravirtual VM. At least I have something to start with.
> >
> > Currently there is no easy way to do that, but you can create some
> > standalone VM based on existing template, then replace its root.img with
> > openwrt. As there is no qrexec agent nor gui agent installed (or there
> > is?), the only way to access the console is "sudo xl console VMNAME".
>
> Thanks for the hints, I'll try that.

This worked somewhat. But when I start the console I could see that the kernel and ramdisk came from Fedora. The Xen VM config file is autogenerated by qubes tools. So I can't change it directly to use the kernel from OpenWrt and just the OpenWrt disk image.

I assume some of the information is taken from /var/lib/qubes/qubes.xml and the kernel should be located at /var/lib/qubes/vm-kernels/some-dir/. Right?
Is it enough just to leave the kernel without ramdisk and modules in that directory?

How would I regenerate the Xen VM config after changing things in qubes.xml?
Or should I leave this file as it is and use some other tool?

Many thanks for your answers so far,

Mathias

[Marek Marczykowski-Górecki]

On Fri, Feb 06, 2015 at 12:13:38PM -0800, goo...@mamawe.net wrote:
> Hi Marek,
>
> Am Freitag, 6. Februar 2015 20:11:21 UTC+1 schrieb ich:
> > Am Freitag, 6. Februar 2015 20:00:10 UTC+1 schrieb Marek Marczykowski-Górecki:
> > > On Fri, Feb 06, 2015 at 10:54:00AM -0800, I wrote:
> > [...]
> > > > Then I'll try to setup a paravirtual VM. At least I have something to start with.
> > >
> > > Currently there is no easy way to do that, but you can create some
> > > standalone VM based on existing template, then replace its root.img with
> > > openwrt. As there is no qrexec agent nor gui agent installed (or there
> > > is?), the only way to access the console is "sudo xl console VMNAME".
> >
> > Thanks for the hints, I'll try that.
>
> This worked somewhat. But when I start the console I could see that the kernel and ramdisk came from Fedora. The Xen VM config file is autogenerated by qubes tools. So I can't change it directly to use the kernel from OpenWrt and just the OpenWrt disk image.
>
> I assume some of the information is taken from /var/lib/qubes/qubes.xml and the kernel should be located at /var/lib/qubes/vm-kernels/some-dir/. Right?

Exactly, then set "some-dir" as a kernel for that VM (for example using
qvm-prefs).

> Is it enough just to leave the kernel without ramdisk and modules in that directory?

There must be such files, but I guess empty files will do (unless you
really need initramfs.

One more possible problem - Qubes passes root=/dev/mapper/dmroot option to the
kernel. So you need to either force some other value - most likely it
will be enough to set root=... in "kernelopts" VM property.

> How would I regenerate the Xen VM config after changing things in qubes.xml?
> Or should I leave this file as it is and use some other tool?

You can provide own config file using qvm-start --custom-config, but use
it as a last resort.

[jonath...@gmail.com]

Hi, I'd just like an update on this, as I am looking to do the same.
OpenWRT seems the best for NetVM as it has all that I could think of.
Also multiple instances for VPNs should not use too much ram.
So if you got it running, could you post your config in git ?
Thanks

[Chris Laprise]

This might not be a good idea. Qubes security model says that NICs are
high-risk devices that should be isolated by themselves. If you combine
that with your firewall, VPN and other sensitive routing infrastructure
then a breech caused by your NIC could allow an attacker to monitor and
control your firewall, VPN etc.

OTOH, combining firewall and VPN functions in the same VM is probably
fine since those processes are relatively low-risk.

Chris