Unikernels and Qubes

[Andrew]

The idea of unikernels in Qubes is not entirely new; it's come up on the
lists a few times. This doesn't seem to have been posted to the list
yet, though: https://northox.github.io/qubes-rumprun/

The above blog post makes some arguments for integrating some unikernels
into Qubes for various things: TCP/IP stack vulnerability mitigation,
in-line filters on communication channels between VMs (I always imagined
using a MirageOS TLS wrap/unrwap unikernel to avoid OpenSSL exploits),
secure file conversion, and generally promoting increased disaggregation
and finer-grained isolation.

This post asks for feedback and poses a few questions to be answered.

> Now, I'm looking for constructive feedback from Qubes' community and
> will try to answer a few questions: What exactly would need to be
> modified on Qubes' side to be part of the default installation?
> What's the effort? What's the best course of action? Is it viable in
> practice? Does it make sense?

I don't have the answers to these questions, but maybe others can chime
in to help answer them.

Andrew

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Fri, Nov 06, 2015 at 02:26:43PM +0000, Andrew wrote:
> The idea of unikernels in Qubes is not entirely new; it's come up on the
> lists a few times. This doesn't seem to have been posted to the list
> yet, though: https://northox.github.io/qubes-rumprun/
>
> The above blog post makes some arguments for integrating some unikernels
> into Qubes for various things: TCP/IP stack vulnerability mitigation,
> in-line filters on communication channels between VMs (I always imagined
> using a MirageOS TLS wrap/unrwap unikernel to avoid OpenSSL exploits),
> secure file conversion, and generally promoting increased disaggregation
> and finer-grained isolation.

Great! This is few steps further than we've already planned for some
time:
https://github.com/QubesOS/qubes-issues/issues/1310

> This post asks for feedback and poses a few questions to be answered.
>
> > Now, I'm looking for constructive feedback from Qubes' community and
> > will try to answer a few questions:

> Does it make sense?

Yes, yes, yes! :)

> What exactly would need to be
> modified on Qubes' side to be part of the default installation?

We're already thinking about starting some alternative kernels and it
should be pretty easy. The easiest way doesn't require much changes,
simply drop the binary into /var/lib/qubes/vm-kernels/SOME_NAME/vmlinuz
and set "SOME_NAME" as VM kernel (qvm-prefs -s VMNAME kernel SOME_NAME).

Some details:
https://github.com/QubesOS/qubes-issues/issues/1354

Also Qubes already start new VMs paused first, exactly for the reason
mentioned on the blog - to setup some configuration entries (previously
in XenStore, now in QubesDB). This can be reused for some additional
settings for such unikernel. That code can launch additional hooks for
some extensions.

Those hooks will substantially change in Qubes 4.0, but since this part
of the integration isn't much work, it will probably not hurt to simply
do it again for new API in Qubes 4.0. All other parts should work the
same in Qubes 3.x and 4.x.

> > What's the effort? What's the best course of action? Is it viable in
> > practice?

The first think would be just starting some unikernel, just to have
testing environment. This would probably include that xenstore
modifications. Take a look here:
https://github.com/QubesOS/qubes-core-admin/blob/master/core-modules/000QubesVm.py#L1045-L1094

Example of hooks usage:
https://github.com/QubesOS/qubes-core-admin-linux/blob/master/appmenus-scripts/qubes-core-appmenus.py#L352-L360

For first you can ignore qrexec and gui - simply hit Ctrl-C when
qvm-start tries to connect qrexec. Then access VM console using sudo xl
console (or virsh -c xen:/// console).

Then it would come to porting vchan (easy), and qrexec (slightly
harder). Vchan and gui-agent are already working in mini-os (newlib)
based stubdomain. Vchan works just as it is (requires libxenctrl and
libxenstore, but it shouldn't be a problem). Stubdomain gui-agent is
written from scratch, with very limited functionality, so probably not
really useful here (doesn't help with qrexec porting). Current qrexec
implementation uses separate process for each connection (also each
command call gets its own vchan connection), so probably it require
substantial changes for running under rumprun unikernel. Maybe R2 qrexec
agent (single threaded application) would be a better starting point, or
at least some inspiration.

Links:
1. Vchan (there is Makefile.stubdom), which really is just an interface
to libxenvchan (an abstraction layer for possible other non-Xen
implementations):
https://github.com/qubesos/qubes-core-vchan-xen
2. Real libxenvchan:
http://xenbits.xen.org/gitweb/?p=xen.git;a=tree;f=tools/libvchan;h=044f954f3e24447ed88858d7f22289c148ba3561;hb=HEAD
3. Patches to make it working in stubdomain:
https://github.com/QubesOS/qubes-vmm-xen/blob/xen-4.4/series.conf#L56-L62
4. gui-agent for stubdomain:
https://github.com/QubesOS/qubes-gui-agent-xen-hvm-stubdom/
5. Linux qrexec-agent:
https://github.com/QubesOS/qubes-core-agent-linux/tree/master/qrexec
https://github.com/QubesOS/qubes-linux-utils/tree/master/qrexec-lib
5a. R2 qrexec-agent:
https://github.com/QubesOS/qubes-core-agent-linux/tree/release2/qrexec
6. Qrexec protocol documentation (rather hard to follow...):
https://www.qubes-os.org/doc/qrexec3-implementation/

If you want to integrate it with Qubes Builder, you'll need to create
Makefile.builder file in the repository, which will point spec file to
build the rpm package. Documentation:
High level:
https://www.qubes-os.org/doc/qubes-builder/
https://www.qubes-os.org/doc/qubes-r3-building/
Details:
https://www.qubes-os.org/doc/qubes-builder-details/
(and every Qubes component's Makefile.builder - take a look at vmm-xen)

Some more examples:
https://github.com/QubesOS/qubes-template-configs

If the above is too much detailed, let me know, I'd gladly provide more
assistance.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBCAAGBQJWPMkLAAoJENuP0xzK19csHKAH/3jRjCdRFdQGmOA85ZhvdMgE
vUFw7cpBKiRmM67ba4IBNrMz9mk2MXzT1/qacqOX7nszMRU0cCnZQLxPKRsumZff
ryLT43f64bZr9KTIKGDg4PzpvMQakjhq0uKg05tTZTHxSPRCCbojifXUFi1m38mz
OBZ7p7nhpOzZR5n/1eu76z3TAxbkuOGBJ+4u95yMWXVOOooBLfYPbFG4Rgvds0jE
a1qNKnTRsp7AhtksfjO4ojmxuO/OLq+akPRHLUSw90jmvMIZBUpimbMybx0gv0CJ
3182X5bXP4SHBDVktSODsl2gOUjWBsD41v/lWzbQhnrwXjDZwvmxppte3dPgroY=
=FcRH
-----END PGP SIGNATURE-----

[Thomas Leonard]

Great! I've been experimenting with running MirageOS unikernels under Qubes using some quick scripts here:

  https://github.com/talex5/qubes-test-mirage
 
With these, I can build a unikernel in an AppVM and then run it. e.g.

~/w/c/s/console (master) $ mirage configure --xen
~/w/c/s/console (master) $ make
~/w/c/s/console (master) $ test-mirage mir-console.xen
Waiting for 'Ready'... OK
Waiting for 'Booting'... Uploading 'mir-console.xen' (4139720 bytes)
OK
Xen Minimal OS!
Initialising timer interface
Initialising console ... done.
hello
world

However, this isn't properly integrated into Qubes (it just starts a Xen domain using "xl"). One problem is that I can't give the VM much memory, because it seems that Qubes tries to return unused memory to dom0. Luckily, it leaves a few MB spare, and that's often enough for Mirage!

> The easiest way doesn't require much changes,
> simply drop the binary into /var/lib/qubes/vm-kernels/SOME_NAME/vmlinuz and set "SOME_NAME" as VM kernel (qvm-prefs -s VMNAME kernel SOME_NAME).

I tried a few things to get Qubes to see the VM, but never got it to work. Using the above command, I get:

[tal@dom0 ~]$ mkdir /var/lib/qubes/vm-kernels/mirage-qubes
[tal@dom0 ~]$ cp /var/lib/test-mirage/mir-test.xen /var/lib/qubes/vm-kernels/mirage-qubes/vmlinuz
[ create new Fedora AppVM called "my-new-vm" using qubes-manager ]
[tal@dom0 ~]$ qvm-prefs -s my-new-vm kernel mirage-qubes
ERROR: Kernel 'mirage-qubes' not properly installed: missing modules.img file

Some details:
https://github.com/QubesOS/qubes-issues/issues/1354

Also Qubes already start new VMs paused first, exactly for the reason
mentioned on the blog - to setup some configuration entries (previously
in XenStore, now in QubesDB). This can be reused for some additional
settings for such unikernel. That code can launch additional hooks for
some extensions.

Those hooks will substantially change in Qubes 4.0, but since this part
of the integration isn't much work, it will probably not hurt to simply
do it again for new API in Qubes 4.0. All other parts should work the
same in Qubes 3.x and 4.x.

> > What's the effort? What's the best course of action? Is it viable in
> > practice?

The first think would be just starting some unikernel, just to have
testing environment. This would probably include that xenstore
modifications. Take a look here:
https://github.com/QubesOS/qubes-core-admin/blob/master/core-modules/000QubesVm.py#L1045-L1094

Example of hooks usage:
https://github.com/QubesOS/qubes-core-admin-linux/blob/master/appmenus-scripts/qubes-core-appmenus.py#L352-L360

For first you can ignore qrexec and gui - simply hit Ctrl-C when
qvm-start tries to connect qrexec. Then access VM console using sudo xl
console (or virsh -c xen:/// console).

Then it would come to porting vchan (easy),

Is this vchan the same protocol as this?

  https://github.com/mirage/ocaml-vchan
 
If so, I guess Mirage already supports it.

A quick tutorial showing how to run MirageOS with Qubes would be very handy.

BTW, I recently added a Mirage "pioneer project" to replace the Linux FirewallVM with a Mirage one:

  https://github.com/mirage/mirage-www/wiki/Pioneer-Projects#qubes-firewallvm
 
Let me know if you have any links or other useful information that should be added there...

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Nice!

> However, this isn't properly integrated into Qubes (it just starts a Xen
> domain using "xl"). One problem is that I can't give the VM much memory,
> because it seems that Qubes tries to return unused memory to dom0. Luckily,
> it leaves a few MB spare, and that's often enough for Mirage!
>
> > The easiest way doesn't require much changes,
> > simply drop the binary into /var/lib/qubes/vm-kernels/SOME_NAME/vmlinuz
> and set "SOME_NAME" as VM kernel (qvm-prefs -s VMNAME kernel SOME_NAME).
>
> I tried a few things to get Qubes to see the VM, but never got it to work.
> Using the above command, I get:
>
> [tal@dom0 ~]$ mkdir /var/lib/qubes/vm-kernels/mirage-qubes
> [tal@dom0 ~]$ cp /var/lib/test-mirage/mir-test.xen
> /var/lib/qubes/vm-kernels/mirage-qubes/vmlinuz
> [ create new Fedora AppVM called "my-new-vm" using qubes-manager ]
> [tal@dom0 ~]$ qvm-prefs -s my-new-vm kernel mirage-qubes
> ERROR: Kernel 'mirage-qubes' not properly installed: missing modules.img
> file

touch modules.img there. Probably you'll also need initramfs. And it
needs to be something that xl will like to load as initramfs. For
example gzipped empty file:
cat /dev/null | gzip > .../initramfs

Yes, it looks so.

> and qrexec (slightly
> > harder). Vchan and gui-agent are already working in mini-os (newlib)
> > based stubdomain. Vchan works just as it is (requires libxenctrl and
> > libxenstore, but it shouldn't be a problem). Stubdomain gui-agent is
> > written from scratch, with very limited functionality, so probably not
> > really useful here (doesn't help with qrexec porting). Current qrexec
> > implementation uses separate process for each connection (also each
> > command call gets its own vchan connection), so probably it require
> > substantial changes for running under rumprun unikernel. Maybe R2 qrexec
> > agent (single threaded application) would be a better starting point, or
> > at least some inspiration.

I forgot to add QubesDB to that list. It is very similar to xenstore,
but somehow simpler (for example no transactions). And is running over
vchan.

> > Links:
> > 1. Vchan (there is Makefile.stubdom), which really is just an interface
> > to libxenvchan (an abstraction layer for possible other non-Xen
> > implementations):
> > https://github.com/qubesos/qubes-core-vchan-xen
> > 2. Real libxenvchan:
> >
> > http://xenbits.xen.org/gitweb/?p=xen.git;a=tree;f=tools/libvchan;h=044f954f3e24447ed88858d7f22289c148ba3561;hb=HEAD
> > 3. Patches to make it working in stubdomain:
> > https://github.com/QubesOS/qubes-vmm-xen/blob/xen-4.4/series.conf#L56-L62
> > 4. gui-agent for stubdomain:
> > https://github.com/QubesOS/qubes-gui-agent-xen-hvm-stubdom/
> > 5. Linux qrexec-agent:
> > https://github.com/QubesOS/qubes-core-agent-linux/tree/master/qrexec
> > https://github.com/QubesOS/qubes-linux-utils/tree/master/qrexec-lib

> > 5a <https://github.com/QubesOS/qubes-linux-utils/tree/master/qrexec-lib5a>.

> > R2 qrexec-agent:
> > https://github.com/QubesOS/qubes-core-agent-linux/tree/release2/qrexec
> > 6. Qrexec protocol documentation (rather hard to follow...):
> > https://www.qubes-os.org/doc/qrexec3-implementation/
> >
> > If you want to integrate it with Qubes Builder, you'll need to create
> > Makefile.builder file in the repository, which will point spec file to
> > build the rpm package. Documentation:
> > High level:
> > https://www.qubes-os.org/doc/qubes-builder/
> > https://www.qubes-os.org/doc/qubes-r3-building/
> > Details:
> > https://www.qubes-os.org/doc/qubes-builder-details/
> > (and every Qubes component's Makefile.builder - take a look at vmm-xen)
> >
> > Some more examples:
> > https://github.com/QubesOS/qubes-template-configs
> >
> > If the above is too much detailed, let me know, I'd gladly provide more
> > assistance.
> >
>
> A quick tutorial showing how to run MirageOS with Qubes would be very handy.
>
> BTW, I recently added a Mirage "pioneer project" to replace the Linux
> FirewallVM with a Mirage one:
>
>
> https://github.com/mirage/mirage-www/wiki/Pioneer-Projects#qubes-firewallvm

Actually the sole firewall functionality may not need qrexec at all. But
QubesDB still will be needed. Basically firewall settings (currently in
iptables syntax...) are exposed through QubesDB, then firewallvm is
running a service which monitor appropriate QubesDB keys (using
qubesdb-watch) and apply them when anything changed.

> Let me know if you have any links or other useful information that should
> be added there...

I'm afraid the only QubesDB documentation for now is this header file:
https://github.com/QubesOS/qubes-core-qubesdb/blob/master/include/qubesdb.h

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBCAAGBQJWSPFPAAoJENuP0xzK19csLbsH/0uiJBqJYj4wUAmmA+bcijI3
pBzJMShHua/C6U+TadNM0uuu5zX0QN29ttUUPGvG47UD7sJoUIkl0QcY8TH9JjCA
iwgTZMd1jOaqzL0LhSeCEJpAopFaYfkt5wQAmogyMr/Axy4I7mlLo0jomO2hr6eF
FM3hZ9K6IH4mSpPJcmbNdXkiuudJX79ctFSVyokTwP6T2vWsUibWe+mWjIb1Qzpz
lV0hgppmmKJsmGnH/18ms8f/HQNEzWj3wCgUdObahiZh5XRz6fAnVntfHNzBA0hM
34oQHB5uNFOB3Ltj5Dz/Kr/nACZYzGLR/KmQTOqVeTMecEDiyiPSiL+KM5Bl4ig=
=C+As
-----END PGP SIGNATURE-----

[Vít Šesták]

Unikernels are cool. But don't overvalue them…

They will likely trim the memory consumption down and encourage compartization. This has an arguably positive effect on security.

They might bring more diversity. Using almost any non-Linux kernel for FirewallVM seems to be a good counter-measure against chained network exploits, especially when combined with minimal-state VMs. (Non-Linux NetVM would be cool, but there might be troubles with drivers.)

But effect of the simplicity is double-edged. On one hand, simplicity makes some security reasoning simpler. On the other hand, there is not so much defense-in-depth. For many non-RCE vulnerabilities, the complexity of Linux kernel may help in multiple ways. For example:
* filesystem permissions — reduces impact of path traversals
* memory barriers — reduces risk of out-of-bounds reads (less information potentially leaked) and out-of-bounds writes (smaller attack surface)

It is also unclear if complexity of Linux kernel is a considerable issue in the Qubes context. In a typical usage, attacker has pretty limited surface. They have usually no or limited remote network access, they usually can't run any code (and thus they can't escalate root privileges) and so on. On the other hand, we have seen some unexpected attack vector like CSRF, DNS rebind attack. In these cases, some supposedly non-exposed interface was exposed.

I have no strong opinion which effect is more significant in general. Maybe this can't be even properly discussed without focusing on some narrower case. For truly minimal unikernel without filesystem and without any stored state, advantages will probably outweight the disadvantages. For a rather-complex unikernel combining multiple processes (e.g. NetVM combining ethernet drivers, Wi-Fi drivers, NetworkManager and NM applet), the security benefit of simplicity might be even negative.

Regards,
Vít Šesták 'v6ak'

[Paul Harvey]

I've recently started my own MirageOS experiments (though I have very
little spare time, so not in a position to contribute much). OCaml is
a great language (syntax notwithstanding) and I'd hope this brings
substantial benefits regarding memory corruption style bugs (language
choice isn't a panacea, but certain bug classes should be almost
entirely eradicated... and I guess replaced with others?)

However, rumprun doesn't (yet!) seem to have much in the way of memory
corruption mitigations (or does it?). On the other hand, unikernels
like rumprun are perhaps best placed to be able to do W^X and similar
memory protections. We just have to accept that minority experimental
projects present security risks of their own which potentially
outweigh architectural benefits, at least in the beginning. Is there
much literature on unikernel attacks?

But who cares? :) I do think unikernels have increasingly important
roles to play in the near future.

Excited to see this interest growing.

On 16 November 2015 at 08:00, Vít Šesták
<groups-no-private-mail--con...@v6ak.com>
wrote:

> --
> You received this message because you are subscribed to the Google Groups
> "qubes-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to qubes-users...@googlegroups.com.
> To post to this group, send email to qubes...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/qubes-users/47b29b58-ce88-43c0-913f-2375bdaef29a%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

[Thomas Leonard]

On 15 November 2015 at 20:55, Marek Marczykowski-Górecki

Ah, that fixed it! I've updated the scripts
(https://github.com/talex5/qubes-test-mirage). It now looks like this:

$ test-mirage mir-console.xen
Waiting for 'Ready'... OK

Uploading 'mir-console.xen' (4184144 bytes)
Waiting for 'Booting'... OK
ERROR: VM already stopped!
--> Creating volatile image: /var/lib/qubes/appvms/mirage-test/volatile.img...
--> Loading the VM (type = AppVM)...
--> Starting Qubes DB...
--> Setting Qubes DB info for the VM...
--> Updating firewall rules...
--> Starting the VM...
MirageOS booting...

Initialising timer interface
Initialising console ... done.
hello
world

...

(it stops watching the qvm-start output when it sees "--> Starting the
qrexec daemon...", since that will never finish currently)

Would be nice if there were a way to stop Qubes passing the Linux
kernel arguments. The next version of mirage will try to parse them
and fail.


--
Dr Thomas Leonard http://roscidus.com/blog/
GPG: DA98 25AE CAD0 8975 7CDA BD8E 0713 3F96 CA74 D8BA

[Vít Šesták]

Paul, I don't think that comparing app written in memory-unsafe language for Linux with app written in memory-safe language for an unikernel is fair. (Except some edge cases based on already available applications.) But sure, memory-safe language might lessen the disadvantages of kernel. Missing filesystem might effectively mitigate path traversals. And so on.

Thomas, have you tried qvm-prefs -s <vm name> kernelopts '<desired kernel options>' ? I hope this will do the trick.

Regards,
Vít Šesták 'v6ak'

[Thomas Leonard]

On 16 November 2015 at 20:41, Vít Šesták
<groups-no-private-mail--con...@v6ak.com>
wrote:

> Paul, I don't think that comparing app written in memory-unsafe language for
> Linux with app written in memory-safe language for an unikernel is fair.
> (Except some edge cases based on already available applications.) But sure,
> memory-safe language might lessen the disadvantages of kernel. Missing
> filesystem might effectively mitigate path traversals. And so on.

Yes, there's a big difference between e.g. rump (written in C,
provides a POSIX API and runs many existing applications) and Mirage
(OCaml, provides its own new APIs, for new services).

For Mirage we're not constrained by POSIX, so we can fix bad APIs. For
FirewallVM we probably wouldn't have a filesystem, but if we did we
could implement it without support for "..", for example.

> Thomas, have you tried qvm-prefs -s <vm name> kernelopts '<desired kernel
> options>' ? I hope this will do the trick.

It seems I can use that to add more arguments, but even when set to
the empty string, my unikernel still gets called with:

root=/dev/mapper/dmroot ro nomodeset console=hvc0 rd_NO_PLYMOUTH 3

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "qubes-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/qubes-users/h03-1hiNMCc/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> qubes-users...@googlegroups.com.
> To post to this group, send email to qubes...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/qubes-users/9cd96afc-aaa7-4327-b331-919de790959b%40googlegroups.com.

>
> For more options, visit https://groups.google.com/d/optout.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Tue, Nov 17, 2015 at 10:10:15AM +0000, Thomas Leonard wrote:
> On 16 November 2015 at 20:41, Vít Šesták
> <groups-no-private-mail--con...@v6ak.com>
> wrote:
> > Paul, I don't think that comparing app written in memory-unsafe language for
> > Linux with app written in memory-safe language for an unikernel is fair.
> > (Except some edge cases based on already available applications.) But sure,
> > memory-safe language might lessen the disadvantages of kernel. Missing
> > filesystem might effectively mitigate path traversals. And so on.
>
> Yes, there's a big difference between e.g. rump (written in C,
> provides a POSIX API and runs many existing applications) and Mirage
> (OCaml, provides its own new APIs, for new services).
>
> For Mirage we're not constrained by POSIX, so we can fix bad APIs. For
> FirewallVM we probably wouldn't have a filesystem, but if we did we
> could implement it without support for "..", for example.
>
> > Thomas, have you tried qvm-prefs -s <vm name> kernelopts '<desired kernel
> > options>' ? I hope this will do the trick.
>
> It seems I can use that to add more arguments, but even when set to
> the empty string, my unikernel still gets called with:
>
> root=/dev/mapper/dmroot ro nomodeset console=hvc0 rd_NO_PLYMOUTH 3

Theoretically you can edit /usr/share/qubes/vm-template.xml, but this is
global file for every VM, so you'll to add the default parameters for
each Linux VM manually using kernelopts parameter (otherwise VMs would
not boot).

This is something which should be improved when integrating unikernels
into Qubes, but I think it's ok for just experiments/development.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBCAAGBQJWSw8HAAoJENuP0xzK19csHi8IAIgtnHcf6EBxurnOCqoGre/X
BxRI6mh8pZiefRN3JlTnFD0iNeBcQszM4UP3nVbM6zawrGNDqbflsO0drgfEpRi/
r8Q0CFewSEbsYEMGYBRgUi2FndzZEhkCuyvu1e7SL+bsb2Uh4jRkH+1PbuXvJoVb
Icj6ShGYBC8eFQrPQcY8kesw5yfJJj8S9wW8YoZ027Huh454B46bqoPyL/W5HvhI
J79txA0VbfIw4BMRc58d3nhyqtVcGgKcb9CQxTo/XZyKXPN4jIK40z2wmxgtU4TK
a2xUDemCQDyF6D5S6n0MfaOQ7Om4M5ZQNXx3c9qavvfyrmUD9jbbn2us4F0Zyjg=
=cDsR
-----END PGP SIGNATURE-----

[Thomas Leonard]

[...]

> I've updated the scripts
> (https://github.com/talex5/qubes-test-mirage). It now looks like this:
>
> $ test-mirage mir-console.xen
> Waiting for 'Ready'... OK
> Uploading 'mir-console.xen' (4184144 bytes)
> Waiting for 'Booting'... OK
> ERROR: VM already stopped!
> --> Creating volatile image: /var/lib/qubes/appvms/mirage-test/volatile.img...
> --> Loading the VM (type = AppVM)...
> --> Starting Qubes DB...
> --> Setting Qubes DB info for the VM...
> --> Updating firewall rules...
> --> Starting the VM...
> MirageOS booting...
> Initialising timer interface
> Initialising console ... done.
> hello
> world
> ...
>
> (it stops watching the qvm-start output when it sees "--> Starting the
> qrexec daemon...", since that will never finish currently)

I had a go at qrexec this weekend. I now have a simple unikernel that
responds to commands from "qvm-run --nogui":

https://github.com/talex5/mirage-qubes

$ test-mirage mir-qubes-test.xen

Waiting for 'Ready'... OK

Uploading 'mir-qubes-test.xen' (4187256 bytes)

Waiting for 'Booting'... OK

--> Creating volatile image: /var/lib/qubes/appvms/mirage-test/volatile.img...
--> Loading the VM (type = AppVM)...
--> Starting Qubes DB...
--> Setting Qubes DB info for the VM...
--> Updating firewall rules...
--> Starting the VM...

--> Starting the qrexec daemon...

Waiting for VM's qrexec agent.connected

MirageOS booting...
Initialising timer interface
Initialising console ... done.

info: Starting qrexec agent; waiting for client...
info: Got connection
info: Handshake done; client version is 2

It currently offers "echo" and "quit" services. e.g.

[tal@dom0 bin]$ qvm-run -p --nogui mirage-test echo
Hi user! Please enter a string:
Hello
You wrote "Hello". Bye.

A few points where I found the documentation a little unclear:

- In addition to "exec_cmdline", I found I had to implement
"just_exec" too (for use without "-p"). It seems that you must still
set up a vchan for the session, but only use it to return the exit
status.

- The exit status message is defined to be the size of an int. I think
it would be better to specify a size, rather than requiring programs
to change the format depending on the platform. For now, I used
(https://github.com/talex5/mirage-qubes/blob/229c4f9678cf871d5adff412827eafedee7b1c1b/qrexec_protocol.ml#L16):

cstruct exit_status {
uint64_t return_code;
} as little_endian

- It appears that exec commands have a trailing \0. I assume this is a
convenience for C programs, but it seems risky to rely on it.

My qrexec module should be reusable. If other people want to try it,
the API is at https://github.com/talex5/mirage-qubes/blob/master/qrexec.mli
and the example code is here:

https://github.com/talex5/mirage-qubes/blob/master/unikernel.ml

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Wow, I was thinking it would be much harder.

Where can I get ocaml version of GnuPG ;) That would make
mirage-keys-vault.

> A few points where I found the documentation a little unclear:
>
> - In addition to "exec_cmdline", I found I had to implement
> "just_exec" too (for use without "-p"). It seems that you must still
> set up a vchan for the session, but only use it to return the exit
> status.

Yes. Generally qrexec protocol documentation should be improved:
https://github.com/QubesOS/qubes-issues/issues/1392

> - The exit status message is defined to be the size of an int. I think
> it would be better to specify a size, rather than requiring programs
> to change the format depending on the platform. For now, I used
> (https://github.com/talex5/mirage-qubes/blob/229c4f9678cf871d5adff412827eafedee7b1c1b/qrexec_protocol.ml#L16):
>
> cstruct exit_status {
> uint64_t return_code;
> } as little_endian

Yes, that's a bug. And I think it should be uint32_t... (it's int, not
long).

> - It appears that exec commands have a trailing \0. I assume this is a
> convenience for C programs, but it seems risky to rely on it.

qrexec control channel comes from dom0, so can be trusted. Anyway will
add some safeguard.

> My qrexec module should be reusable. If other people want to try it,
> the API is at https://github.com/talex5/mirage-qubes/blob/master/qrexec.mli
> and the example code is here:
>
> https://github.com/talex5/mirage-qubes/blob/master/unikernel.ml

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBCAAGBQJWUlLyAAoJENuP0xzK19csNwkH/j2Hv9mscc+JulhdgFkgy/lX
tE2JfDC56i0tD2U4xlyfVqBF73ITDnigfg6W3UUK3Ha2zvLUEkfDTDrhfHeyKMTm
SK0KV6sISdn06LV+DqTlEAJyulTzQ1E8FmSkqZS4Y+PNrHuAKPPIle+XuA2XWwOv
3/M7ENqvR5JyzYI07KEapBXSoEDMgXxxJx85jySvzBuGGDRJzZRNWn67X2GhPxyS
gZ52/u0b3mrZV9+hLmKT3CCnKx1WXngKrX/duR2zR9A73l+zm/Lb2uifACMLeYaS
9ZWm4YNfWML7ZCeOC+UVdSTrSTTBxgS8iiKqQi8AQSfR5RMMlKNA03LGGXbTFHo=
=cSPU
-----END PGP SIGNATURE-----

[Vít Šesták]

On Monday, November 23, 2015 at 12:42:45 AM UTC+1, Marek Marczykowski-Górecki wrote:

Where can I get ocaml version of GnuPG ;) That would make
mirage-keys-vault.

Well, I am somewhat skeptic about OCaml GnuPG port. In my opinion, cyprography primitives are hard even in something low-level. Preventing side channels will be even harder when written in something high-level like OCaml. Maybe some combined C + OCaml port would be useful.

I am sorry if I look too skeptically in this topic, but I see some potential drawbacks there and I hope that catching them early might be less painful than catching them late. In fact, I still appreciate the effort put in unikernels. Benefits were mentioned mainly by others.

Regards,
Vít Šesták 'v6ak'

[Thomas Leonard]

On 23 November 2015 at 17:41, Vít Šesták
<groups-no-private-mail--con...@v6ak.com>
wrote:

> On Monday, November 23, 2015 at 12:42:45 AM UTC+1, Marek
> Marczykowski-Górecki wrote:
>>
>> Where can I get ocaml version of GnuPG ;) That would make
>> mirage-keys-vault.

I don't think Mirage has GnuPG support currently, though it does have
TLS so many of the primitives should be available:

https://github.com/mirleft/ocaml-tls

> Well, I am somewhat skeptic about OCaml GnuPG port. In my opinion,
> cyprography primitives are hard even in something low-level. Preventing side
> channels will be even harder when written in something high-level like
> OCaml. Maybe some combined C + OCaml port would be useful.

Indeed, Mirage uses C for low-level crypto primitives:

"We use public domain (or BSD licenced) C sources for the simple cores
of AES, 3DES, MD5, SHA and SHA2. The impact of errors in this code is
constrained: they contain no recursion, and they perform no
allocation, simply filling in caller-supplied fixed-size buffer by
appropriate bytes."

https://mirage.io/blog/introducing-nocrypto

> I am sorry if I look too skeptically in this topic, but I see some potential
> drawbacks there and I hope that catching them early might be less painful
> than catching them late. In fact, I still appreciate the effort put in
> unikernels. Benefits were mentioned mainly by others.
>
> Regards,
> Vít Šesták 'v6ak'

[Thomas Leonard]

On Sunday, November 22, 2015 at 11:42:45 PM UTC, Marek Marczykowski-Górecki wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Sun, Nov 22, 2015 at 10:53:25PM +0000, Thomas Leonard wrote:

[...]

I asked about this on the Mirage list. Hannes Mehnert (co-author of ocaml-tls) said "If someone has energy and resources for OpenPGP: I'm happy to help out":

http://lists.xenproject.org/archives/html/mirageos-devel/2015-11/msg00144.html
http://lists.xenproject.org/archives/html/mirageos-devel/2015-11/msg00151.html

> A few points where I found the documentation a little unclear:
>
> - In addition to "exec_cmdline", I found I had to implement
> "just_exec" too (for use without "-p"). It seems that you must still
> set up a vchan for the session, but only use it to return the exit
> status.

Yes. Generally qrexec protocol documentation should be improved:
https://github.com/QubesOS/qubes-issues/issues/1392

> - The exit status message is defined to be the size of an int. I think
> it would be better to specify a size, rather than requiring programs
> to change the format depending on the platform. For now, I used
> (https://github.com/talex5/mirage-qubes/blob/229c4f9678cf871d5adff412827eafedee7b1c1b/qrexec_protocol.ml#L16):
>
> cstruct exit_status {
>   uint64_t return_code;
> } as little_endian

Yes, that's a bug. And I think it should be uint32_t... (it's int, not
long).

Ah, you're right. gcc on Debian amd64 at least uses 32-bit ints.
 

> - It appears that exec commands have a trailing \0. I assume this is a
> convenience for C programs, but it seems risky to rely on it.

qrexec control channel comes from dom0, so can be trusted. Anyway will
add some safeguard.

> My qrexec module should be reusable. If other people want to try it,
> the API is at https://github.com/talex5/mirage-qubes/blob/master/qrexec.mli
> and the example code is here:
>
> https://github.com/talex5/mirage-qubes/blob/master/unikernel.ml

 
I've now added basic support for the GUI protocol too, so that the Qubes sees the VM as started successfully. Initially, I just opened a vchan server on port 6000. That worked, but I ended up with lots of old qubes-quid processes in dom0, so I've added the handshake too now:

~/w/qubes (master) $ test-mirage mir-qubes-test.xen mirage-test

Waiting for 'Ready'... OK

Uploading 'mir-qubes-test.xen' (4396616 bytes) to "mirage-test"

Waiting for 'Booting'... OK
--> Creating volatile image: /var/lib/qubes/appvms/mirage-test/volatile.img...
--> Loading the VM (type = AppVM)...
--> Starting Qubes DB...
--> Setting Qubes DB info for the VM...
--> Updating firewall rules...
--> Starting the VM...
--> Starting the qrexec daemon...
Waiting for VM's qrexec agent.connected

--> Starting Qubes GUId...
Connecting to VM's GUI agent: .connected
--> Sending monitor layout...
--> Waiting for qubes-session...
Connecting to mirage-test console...

MirageOS booting...
Initialising timer interface
Initialising console ... done.

2015-11-29 19:10.23: info: qrexec-agent: waiting for client...
2015-11-29 19:10.23: info: gui-agent: waiting for client...
2015-11-29 19:10.23: info: qrexec-agent: client connected, using protocol version 2
2015-11-29 19:10.23: info: gui-agent: client connected (screen size: 6720x2160)
2015-11-29 19:10.23: info: agents connected in 0.113 s (CPU time used since boot: 0.007 s)
gnttab_stubs.c: initialised mini-os gntmap
2015-11-29 19:10.23: info: qrexec-agent: "QUBESRPC qubes.SetMonitorLayout dom0" returned exit status 1
2015-11-29 19:10.23: info: qrexec-agent: "QUBESRPC qubes.WaitForSession none" returned exit status 1

With this, the VM can be started and stopped with qubes-manager and you get a green status indicator.

Booting is a little slow though - starting the AppVM with qvm-start takes about 2.4s in total, of which Mirage reports it only used 0.007 s of CPU.

(BTW, the formatting on the GUI protocol page seems to be messed up at the end: https://www.qubes-os.org/doc/gui/)

What would I need to do to implement a Firewall VM? Do I just watch XenStore for new network interfaces? Can I get the name of the client VMs?

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Some of that time is probably preparing environment for the VM (getting
memory from other VMs, setting up block backends - even when not used by
the VM, etc).

> (BTW, the formatting on the GUI protocol page seems to be messed up at the
> end: https://www.qubes-os.org/doc/gui/)

Uhmm...

> What would I need to do to implement a Firewall VM? Do I just watch
> XenStore for new network interfaces? Can I get the name of the client VMs?

You'll need to have proper xen network backend - so yes, watch xenstore
for the interfaces. You'll get VM IP address from there also. And you
shouldn't be able to get VM name.

Then you'll need to watch QubesDB for actual firewall settings.
Currently they are in iptables syntax, so probably not the most
convenient one... This will be improved in Qubes 4.0 (or even 4.1).
But for now, it still should be possible to handle that format - there
are only few of iptables options combination, so you should be able to
parse that.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----

Version: GnuPG v2

iQEcBAEBCAAGBQJWW2GVAAoJENuP0xzK19csBWIIAInx0FfAs0BCP8jEdvRua0xF
FstkIPnM+h52qqpSMD9dP98h615usWBIDsm/G/R3flHeCAXui3aOvapwk3IIuLt4
SN873yI9bd6IB69lNDOBXybPs6IgO5QoD96bemZqzEqizxwFSycG/5Y55gYyi3dH
2b2ch4hNBARGpx2tyIh3oHI3w6AABXQk75BC+eOUm+1ksxQl7AQptEw6D/gsZQux
34UhMxNOVn56G7Kn8yDIRjjFZlLCZw8SDh6QH+0CE0kupmk2AG+plMkqXmb5WlOG
WSbUAWv3koBojJcgKRs4YjGp465l1B2I3JLAApY/SES20cStS4yQ2EiCfTzMoFk=
=ETUq
-----END PGP SIGNATURE-----

[Thomas Leonard]

I've just added some initial QubesDB support (to get the IP address). It took me a while to get it working because initially I didn't spot that the C struct requires some padding before data_len. This worked:

  cstruct msg_header {
    uint8_t ty;
    uint8_t path[64];
    uint8_t padding[3];
    uint32_t data_len;
    (* rest of message is data *)
  } as little_endian

My first attempts at sending the initial MULTIREAD without this padding caused dom0 to get into a bad state (qubes-manager wouldn't run, etc) and I had to reboot the laptop a couple of times.
 

Currently they are in iptables syntax, so probably not the most
convenient one... This will be improved in Qubes 4.0 (or even 4.1).
But for now, it still should be possible to handle that format - there
are only few of iptables options combination, so you should be able to
parse that.

I was wondering about just hard-coding the rules into the unikernel, at first at least. I have a fixed set of AppVMs, and a firewall policy in OCaml is going to be more readable and more flexible than iptables or the GUI. On the other hand, it would be nice to make it a drop-in replacement for the existing firewall.

 

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Ouch, that's really bad. Do you have some logs? At least
/var/log/qubes/qubesdb.xxx.log?

Anyway saved here:
https://github.com/QubesOS/qubes-issues/issues/1470

> > Currently they are in iptables syntax, so probably not the most
> > convenient one... This will be improved in Qubes 4.0 (or even 4.1).
> > But for now, it still should be possible to handle that format - there
> > are only few of iptables options combination, so you should be able to
> > parse that.
> >
>
> I was wondering about just hard-coding the rules into the unikernel, at
> first at least. I have a fixed set of AppVMs, and a firewall policy in
> OCaml is going to be more readable and more flexible than iptables or the
> GUI. On the other hand, it would be nice to make it a drop-in replacement
> for the existing firewall.

Yup, as said before - in Qubes 4.0, we'll have some nice, implementation
agnostic firewall rules. Exactly for that reason.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJWW50yAAoJENuP0xzK19csHckIAIhH3zNudaDY8nO4BYqn7Q+5
4ONf+W2UdcMEX4bncDpKI4PW13Ey6VYcW/qoJF3u4ApYTYdoKPU0DyHzVEoCkzIp
Xl8V4yO7qqmmsyjGKjuUTS/oXqTuIbsi8BvNWyQlSiCbM9fr2LRnbAiNic9/F0fA
OKu2xEyygry9wNLPgRyItRP96Odr1DuZ2yrwdWLcllsOnY6z+/tmWOUMfLOef8P0
9VcVvgvi/+PmSqmiZhAximvnCmchCPFZfD/yNM1iSHTI+J59IChtjXQLmPlcQBuT
2UK4BNdTxF2crLNrmcCilsoWUrnT3dfqPi1w9/4t+MkQ2Cl39FdeGGJKWtiNX+s=
=V4xJ
-----END PGP SIGNATURE-----

[Manuel Amador (Rudd-O)]

+the author of the blog post.

In sum: the unikernel author can simply package an image that contains a
kernel, and then he can make it available in a signed Yum repo for
people to test.

Alternatively, an image that was a full Linux image but with everything
necessary to produce unikernels on demand -- a sort of factory of
unikernels -- would also be GREATLY NEEDED to speed up contributions in
this space.

Enjoy!

--
Rudd-O
http://rudd-o.com/

[Manuel Amador (Rudd-O)]

On 11/22/2015 10:53 PM, Thomas Leonard wrote:
> My qrexec module should be reusable. If other people want to try it,
> the API is at https://github.com/talex5/mirage-qubes/blob/master/qrexec.mli
> and the example code is here:
>
> https://github.com/talex5/mirage-qubes/blob/master/unikernel.ml

This is BEYOND AWESOME.

You're truly setting foot on the moon here. Thanks!

--
Rudd-O
http://rudd-o.com/

[Manuel Amador (Rudd-O)]

Quick request:

Can you document your learnings in some form of markdown in some repo so
that people can follow in your footsteps?

It's time to turn this thing into the future of computing, and the more
people can learn the ropes without having to do digging and fishing, the
better.

--
Rudd-O
http://rudd-o.com/

[Thomas Leonard]

On Monday, December 28, 2015 at 7:56:59 PM UTC, Manuel Amador (Rudd-O) wrote:

Quick request:

Can you document your learnings in some form of markdown in some repo so
that people can follow in your footsteps?

It's time to turn this thing into the future of computing, and the more
people can learn the ropes without having to do digging and fishing, the
better.

Good idea. I've written up my notes on how I replaced sys-firewall with a unikernel here:

  http://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-qubesos/

Hopefully that's a useful starting point for other people.

[Manuel Amador (Rudd-O)]

On 01/01/2016 07:20 PM, Thomas Leonard wrote:
>
> http://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-qubesos/

Excellento!

--
Rudd-O
http://rudd-o.com/

[Manuel Amador (Rudd-O)]

On 01/01/2016 07:20 PM, Thomas Leonard wrote:
>

> Good idea. I've written up my notes on how I replaced sys-firewall
> with a unikernel here:
>
>
> http://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-qubesos/
>
>
> Hopefully that's a useful starting point for other people.

Just read the whole thing -- I must confess to be super excited about
this. Honestly, if this thing had support for the configurable firewall
rules, I would be the first to advocate it be distributed as an image by
ITL themselves, and I would locally build one for myself.

Real cool work. You rock.

--
Rudd-O
http://rudd-o.com/

[7v5w7go9ub0o]

THANK YOU for your work, and this wonderful write up!!

- I (newbie) wonder if it'd make sense to construct a unikernel NetVM as
well - for all of the performance and security reasons justifying the
FirewallVM?

- Is there a mechanism which would facilitate using net/firewall
unikernel VMs as Qubes presently uses DispVMs - i.e. at session end or
restart, they are each discarded (or perhaps reinitialized to a "clean"
condition), rather than having the possibly compromised contents saved
and used again?

[William Waites]

On Fri, 1 Jan 2016 11:20:05 -0800 (PST), Thomas Leonard <tal...@gmail.com> said:

> Good idea. I've written up my notes on how I replaced
> sys-firewall with a unikernel here:

> http://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-qubesos/

> Hopefully that's a useful starting point for other people.

This is brilliant! I've just followed and apart from a very minor
documentation issue (need to install patch(1) otherwise opam is
unhappy) I've just done the same. It took about 20 minutes and I'm
watching streaming video through it as a test. So far so good.

This is very important both for keeping a lid on resource usage and
being able to reason about what different subsystems are doing. Very
cool.

-w

[Tim W]

I think most would be very keen on using them for net/fw and other proxy vms   I know I sure would like it.

[J. Eppler]

Hello Thomas Leonard,

Good idea. I've written up my notes on how I replaced sys-firewall with a unikernel here:

  http://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-qubesos/

Hopefully that's a useful starting point for other people.

Thanks for your work and the article :-)

Best regards
  J. Eppler

[Thomas Leonard]

On Friday, January 1, 2016 at 11:30:20 PM UTC, William Waites wrote:

On Fri, 1 Jan 2016 11:20:05 -0800 (PST), Thomas Leonard <tal...@gmail.com> said:

    > Good idea. I've written up my notes on how I replaced
    > sys-firewall with a unikernel here:
 
    > http://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-qubesos/

    > Hopefully that's a useful starting point for other people.

This is brilliant! I've just followed and apart from a very minor
documentation issue (need to install patch(1) otherwise opam is
unhappy) I've just done the same. It took about 20 minutes and I'm
watching streaming video through it as a test. So far so good.

Thanks - great to hear someone else got it working! I've fixed the instructions.
 

[Thomas Leonard]

I think someone mentioned that the format of the rules might change to something easier to parse than raw iptables rules (in Qubes 4?). That would help.

A few other things need finishing off before it's ready for general use (mostly, expiry of NAT entries, or it will eventually run out of free ports, hang, and need to be restarted). These are mostly Mirage issues, so I sent them here:

http://lists.xenproject.org/archives/html/mirageos-devel/2016-01/msg00000.html

Also, it would be nice if Qubes made it easier to install unikernels. These things would help:

1. Don't require kernels to have initramfs and modules.img files.
2. Don't pass Linux-specific kernel arguments.
3. Don't require a template HD image.
4. Provide a way to transfer files safely to dom0 (not `qvm-run | tar`).
5. Provide a way to download, check signature and install images easily (e.g. via 0install).

[Vít Šesták]

Looks great! However, there are two unlisted features I use in the Linux FW and I probably can't use in the Mirage one:
* explicitly allowed port forwarding to AppVM
* explicitly allowed inter-VM communication

None of these is configurable in GUI, but both of them are useful in some cases. OK, they are not needed for PoC, but it would be useful to have them before replacing the current sys-firewall.

Regards,
Vít Šesták 'v6ak'

[William Waites]

(trimming the Cc list)

On Sat, 2 Jan 2016 11:38:17 -0800 (PST), Vít Šesták said:

> Looks great! However, there are two unlisted features I use in
> the Linux FW and I probably can't use in the Mirage one:

> * explicitly allowed port forwarding to AppVM
> * explicitly allowed inter-VM communication

From what I can tell these "by hand" cases are handled easily
enough. Just modify the `rules.ml' file and rebuild the
image. Approximately as easy as editing a config file (apart from the
slight bump of getting the image to dom0, but I hear that is being
worked on).

Cheers,
-w

[Thomas Leonard]

For people planning to update regularly - which is to say, anyone using version 0.1 ;-) - I'd sugggest using these scripts:

  https://github.com/talex5/qubes-test-mirage

Then the process is:

1. vi rules.ml
2. make
3. test-mirage mir-qubes-firewall.xen mirage-firewall

(of course, you'll need to stop the clients first and restart them afterwards)

I've already pushed some code to drop the NAT table if it runs out of memory and/or ports, so you might want to try updating already ("git pull && make").

For allowing inter-VM communication, editing rules.ml should be very easy.

For port forwarding, you'll need some way to identify the target client. The rule will probably be something like:

  | { dst = `Firewall_uplink; proto = `TCP { dport = 80 } } -> `Nat_to (my_server, 8080)

But you'll need to specify somewhere what "my_server" is. As a hack, I think you could just do this at the top of the file:

let my_server = `Unknown_client (Ipaddr.of_string_exn "10.137.3.55")

(inserting your AppVM's IP address, of course)

All completely untested :-)

[Vít Šesták]

On Friday, January 1, 2016 at 10:28:23 PM UTC+1, 7v5w7go9ub0o wrote:

- I (newbie) wonder if it'd make sense to construct a unikernel NetVM as
well - for all of the performance and security reasons justifying the
FirewallVM?

This approach might be also good for some ProxyVMs (probably good for VPN VM, but controversial for Tor VM), but for NetVM, it would be much more work with more risks and less benefits:

First, the traditional sys-net does not seem to suffer from netfront bugs (there is no parent NetVM that can perform the attack, is there?), so the main security argument does not hold. Maybe startup pefrormance would be the main benefit.

But it would be probably much harder than firewall, for multiple reasons. There are few relatively minor reasons (sys-net serves as a ClockVM, UpdateVM and has some GUI for networking) and one major one: Hardware support. The FirewallVM communicates just with Xen and Qubes infrastructure. In contrast, traditional NetVM communicates with physical hardware, which might vary and it would be a challenge to ensure it works with wide range of hardware. I might be wrong, but I guess this unrealistic for Qubes as a general solution. Theoretically, it might be implemented for some specific hardware (e.g., Purism Libre) that is supported more than ordinary laptop, but I don't believe it is worth the work.

Moreover, PCI network card drivers for Linux are hopefully well-reviewed. In contrast, Unikernel might require writing a new drivers for a PCI network card. Like any new software, such driver can hardly have been reviewed at the time of release. And I am not sure if it can take advantage of OcaML memory safety, because PCI uses DMA.

And one more practical issue: It might be useful to have some software like Wireshark in sys-net. Which would be hardly possible with unikernel.

- Is there a mechanism which would facilitate using net/firewall
unikernel VMs as Qubes presently uses DispVMs - i.e. at session end or
restart, they are each discarded (or perhaps reinitialized to a "clean"
condition), rather than having the possibly compromised contents saved
and used again?

For sys-net, it seems to be impossible, as it has to store some data for known networks etc. It would be, however, possible to reduce the amount of stored data: https://github.com/QubesOS/qubes-issues/issues/1006

For firewall, it seems to be what Thomas Leonard has described in his blogpost http://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-qubesos/ .

For ProxyVMs (VPN, Tor), it might be possible to achieve this in multiple ways (some would require some changes in Qubes):
a) Having all the data in a specific TemplateVM with no /rw in fstab. (A hacky solution for today, not perfect.)
b) Using some unikernel
c) Using some other stateless VM or minimal-state VM
d) Using multi-instance DVM

With DVM, there are multiple problems today:
* Qubes currently allows only one DVM (though with multiple instances).
* Qubes currently uses cloning VM instance snapshots in DVM, which might have some bad impact on cryptography. Although issues with /dev/random are somehow resolved, using Tor started on boot still might be can of worms in terms of security. This seems to be changed in Qubes 4.0.
* Qubes currently does not support using DVMs as NetVMs.

Regards,
Vít Šesták 'v6ak'

[Vít Šesták]

For allowing inter-VM communication, editing rules.ml should be very easy.

For port forwarding, you'll need some way to identify the target client. The rule will probably be something like:

  | { dst = `Firewall_uplink; proto = `TCP { dport = 80 } } -> `Nat_to (my_server, 8080)

This looks cooler than I though, but it still requires VM reboot. Which is undesired, because Qubes has some issues with reattaching NetVMs, so it would probably also require reboot of relevant AppVMs.

Regards,
Vít Šesták 'v6ak'

[Thomas Leonard]

On Saturday, January 2, 2016 at 8:25:56 PM UTC, Vít Šesták wrote:

For allowing inter-VM communication, editing rules.ml should be very easy.

For port forwarding, you'll need some way to identify the target client. The rule will probably be something like:

  | { dst = `Firewall_uplink; proto = `TCP { dport = 80 } } -> `Nat_to (my_server, 8080)

I just tested this. Almost got it right: you need `NAT_to, not `Nat_to.
Then it works (tested with curl from NetVM), except that the log message it prints always misleadingly says it redirected to "NetVM" (now fixed).
 

This looks cooler than I though, but it still requires VM reboot. Which is undesired, because Qubes has some issues with reattaching NetVMs, so it would probably also require reboot of relevant AppVMs.

Yes, if you want to change it frequently, you'll want to go via QubesDB, XenStore or qrexec. This works for static servers, though.

[William Waites]

Incidentally with the unikernel we are *very* close to having
bit-for-bit repeatable builds. Comparing the image from two subsequent
builds they differ by exactly six bytes -- i'm not sure of their
meaning, but it is very close...

-w

[Thomas Leonard]

Probably this:

  http://caml.inria.fr/mantis/view.php?id=7037
 
Should be fixed in the next version of OCaml.

[Vít Šesták]

On Friday, January 1, 2016 at 8:20:06 PM UTC+1, Thomas Leonard wrote:

  http://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-qubesos/

Hopefully that's a useful starting point for other people.

Few comments on your article:

* Startup time includes creation of volatile.img, which is not needed for the unikernel. It takes about 0.5s on my laptop. If you want to cut the time even further, you might try to modify /usr/lib/qubes/prepare-volatile-img.sh to do nothing for the particular VM. Maybe the automatic volatile.img creation will be removed in Qubes 4.0 thanks to the refactoring.
* The sys-net isolation is true for IOMMU-enabled (Intel VT-d or AMD IOMMU) systems only. For non-IOMMU systems, the security benefit is not so significant, as sys-net can perform DMA attacks.
* I remember being able to run Linux-based sys-firewall with 200MiB of RAM (the actual usage, not just he minimum). So, the RAM saving is not as significant as you suggest, but it is still notable.

Regards,
Vít Šesták 'v6ak'

[Derek Fawcus]

On Sun, Jan 03, 2016 at 03:06:33AM -0800, V??t ??est??k wrote:
> * I remember being able to run Linux-based sys-firewall with 200MiB of RAM
> (the actual usage, not just he minimum). So, the RAM saving is not as
> significant as you suggest, but it is still notable.

My current FW is using 199M of memory.
It is based on the Fedora 21 minimal template, with very little added.

DF

[Tim W]

Until this unikernel got to this point that was my plan as well.   To use the min template for at least fw.    There may be a reason for it but to me I am not sure why sys-fw and sys-net are built off the full template vs the minimal.

[Thomas Leonard]

On Friday, January 1, 2016 at 9:28:23 PM UTC, 7v5w7go9ub0o wrote:

On 01/01/2016 02:20 PM, Thomas Leonard wrote:
> On Monday, December 28, 2015 at 7:56:59 PM UTC, Manuel Amador (Rudd-O)
> wrote:
>> Quick request:
>>
>> Can you document your learnings in some form of markdown in some repo so
>> that people can follow in your footsteps?
>>
>> It's time to turn this thing into the future of computing, and the more
>> people can learn the ropes without having to do digging and fishing, the
>> better.
>>
> Good idea. I've written up my notes on how I replaced sys-firewall with a
> unikernel here:
>
>    
> http://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-qubesos/
>
> Hopefully that's a useful starting point for other people.
>

THANK YOU for your work, and this wonderful write up!!

- I (newbie) wonder if it'd make sense to construct a unikernel NetVM as
well - for all of the performance and security reasons justifying the
FirewallVM?

NetVM needs to support lots of different network hardware and many different wifi authentication schemes, so Mirage wouldn't be a good fit for this (though other unikernels might). Also, the applet requires a GUI, D-BUS, etc. So I suspect this would be quite hard.
 

- Is there a mechanism which would facilitate using net/firewall
unikernel VMs as Qubes presently uses DispVMs - i.e. at session end or
restart, they are each discarded (or perhaps reinitialized to a "clean"
condition), rather than having the possibly compromised contents saved
and used again?

Not sure if this is quite what you mean, but mirage-firewall doesn't use a hard disk (and ideally Qubes wouldn't give it access to one), so it has no way to preserve data across a reboot and will always restart from the same clean state.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Sat, Jan 02, 2016 at 06:09:13AM -0800, Thomas Leonard wrote:
> On Friday, January 1, 2016 at 8:40:32 PM UTC, Manuel Amador (Rudd-O) wrote:
> >
> > On 01/01/2016 07:20 PM, Thomas Leonard wrote:
> > >
> > > Good idea. I've written up my notes on how I replaced sys-firewall
> > > with a unikernel here:
> > >
> > >
> > >
> > http://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-qubesos/
> > >
> > >
> > > Hopefully that's a useful starting point for other people.

That's GREAT! Thank you!

> >
> > Just read the whole thing -- I must confess to be super excited about
> > this. Honestly, if this thing had support for the configurable firewall
> > rules, I would be the first to advocate it be distributed as an image by
> > ITL themselves, and I would locally build one for myself.
> >
>
> I think someone mentioned that the format of the rules might change to
> something easier to parse than raw iptables rules (in Qubes 4?). That would
> help.

We had some discussion about this on 32c3, but definitely want to continue
here to take input from more people. IMHO it would be better to start a
new thread about that.

> A few other things need finishing off before it's ready for general use
> (mostly, expiry of NAT entries, or it will eventually run out of free
> ports, hang, and need to be restarted). These are mostly Mirage issues, so
> I sent them here:
>
> http://lists.xenproject.org/archives/html/mirageos-devel/2016-01/msg00000.html
>
> Also, it would be nice if Qubes made it easier to install unikernels. These
> things would help:
>
> 1. Don't require kernels to have initramfs and modules.img files.

modules.img requirement is already lifted in 3.1. initramfs is still
needed there, but we can change that in 4.0.

> 2. Don't pass Linux-specific kernel arguments.

Surely doable, in 4.x. But we need to think how to do that, without
breaking existing functionality.

> 3. Don't require a template HD image.

This one is tricky. But a Standalone VM with empty root.img (0 bytes)
isn't be a big problem, is it? VM can't extend the image size on its
own.

> 4. Provide a way to transfer files safely to dom0 (not `qvm-run | tar`).

Take a look here:
https://github.com/QubesOS/qubes-issues/issues/1324

Generally we don't want generic-purpose easy way to copy untrusted data
into dom0, to discourage users from doing that.

> 5. Provide a way to download, check signature and install images easily
> (e.g. via 0install).

qubes-dom0-update already exists, we use it for example to distribute
template or kernel images for example. Could be also used for unikernel
images distribution.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJWim/PAAoJENuP0xzK19csrmEH/AyK/GiXLVbNx2jU+cb2KBjd
uof4Q+Q/Gf9yOnkpHKatrfhsVl4lBU400cLpwBbsMuTOOS+kG20tJE8Ym5sJs28U
rnWOb+YxtMyUYBRRQzq6qSrVWiCj2lNgSGU0ZBIlc4Kd1cvOhGPCrNGRIOT8wi82
c2WbJuz31bQAzeGxwWR1DzMGyKKAiAS48uvDlv4a8T1BtAnsuEDI9dnjAXHrKmS8
+orRDxNSjsbCWRgcMZ2TypOAPjSQDz1Uk9leoy/sUVUCnGSI4xEHstxcKH5Ta4MZ
yPvB3Lfd3mAni/U+0L8N1TaX2zNWoLU+uk/Nnc1WbQyNco6dVvKMesXRtEKgXxg=
=Wo3y
-----END PGP SIGNATURE-----

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Sun, Jan 03, 2016 at 03:06:33AM -0800, Vít Šesták wrote:
> On Friday, January 1, 2016 at 8:20:06 PM UTC+1, Thomas Leonard wrote:
> >
> >
> >
> > http://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-qubesos/
> >
> > Hopefully that's a useful starting point for other people.
> >
>
> Few comments on your article:
>
> * Startup time includes creation of volatile.img, which is not needed for
> the unikernel. It takes about 0.5s on my laptop. If you want to cut the
> time even further, you might try to modify
> /usr/lib/qubes/prepare-volatile-img.sh to do nothing for the particular VM.
> Maybe the automatic volatile.img creation will be removed in Qubes 4.0
> thanks to the refactoring.

It is already moved into initramfs in Qubes 3.1. So in case of
unikernel, it isn't executed.

BTW This is the reason why kernel package from 3.0 isn't compatible with
3.1...

(...)

> * I remember being able to run Linux-based sys-firewall with 200MiB of RAM
> (the actual usage, not just he minimum). So, the RAM saving is not as
> significant as you suggest, but it is still notable.

I think I've seen someone on 32c3 with sys-firewall trimmed to 150MB.
Anyway still five times bigger than unikernel-based.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJWim/oAAoJENuP0xzK19csJ7AH/3dBsk1aghcet61cCF0+bdnG
Gf3BLm3dplSI9a70F4/ph7BaiX9xNW5Laz7QGY9i1rd4uwB3YATFXi9s6UWP+ze+
xHrOVt3FirNS0Mhn86eYK5/OdM8f7f1GlWJg1wf5KTcj7MkP7jk1Cbh1UThy5ZuK
05AIiSVQduaOgU773bxSn5gX2e1TaOrVSamS1ppQPLoQEkYYvEMCaIsMdgHaEVyN
Tdwc9dfmVNfiNcLbAkngb6lEXY2KhZryxmG3qvIUcH0zSmZwJ2Y3woer283ccyM/
SZvYZEsebDXfvFXT8SqdmIockz7mdXQmbr65AVTHwA7F9xlK5qdJIUYJUV0VAeg=
=HC4/
-----END PGP SIGNATURE-----

[Derek Fawcus]

On Sun, Jan 03, 2016 at 07:09:16pm -0800, Tim W wrote:
> On Sunday, January 3, 2016 at 7:28:47 PM UTC-5, Derek Fawcus wrote:
> > My current FW is using 199M of memory.
> > It is based on the Fedora 21 minimal template, with very little added.
>

> Until this unikernel got to this point that was my plan as well. To use
> the min template for at least fw. There may be a reason for it but to me
> I am not sure why sys-fw and sys-net are built off the full template vs the
> minimal.

As to the net-vm, I've not yet got that working on the minimal template.

I _think_ it may be missing firmware, but haven't yet taken the time to
figure it out because running of my 'general' Fedora 21 template, it is
only consuming 200M.

What I really need to achieve is constraining the memory used by my untrusted VM,
mainly used for firefox. As it takes 2G on a 4G machine, and prevents me
from starting any new VMs. I've not yet tried limiting its memory, but guess
I should treak the VM settings, and see how it handles swapping.

DF

[Vít Šesták]

I use a template derived from Fedora minimal for sys-net and sys-firewall, just with few extra packages installed (including haveged). I initially had some troubles with the firmware, but there was some complaint in some log (probably dmesg) about missing firmware with the identification. So, I installed a seemingly unrelated firmware package (the name of WiFi card suggested that it needed some different firmware…) and it started working then. The sys-net VM consumes constantly 250MiB, sys-firewall consumes 200MiB if there is low RAM. Both of them probably could go even lower, but I have a plenty of RAM, so I prefer consuming more RAM to potential issues like OOMs and swapping.

I am, however, not sure if minimal template consumes less memory than a standard one. It naturally consumes less disk space and requires updates less frequently, but having installed some software that never runs does not increase RAM usage, does it?

But this debate is not directly connected to unikernels. If one wishes to continue it, start a new thread, please.

Regards,
Vít Šesták 'v6ak'

[7v5w7go9ub0o]

On 01/04/2016 05:49 AM, Thomas Leonard wrote:
> On Friday, January 1, 2016 at 9:28:23 PM UTC, 7v5w7go9ub0o wrote:
>>

[snip]

>> THANK YOU for your work, and this wonderful write up!!
>>
>> - I (newbie) wonder if it'd make sense to construct a unikernel NetVM as
>> well - for all of the performance and security reasons justifying the
>> FirewallVM?
>>
> NetVM needs to support lots of different network hardware and many
> different wifi authentication schemes, so Mirage wouldn't be a good fit for
> this (though other unikernels might). Also, the applet requires a GUI,
> D-BUS, etc. So I suspect this would be quite hard.
>
>
>> - Is there a mechanism which would facilitate using net/firewall
>> unikernel VMs as Qubes presently uses DispVMs - i.e. at session end or
>> restart, they are each discarded (or perhaps reinitialized to a "clean"
>> condition), rather than having the possibly compromised contents saved
>> and used again?
>>
> Not sure if this is quite what you mean, but mirage-firewall doesn't use a
> hard disk (and ideally Qubes wouldn't give it access to one), so it has no
> way to preserve data across a reboot and will always restart from the same
> clean state.
>

Ah! The "always start with a clean state" firewall is exactly what I
want. (..and a little script at shutdown that clears out/restores any
changes/additions made to user files should handle the netVM.)

Thank you again!

[Manuel Amador (Rudd-O)]

On 01/04/2016 03:09 AM, Tim W wrote:
> Until this unikernel got to this point that was my plan as well. To
> use the min template for at least fw. There may be a reason for it
> but to me I am not sure why sys-fw and sys-net are built off the full
> template vs the minimal.

Needs a buncha packages. Lemme give you the list:

- tar
- qubes-tor-repo
- qubes-tor
- dconf
- NetworkManager
- NetworkManager-wifi
- network-manager-applet
- linux-firmware
- dbus-x11
- gnome-keyring
- wireless-tools
- wpa_supplicant
- iwl7260-firmware
- tinyproxy
- which
- pciutils
- usbutils


--
Rudd-O
http://rudd-o.com/

[Tim W]

Is that a complete list?   I would like to try it out

[Thomas Leonard]

On Monday, January 4, 2016 at 1:13:20 PM UTC, Marek Marczykowski-Górecki wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Sun, Jan 03, 2016 at 03:06:33AM -0800, Vít Šesták wrote:
> On Friday, January 1, 2016 at 8:20:06 PM UTC+1, Thomas Leonard wrote:
> >
> >
> >  
> > http://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-qubesos/
> >
> > Hopefully that's a useful starting point for other people.
> >
>
> Few comments on your article:
>
> * Startup time includes creation of volatile.img, which is not needed for
> the unikernel. It takes about 0.5s on my laptop. If you want to cut the
> time even further, you might try to modify
> /usr/lib/qubes/prepare-volatile-img.sh to do nothing for the particular VM.
> Maybe the automatic volatile.img creation will be removed in Qubes 4.0
> thanks to the refactoring.

It is already moved into initramfs in Qubes 3.1. So in case of
unikernel, it isn't executed.

BTW This is the reason why kernel package from 3.0 isn't compatible with
3.1...

(...)

> * I remember being able to run Linux-based sys-firewall with 200MiB of RAM
> (the actual usage, not just he minimum). So, the RAM saving is not as
> significant as you suggest, but it is still notable.

I think I've seen someone on 32c3 with sys-firewall trimmed to 150MB.
Anyway still five times bigger than unikernel-based.

I'm not actually sure how much mirage-firewall needs. I just picked 30 because it was 10x smaller than sys-firewall. I've added meminfo reporting in Git now, and after a couple of days of use I see:

2016-01-04 07:26.03: INF [memory_pressure] Writing meminfo: free 10080 / 28336 kB (35.57 %)

So it could probably run with about 23 MB without changes if needed (it likes to keep a few MB free). Haven't tried it though. 30 MB isn't bothering me...
 

[Cube]

Great work and I look forward to it in the main distribution.

It seems that the idea mainly applies to NetVM's and ProxyVM's, primarily the firewall which is excellent. For ProxyVM's and VPN VM's I'd caution that for VPN's at least probably need more facilities. I'm using /rw/config/rc.local to launch the VPN and am using a custom build of OpenVPN. It's also convenient to pop into a shell/firefox in that VM, especially when I was setting it up and debugging.

Just a few thoughts, if a VPN unikernel can be done that is flexible enough that would be great.

[wahyzcrak]

I read your article this weekend and was really excited about the idea of being able to output traffic between vms into pcap. I had considered trying to figure out a way to use wireshark in the network vm but decided that it was probably a bad idea. I am planning on trying out your write-up on the mirage unikernel firewall once I get a few other things going.

Very nice write up though, thank you for sharing that.

[Manuel Amador (Rudd-O)]

On 01/05/2016 08:07 AM, Tim W wrote:
> Is that a complete list? I would like to try it out

I believe it is.

--
Rudd-O
http://rudd-o.com/

[Thomas Leonard]

On Saturday, January 2, 2016 at 8:16:50 PM UTC, Thomas Leonard wrote:

On Saturday, January 2, 2016 at 8:00:14 PM UTC, William Waites wrote:

(trimming the Cc list)

On Sat, 2 Jan 2016 11:38:17 -0800 (PST), Vít Šesták said:

    > Looks great! However, there are two unlisted features I use in
    > the Linux FW and I probably can't use in the Mirage one:

    > * explicitly allowed port forwarding to AppVM
    > * explicitly allowed inter-VM communication

From what I can tell these "by hand" cases are handled easily
enough. Just modify the `rules.ml' file and rebuild the
image. Approximately as easy as editing a config file (apart from the
slight bump of getting the image to dom0, but I hear that is being
worked on).

For people planning to update regularly - which is to say, anyone using version 0.1 ;-) - I'd sugggest using these scripts:

  https://github.com/talex5/qubes-test-mirage

Then the process is:

1. vi rules.ml
2. make
3. test-mirage mir-qubes-firewall.xen mirage-firewall

(of course, you'll need to stop the clients first and restart them afterwards)

I've already pushed some code to drop the NAT table if it runs out of memory and/or ports, so you might want to try updating already ("git pull && make").

I'm currently upstreaming some of the changes I made to various libraries, with some API changes, so if you 'git pull' and find it doesn't build any more:

1. Try `opam update; opam upgrade`
2. Check the pins reported by `opam pin` match the ones in the current README.md
3. Report a bug if it still doesn't build for you: https://github.com/talex5/qubes-mirage-firewall/issues

Also, please report any other bugs you find. I'm still tracking down a mysterious "EACCES" from XenStore and an occasional "out of memory". I'm currently running in a 20 MB VM, and that seems to work fine for many hours, so no reason why it shouldn't work indefinitely.

[Cyril LEVIS]

Hi, I'm running Qubes 3.1rc2

Is it only me or since few update we can't start mirage-fw?
qvm-start seems to wait qrexec-daemon...

"Waiting for VM's qrexec agent.failed"
ERROR: Cannot excecute qrexec-daemon

I have try to set qrexec_timeout to 0 but this change nothing.

Do you have idea?

Thanks


Le vendredi 6 novembre 2015 15:26:57 UTC+1, Andrew a écrit :

The idea of unikernels in Qubes is not entirely new; it's come up on the
lists a few times.  This doesn't seem to have been posted to the list
yet, though: https://northox.github.io/qubes-rumprun/

The above blog post makes some arguments for integrating some unikernels
into Qubes for various things: TCP/IP stack vulnerability mitigation,
in-line filters on communication channels between VMs (I always imagined
using a MirageOS TLS wrap/unrwap unikernel to avoid OpenSSL exploits),
secure file conversion, and generally promoting increased disaggregation
and finer-grained isolation.

This post asks for feedback and poses a few questions to be answered.

> Now, I'm looking for constructive feedback from Qubes' community and
> will try to answer a few questions: What exactly would need to be
> modified on Qubes' side to be part of the default installation?
> What's the effort? What's the best course of action? Is it viable in
> practice? Does it make sense?

I don't have the answers to these questions, but maybe others can chime
in to help answer them.

Andrew

[Thomas Leonard]

On Saturday, January 16, 2016 at 6:16:25 PM UTC, Cyril LEVIS wrote:

Hi, I'm running Qubes 3.1rc2

Is it only me or since few update we can't start mirage-fw?
qvm-start seems to wait qrexec-daemon...

"Waiting for VM's qrexec agent.failed"
ERROR: Cannot excecute qrexec-daemon

I have try to set qrexec_timeout to 0 but this change nothing.

It should connect instantly. What does the log show (right-click on mirage-firewall in Qubes VM Manager and choose Logs -> guest-mirage-firewall.log).
 

[Cyril LEVIS]

Here:

.[32;1mMirageOS booting....[0m
Initialising timer interface
Initialising console ... done.
getenv(OCAMLRUNPARAM) -> null
getenv(CAMLRUNPARAM) -> null
getenv(PATH) -> null
Unsupported function lseek called in Mini-OS kernel
Unsupported function lseek called in Mini-OS kernel
Unsupported function lseek called in Mini-OS kernel
getenv(OCAMLRUNPARAM) -> null
getenv(CAMLRUNPARAM) -> null
getenv(TMPDIR) -> null
getenv(TEMP) -> null
Netif: add resume hook
gnttab_stubs.c: initialised mini-os gntmap
Fatal error: exception Not_found
Mirage exiting with status 2
Do_exit called!

[Thomas Leonard]

On Saturday, January 16, 2016 at 8:04:37 PM UTC, Cyril LEVIS wrote:

Here:

.[32;1mMirageOS booting....[0m
Initialising timer interface
Initialising console ... done.
getenv(OCAMLRUNPARAM) -> null
getenv(CAMLRUNPARAM) -> null
getenv(PATH) -> null
Unsupported function lseek called in Mini-OS kernel
Unsupported function lseek called in Mini-OS kernel
Unsupported function lseek called in Mini-OS kernel
getenv(OCAMLRUNPARAM) -> null
getenv(CAMLRUNPARAM) -> null
getenv(TMPDIR) -> null
getenv(TEMP) -> null
Netif: add resume hook
gnttab_stubs.c: initialised mini-os gntmap
Fatal error: exception Not_found
Mirage exiting with status 2
Do_exit called!

OK, this is happening right at the start. I bet it's the code at the top of unikernel.ml that turns on debug logging for XenStore. I was trying to track down an EACCES error (which turned out to be due to a local patch I'd made to XenStore). However, the public version of XenStore doesn't have any logging, so it will fail to find the logger when it tries to set the level.

Could you "git pull" and see if that fixes it?

Thanks,
 

[Cyril LEVIS]

Yes it works! Thanks a lot Thomas!

[dangm...@gmail.com]

On Friday, November 6, 2015 at 6:26:57 AM UTC-8, Andrew wrote:
> The idea of unikernels in Qubes is not entirely new; it's come up on the
> lists a few times. This doesn't seem to have been posted to the list
> yet, though: https://northox.github.io/qubes-rumprun/
>
> The above blog post makes some arguments for integrating some unikernels
> into Qubes for various things: TCP/IP stack vulnerability mitigation,
> in-line filters on communication channels between VMs (I always imagined
> using a MirageOS TLS wrap/unrwap unikernel to avoid OpenSSL exploits),
> secure file conversion, and generally promoting increased disaggregation
> and finer-grained isolation.
>
> This post asks for feedback and poses a few questions to be answered.
>
> > Now, I'm looking for constructive feedback from Qubes' community and
> > will try to answer a few questions: What exactly would need to be
> > modified on Qubes' side to be part of the default installation?
> > What's the effort? What's the best course of action? Is it viable in
> > practice? Does it make sense?
>
> I don't have the answers to these questions, but maybe others can chime
> in to help answer them.
>
> Andrew

Has anyone got a Mirage-VPN ProxyVM running?

I use very many different VPN connections simultaneously. I am constantly running out of RAM to open any more VMs.

I am already maxed out at 16GB, so I need to find ways to cut back on RAM. (And disk space).

Is it possible to run OpenVPN? (Sorry for necro)