Configure appvms to write-out all COW images to /dev/shm

[Jazz Kittens]

Ideally I'd like to have:

1) volatile.img write out to /dev/shm
2) private.img write out to /dev/shm

In the case of 1 the linked discussion below states:

> /var/lib/qubes/appvms/fedora-18-x64-dvm/dvm.conf):

> 'script:file:/var/lib/qubes/appvms/fedora-18-x64-dvm/volatile.img,xvdc,w',

https://groups.google.com/forum/#!topic/qubes-devel/QwL5PjqPs-4

I have attempted this on an appvm. However, something reverts the config file back to its default state after it has been run. Any ideas?

In the case of '2', ideally I'd like the same kind of setup as the rootfs image: a read only image that writes out COW data to another image. This is so I can boot the appvm with said image in rw mode, make configuration changes to /home, and reboot it into ro mode - where any further changes are written to a cow image in /dev/shm instead.

I'm a little unsure about where /home changes are made in the template, or if the configuration i've described is possible.

To make life easier for answers:

A) What is reverting changes in the appvm.conf file, and can this behavior be changed?
B) Can appvm specific private.img files be configured to work in a similar fashion as volatile.img files i.e. RO and COW, and is it possible that they can be changed/configured on the fly?
C) Where are changes to /home in the template vms written to?

Crazy request, but it's something I'm willing to work upon.

[Jean-Philippe Ouellet]

The appvm.conf file is not actually what's passed to xen, it's just
there for convenience to view for debugging. A copy of the same
contents is piped through directly.

If you wish to start an AppVM with custom config, use qvm-start
--custom-config=/tmp/some-appvm.conf

As for /dev/shm, you probably don't want that as it'll put large
memory pressure on dom0.

If the reason you are trying to put things there is actually to resist
local forensics (by only storing vm contents in memory), then a better
solution would be to encrypt the copy-on-write deltas with an
ephemeral single-boot key which is forgotten on shutdown.

This has been on my to-do list for some time, but still haven't gotten
around to it.

It should be fairly straightforward to achieve with another dm-crypt
loop dev, but I have not actually done it myself yet.

See also https://github.com/QubesOS/qubes-issues/issues/904

[Jean-Philippe Ouellet]

On Sun, Mar 12, 2017 at 11:01 PM, Jean-Philippe Ouellet <j...@vt.edu> wrote:
> If the reason you are trying to put things there is actually to resist
> local forensics (by only storing vm contents in memory), then a better
> solution would be to encrypt the copy-on-write deltas with an
> ephemeral single-boot key which is forgotten on shutdown.

Of course I mean a different key per boot of each AppVM, not of the host.

[Jean-Philippe Ouellet]

There is also a question of where that dm-crypt should live.

Main argument in favor of AppVM instead of dom0:
- Makes it easy to ensure the key is forgotten on shutdown, as it is
only lives in domU memory, whereas in dom0 the cryptsetup mem could
theoretically get swapped

Main argument in favor of dom0 over AppVM:
- If the AppVM is compromised and wishes to persist some incriminating
data on disk (or wants to communicate with malicious disk firmware via
contents written or something) then it could bypass a crypto layer
implemented in the domU kernel.

Third option: (my personal favorite)
- Do the crypto in a separate VM that just exists for interposing blk devs.
- You get dom0-like anti-leak properties, & AppVM-like
anti-key-persistence properties
- Downside is increased runtime memory overhead and increased
per-write latency (and possibly throughput) performance penalty
- If this were done in a minimal stubdomain (like based on mirage
unikernel), the memory overhead and likely also performance penalty
could be largely minimized
- If we perform authentication instead of only encryption, then we
also have one of the building blocks already in place for untrusted
storage domain (as described in qubes arch spec [1])

If you happen to be a student looking for something to do this summer,
I would highly encourage you to consider applying for Google Summer of
Code [2] to work on this ;)

It would be a very welcome contribution, and if you do want to
participate in GSoC I would gladly volunteer to be the project mentor
for this.

[1]: https://www.qubes-os.org/attachment/wiki/QubesArchitecture/arch-spec-0.3.pdf
[2]: https://www.qubes-os.org/gsoc/

[Jean-Philippe Ouellet]

Not entirely relevant, but see also:
https://github.com/rustybird/qubes-split-dm-crypt