Attaching files with qvm-block: is there a security problem with this workaround?
131 views
Skip to first unread message
heombeeh
Dec 24, 2019, 4:48:05 AM12/24/19
to qubes...@googlegroups.com
Hi all,
In the "Attaching a File" [1] section of the docs we are directed to use losetup to create a loop device for a file we want to attach to a VM. Before release 4.0 I would use the `--attach-file` flag to accomplish this, but this feature got removed.
My question concerns the relation between the security of the old method vs the new method.
Suppose that I have a ext4 filesystem in a file called "myfs.img" on dom0. I was under the impression that if I used qvm-block attach with `--attach-file` to connect this filesystem to an appvm, then dom0 would *not* get exposed to any internal aspects of "myfs.img". By this I mean dom0 would *not* parse/process any part of the filesystem header, so that it would not be vulnerable to malicious modifications of the data of "myfs.img" by the appvm.
Now if my understanding of the security of the old method is correct, then it seems to me there is a problem with the new method. As per the directions on the docs page, when I create the loop device for my "myfs.img" file, I can see evidence that the OS has parsed some portion of the ext4 filesystem metadata within "myfs.img". For example, if I run `lsblk -f`, the output shows that the loop device points to an ext4 filesystem, and it also prints the filesystem UUID. This is in contrast to the block devices of the qubes pool that contains VM blocks, where `lsblk -f` does *not* detect their filesystem types/UUIDs.
If I am correct that the new loop device method causes dom0 to become exposed, what is the correct way to attach a file in dom0 to a VM, so that dom0 is not exposed to an attack?
Also, what is the method that qubes uses to protect dom0 from the filesystems contained in VM blocks in the qubes pool? Is it the udev rules that are contained in `/etc/udev/rules.d`?
Note: I tried adding a udev rule copied from the default rules to prevent the OS from processing the filesystem data of the loop device. This worked, however it also somehow prevented qvm-block from seeing the loop device, so I was unable to attach it to my VM.
Thanks in advance to any one who can help answer my questions.
[1] https://www.qubes-os.org/doc/block-devices/
In the "Attaching a File" [1] section of the docs we are directed to use losetup to create a loop device for a file we want to attach to a VM. Before release 4.0 I would use the `--attach-file` flag to accomplish this, but this feature got removed.
My question concerns the relation between the security of the old method vs the new method.
Suppose that I have a ext4 filesystem in a file called "myfs.img" on dom0. I was under the impression that if I used qvm-block attach with `--attach-file` to connect this filesystem to an appvm, then dom0 would *not* get exposed to any internal aspects of "myfs.img". By this I mean dom0 would *not* parse/process any part of the filesystem header, so that it would not be vulnerable to malicious modifications of the data of "myfs.img" by the appvm.
Now if my understanding of the security of the old method is correct, then it seems to me there is a problem with the new method. As per the directions on the docs page, when I create the loop device for my "myfs.img" file, I can see evidence that the OS has parsed some portion of the ext4 filesystem metadata within "myfs.img". For example, if I run `lsblk -f`, the output shows that the loop device points to an ext4 filesystem, and it also prints the filesystem UUID. This is in contrast to the block devices of the qubes pool that contains VM blocks, where `lsblk -f` does *not* detect their filesystem types/UUIDs.
If I am correct that the new loop device method causes dom0 to become exposed, what is the correct way to attach a file in dom0 to a VM, so that dom0 is not exposed to an attack?
Also, what is the method that qubes uses to protect dom0 from the filesystems contained in VM blocks in the qubes pool? Is it the udev rules that are contained in `/etc/udev/rules.d`?
Note: I tried adding a udev rule copied from the default rules to prevent the OS from processing the filesystem data of the loop device. This worked, however it also somehow prevented qvm-block from seeing the loop device, so I was unable to attach it to my VM.
Thanks in advance to any one who can help answer my questions.
[1] https://www.qubes-os.org/doc/block-devices/
Marek Marczykowski-Górecki
Dec 24, 2019, 6:05:27 AM12/24/19
to heombeeh, qubes...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
On Tue, Dec 24, 2019 at 09:47:54AM +0000, 'heombeeh' via qubes-devel wrote:
> Hi all,
> In the "Attaching a File" [1] section of the docs we are directed to use losetup to create a loop device for a file we want to attach to a VM. Before release 4.0 I would use the `--attach-file` flag to accomplish this, but this feature got removed.
>
> My question concerns the relation between the security of the old method vs the new method.
In practice this documentation isn't any different. --attach-file did
exactly that.
> Suppose that I have a ext4 filesystem in a file called "myfs.img" on dom0. I was under the impression that if I used qvm-block attach with `--attach-file` to connect this filesystem to an appvm, then dom0 would *not* get exposed to any internal aspects of "myfs.img". By this I mean dom0 would *not* parse/process any part of the filesystem header, so that it would not be vulnerable to malicious modifications of the data of "myfs.img" by the appvm.
I'd say it would be better to isolate as mach things as possible in VMs.
Storing myfs.img too. Handling potentially malicious data is tricky and
should be done carefully.
> Now if my understanding of the security of the old method is correct, then it seems to me there is a problem with the new method. As per the directions on the docs page, when I create the loop device for my "myfs.img" file, I can see evidence that the OS has parsed some portion of the ext4 filesystem metadata within "myfs.img". For example, if I run `lsblk -f`, the output shows that the loop device points to an ext4 filesystem, and it also prints the filesystem UUID. This is in contrast to the block devices of the qubes pool that contains VM blocks, where `lsblk -f` does *not* detect their filesystem types/UUIDs.
For this specific reason, we exclude loop devices based on VM images
(based on name/location) from udev parsing.
> If I am correct that the new loop device method causes dom0 to become exposed, what is the correct way to attach a file in dom0 to a VM, so that dom0 is not exposed to an attack?
Yes and no. It was always recommended to not store user (especially
untrusted) data in dom0 manually. Documentation specifically handle the
case of "source VM", not "dom0".
> Also, what is the method that qubes uses to protect dom0 from the filesystems contained in VM blocks in the qubes pool? Is it the udev rules that are contained in `/etc/udev/rules.d`?
Yes, those rules.
> Note: I tried adding a udev rule copied from the default rules to prevent the OS from processing the filesystem data of the loop device. This worked, however it also somehow prevented qvm-block from seeing the loop device, so I was unable to attach it to my VM.
Yes, devices excluded from udev also are excluded from qvm-block. But we
have documented method to force showing them:
https://www.qubes-os.org/doc/mount-lvm-image/
It works also for loop devices - simply use "loopX" instead of "dm-X"
> Thanks in advance to any one who can help answer my questions.
>
> [1] https://www.qubes-os.org/doc/block-devices/
>
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAl4B8PAACgkQ24/THMrX
1yyb2wf/R7sL4i4TzFCsOUOJq1L2f3dF4uLFzKLK3fDbDWlpCs5gfi3yJ9vFDLq0
WjSV83EsythIrpU9GoaC8CqsC8CpMvY3k/cawY4jR7aRz+W4mICC+Afcx0vJnu+q
OZlCb7YBZlJnsXsS6dleGiDcpcq8MVo0EUgnrWcIqkUA5iGjhT0lFHtRveKNmkx/
3YrFUhA4NZzKOJKVXqaQijB/9dHCZpC+9yo5oD0EGnNwVaho8GnQaFBsfyJbsV08
uewVRbYWLVuq8va8ZngKEzU8pm3EyKovxH/WODNVDptY0OSo6UoZUNG81/K1iNeo
Lfm+2m21CWuXz5d79f2akvCMU3AYSg==
=v63Z
-----END PGP SIGNATURE-----
Hash: SHA256
On Tue, Dec 24, 2019 at 09:47:54AM +0000, 'heombeeh' via qubes-devel wrote:
> Hi all,
> In the "Attaching a File" [1] section of the docs we are directed to use losetup to create a loop device for a file we want to attach to a VM. Before release 4.0 I would use the `--attach-file` flag to accomplish this, but this feature got removed.
>
> My question concerns the relation between the security of the old method vs the new method.
exactly that.
> Suppose that I have a ext4 filesystem in a file called "myfs.img" on dom0. I was under the impression that if I used qvm-block attach with `--attach-file` to connect this filesystem to an appvm, then dom0 would *not* get exposed to any internal aspects of "myfs.img". By this I mean dom0 would *not* parse/process any part of the filesystem header, so that it would not be vulnerable to malicious modifications of the data of "myfs.img" by the appvm.
Storing myfs.img too. Handling potentially malicious data is tricky and
should be done carefully.
> Now if my understanding of the security of the old method is correct, then it seems to me there is a problem with the new method. As per the directions on the docs page, when I create the loop device for my "myfs.img" file, I can see evidence that the OS has parsed some portion of the ext4 filesystem metadata within "myfs.img". For example, if I run `lsblk -f`, the output shows that the loop device points to an ext4 filesystem, and it also prints the filesystem UUID. This is in contrast to the block devices of the qubes pool that contains VM blocks, where `lsblk -f` does *not* detect their filesystem types/UUIDs.
(based on name/location) from udev parsing.
> If I am correct that the new loop device method causes dom0 to become exposed, what is the correct way to attach a file in dom0 to a VM, so that dom0 is not exposed to an attack?
untrusted) data in dom0 manually. Documentation specifically handle the
case of "source VM", not "dom0".
> Also, what is the method that qubes uses to protect dom0 from the filesystems contained in VM blocks in the qubes pool? Is it the udev rules that are contained in `/etc/udev/rules.d`?
> Note: I tried adding a udev rule copied from the default rules to prevent the OS from processing the filesystem data of the loop device. This worked, however it also somehow prevented qvm-block from seeing the loop device, so I was unable to attach it to my VM.
have documented method to force showing them:
https://www.qubes-os.org/doc/mount-lvm-image/
It works also for loop devices - simply use "loopX" instead of "dm-X"
> Thanks in advance to any one who can help answer my questions.
>
> [1] https://www.qubes-os.org/doc/block-devices/
>
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAl4B8PAACgkQ24/THMrX
1yyb2wf/R7sL4i4TzFCsOUOJq1L2f3dF4uLFzKLK3fDbDWlpCs5gfi3yJ9vFDLq0
WjSV83EsythIrpU9GoaC8CqsC8CpMvY3k/cawY4jR7aRz+W4mICC+Afcx0vJnu+q
OZlCb7YBZlJnsXsS6dleGiDcpcq8MVo0EUgnrWcIqkUA5iGjhT0lFHtRveKNmkx/
3YrFUhA4NZzKOJKVXqaQijB/9dHCZpC+9yo5oD0EGnNwVaho8GnQaFBsfyJbsV08
uewVRbYWLVuq8va8ZngKEzU8pm3EyKovxH/WODNVDptY0OSo6UoZUNG81/K1iNeo
Lfm+2m21CWuXz5d79f2akvCMU3AYSg==
=v63Z
-----END PGP SIGNATURE-----
heombeeh
Dec 24, 2019, 10:00:43 PM12/24/19
to Marek Marczykowski-Górecki, qubes...@googlegroups.com
> > If I am correct that the new loop device method causes dom0 to become exposed, what is the correct way to attach a file in dom0 to a VM, so that dom0 is not exposed to an attack?
>
> Yes and no. It was always recommended to not store user (especially
> untrusted) data in dom0 manually. Documentation specifically handle the
> case of "source VM", not "dom0".
>
> > Also, what is the method that qubes uses to protect dom0 from the filesystems contained in VM blocks in the qubes pool? Is it the udev rules that are contained in `/etc/udev/rules.d`?
>
> Yes, those rules.
>
> > Note: I tried adding a udev rule copied from the default rules to prevent the OS from processing the filesystem data of the loop device. This worked, however it also somehow prevented qvm-block from seeing the loop device, so I was unable to attach it to my VM.
>
> Yes, devices excluded from udev also are excluded from qvm-block. But we
> have documented method to force showing them:
> https://www.qubes-os.org/doc/mount-lvm-image/
>
> It works also for loop devices - simply use "loopX" instead of "dm-X"
Thanks a lot for these clarifications.
>
> Yes and no. It was always recommended to not store user (especially
> untrusted) data in dom0 manually. Documentation specifically handle the
> case of "source VM", not "dom0".
>
> > Also, what is the method that qubes uses to protect dom0 from the filesystems contained in VM blocks in the qubes pool? Is it the udev rules that are contained in `/etc/udev/rules.d`?
>
> Yes, those rules.
>
> > Note: I tried adding a udev rule copied from the default rules to prevent the OS from processing the filesystem data of the loop device. This worked, however it also somehow prevented qvm-block from seeing the loop device, so I was unable to attach it to my VM.
>
> Yes, devices excluded from udev also are excluded from qvm-block. But we
> have documented method to force showing them:
> https://www.qubes-os.org/doc/mount-lvm-image/
>
> It works also for loop devices - simply use "loopX" instead of "dm-X"
Based on your recommendation against handling filesystem images within dom0, I'm thinking that I can instead create a "light storage" AppVM. This VM can simply have a directory of sparse files together with associated loop devices, and I can then attach them to other AppVMs normally using qvm-block.
However, just as in dom0, it will be good to protect this "light storage" VM from those client VMs that utilize its filesystem blocks. Can I protect this VM similarly to how dom0 is protected by using similar udev rules?
For example, suppose I keep my sparse files in `/home/user/myimages/{name}/disk.img`, and then add the following udev rule:
$ cat /etc/udev/rules.d/00-qubes-ignore-devices.rules
ACTION!="remove", SUBSYSTEM=="block", KERNEL=="loop*", ATTR{loop/backing_file}=="*/myimages/*/*.img*", ENV{DM_UDEV_DISABLE_DISK_RULES_FLAG}="1", ENV{UDEV_DISABLE_PERSISTENT_STORAGE_RULES_FLAG}="1"
With this active, it seems to work:
$ sudo losetup --find --show ./myimages/blah/disk.img
/dev/loop0
$ lsblk -f /dev/loop?
NAME FSTYPE LABEL UUID MOUNTPOINT
loop0
Is this enough to protect this "light storage" VM from the other VMs that attach to the filesystem images?
Are there any other precautions that I should configure for this VM?
As I encountered with dom0, using the rule above prevents Qubes from detecting the loop devices associated with the disk image file. Using similar commands as shown in the docs you linked, I fixed this with:
$ qubesdb-write /qubes-block-devices/loop0/desc 'disk.img'
$ qubesdb-write /qubes-block-devices/loop0/size 536870912
$ qubesdb-write /qubes-block-devices/loop0/mode w
After doing this `qvm-block ls` detects the new disk, but the Qubes GUI (i.e. the widget in the top right corner of the window manager) does not detect the change. How can I trigger a "refresh" so that the GUI sees my changes to qubesdb?
Thanks for your assistance.
Marek Marczykowski-Górecki
Dec 25, 2019, 3:33:11 AM12/25/19
to heombeeh, qubes...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hash: SHA256
On Wed, Dec 25, 2019 at 03:00:36AM +0000, 'heombeeh' via qubes-devel wrote:
> > > If I am correct that the new loop device method causes dom0 to become exposed, what is the correct way to attach a file in dom0 to a VM, so that dom0 is not exposed to an attack?
> >
> > Yes and no. It was always recommended to not store user (especially
> > untrusted) data in dom0 manually. Documentation specifically handle the
> > case of "source VM", not "dom0".
> >
> > > Also, what is the method that qubes uses to protect dom0 from the filesystems contained in VM blocks in the qubes pool? Is it the udev rules that are contained in `/etc/udev/rules.d`?
> >
> > Yes, those rules.
> >
> > > Note: I tried adding a udev rule copied from the default rules to prevent the OS from processing the filesystem data of the loop device. This worked, however it also somehow prevented qvm-block from seeing the loop device, so I was unable to attach it to my VM.
> >
> > Yes, devices excluded from udev also are excluded from qvm-block. But we
> > have documented method to force showing them:
> > https://www.qubes-os.org/doc/mount-lvm-image/
> >
> > It works also for loop devices - simply use "loopX" instead of "dm-X"
>
> Thanks a lot for these clarifications.
>
> Based on your recommendation against handling filesystem images within dom0, I'm thinking that I can instead create a "light storage" AppVM. This VM can simply have a directory of sparse files together with associated loop devices, and I can then attach them to other AppVMs normally using qvm-block.
Sounds good.
> > > If I am correct that the new loop device method causes dom0 to become exposed, what is the correct way to attach a file in dom0 to a VM, so that dom0 is not exposed to an attack?
> >
> > Yes and no. It was always recommended to not store user (especially
> > untrusted) data in dom0 manually. Documentation specifically handle the
> > case of "source VM", not "dom0".
> >
> > > Also, what is the method that qubes uses to protect dom0 from the filesystems contained in VM blocks in the qubes pool? Is it the udev rules that are contained in `/etc/udev/rules.d`?
> >
> > Yes, those rules.
> >
> > > Note: I tried adding a udev rule copied from the default rules to prevent the OS from processing the filesystem data of the loop device. This worked, however it also somehow prevented qvm-block from seeing the loop device, so I was unable to attach it to my VM.
> >
> > Yes, devices excluded from udev also are excluded from qvm-block. But we
> > have documented method to force showing them:
> > https://www.qubes-os.org/doc/mount-lvm-image/
> >
> > It works also for loop devices - simply use "loopX" instead of "dm-X"
>
> Thanks a lot for these clarifications.
>
> Based on your recommendation against handling filesystem images within dom0, I'm thinking that I can instead create a "light storage" AppVM. This VM can simply have a directory of sparse files together with associated loop devices, and I can then attach them to other AppVMs normally using qvm-block.
> However, just as in dom0, it will be good to protect this "light storage" VM from those client VMs that utilize its filesystem blocks. Can I protect this VM similarly to how dom0 is protected by using similar udev rules?
>
> For example, suppose I keep my sparse files in `/home/user/myimages/{name}/disk.img`, and then add the following udev rule:
>
> $ cat /etc/udev/rules.d/00-qubes-ignore-devices.rules
> ACTION!="remove", SUBSYSTEM=="block", KERNEL=="loop*", ATTR{loop/backing_file}=="*/myimages/*/*.img*", ENV{DM_UDEV_DISABLE_DISK_RULES_FLAG}="1", ENV{UDEV_DISABLE_PERSISTENT_STORAGE_RULES_FLAG}="1"
It is a bit fragile, because newer udev version could ship rules
ignoring those settings (yet another reason why updating dom0 isn't
straightforward), but in the current version it is still effective.
> With this active, it seems to work:
>
> $ sudo losetup --find --show ./myimages/blah/disk.img
> /dev/loop0
> $ lsblk -f /dev/loop?
> NAME FSTYPE LABEL UUID MOUNTPOINT
> loop0
>
>
> Is this enough to protect this "light storage" VM from the other VMs that attach to the filesystem images?
> Are there any other precautions that I should configure for this VM?
>
> As I encountered with dom0, using the rule above prevents Qubes from detecting the loop devices associated with the disk image file. Using similar commands as shown in the docs you linked, I fixed this with:
>
> $ qubesdb-write /qubes-block-devices/loop0/desc 'disk.img'
> $ qubesdb-write /qubes-block-devices/loop0/size 536870912
> $ qubesdb-write /qubes-block-devices/loop0/mode w
>
> After doing this `qvm-block ls` detects the new disk, but the Qubes GUI (i.e. the widget in the top right corner of the window manager) does not detect the change. How can I trigger a "refresh" so that the GUI sees my changes to qubesdb?
qubesdb-write /qubes-block-devices ''
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
1ywERAf9FfzECJsRTTIowvtRNRQarx79r7vndlPa1J5bQTDn6Gwd+Rn7kMXOd3ne
TlCOLyWv/1xvYge7lFcCKn8FYGaX6g4fFLNGdXdL40fjrtyEJs3MVAC2Y3sZnFF6
ojMcRdMV6lkZiZBNKbRjGePC4saEv7lwt/4XIgq2ietcuycSQyVC4eCjF0tMicoX
ThiYY19MGOZyRV2woPza79eBTP4LLXFDZ+DapaQU/l5DpLEqOyt4n8BEYPD2zTJJ
/56V3pKhIA5DTKzBO6bNbqzzUF/py3nl1Qmw35tdv4vLmPZWYatGltT+2Yvtv2HL
ngd8A14XxeyZ9K1SKrGnswcvichVGQ==
=bbeH
-----END PGP SIGNATURE-----
Demi M. Obenour
Dec 25, 2019, 5:53:48 PM12/25/19
to qubes...@googlegroups.com
thinking of writing a udev rule that prevents the loop device from
being parsed, but still runs the Qubes hotplug scripts so that the
device shows up in the GUI.
Sincerely,
Demi
Marek Marczykowski-Górecki
Dec 25, 2019, 6:34:16 PM12/25/19
to Demi M. Obenour, qubes...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hash: SHA256
it only uses whatever is already available in udev and sysfs.
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
1yxRmQf/R3as1SV1Xiu4R/GCEf7UwLvmucUNYZVXZzVc/aSoAdNPHFec4FtosCJ1
/QjKf8hctrwdLf4yqG62I3Xa3vQHd49gkUkm9U03U06BySA5pPEQ/ZSP75CMl7iJ
ARnUdxKxrtTOuM23kRlnGYZe4UpwjBzxnM12SDyV+Y/3MsKmB0mYBYm3xZFnZ3qX
/K46EpIVtL/esvLPEcTX0zW8yPD+k4hJq3YWqA43QVfjRnCUHxZvblB5ZNYTegtZ
A8cOVmqYXAckXI1LOcFouOlfd/i+wS0RqZAOFDM4HwXrKvm7HPjwVFbqhYB0xvw3
xWvWtY6gGBa74Ytn0yz/PwoU/9umJw==
=61av
-----END PGP SIGNATURE-----
Reply all
Reply to author
Forward
0 new messages