How to attach private storage from one AppVM to another AppVM (LVM)?

[heinrich...@googlemail.com]

From one AppVM I need to temporarily access a large amount of files from another AppVM. Can this be done without copying the files around?

Background:

I have a large amount of files stored in AppVM "BIG". That's hundreds of GB in a separate pool on a spinning HDD.

I also have a small AppVM "SMALL" running a program that needs to access files from "BIG". This AppVM resides on a small SSD.

In the past I copied files from BIG to SMALL. But this takes time and I need to sort the files beforehand because there is not enough space on the SSD. I don't want to do that anymore. It would be okay to allow AppVM "SMALL" to access files from "BIG"'s private storage directly.

Googling around tells me to mount "private.img", but I'm using LVM so that's not an option. But how can this be done? Can it be done? (Or is there even a better "file sharing" approach for this amount of data without having to revert to a NAS?)

Any tips are appreciated.

(I'm on Qubes OS v4 latest)

[unman]

On Mon, Dec 07, 2020 at 06:56:47AM -0800, 'heinrich...@googlemail.com' via qubes-users wrote:
> >From one AppVM I need to temporarily access a large amount of files from
> another AppVM. Can this be done without copying the files around?
>

> *Background: *

> I have a large amount of files stored in AppVM "BIG". That's hundreds of GB
> in a separate pool on a spinning HDD.
> I also have a small AppVM "SMALL" running a program that needs to access
> files from "BIG". This AppVM resides on a small SSD.
>
> In the past I copied files from BIG to SMALL. But this takes time and I
> need to sort the files beforehand because there is not enough space on the
> SSD. I don't want to do that anymore. It would be okay to allow AppVM
> "SMALL" to access files from "BIG"'s private storage directly.
>
> Googling around tells me to mount "private.img", but I'm using LVM so
> that's not an option. But how can this be done? Can it be done? (Or is
> there even a better "file sharing" approach for this amount of data without
> having to revert to a NAS?)
>
> Any tips are appreciated.
>
> (I'm on Qubes OS v4 latest)
>

Take a look at https://qubes-os.org/doc/mount-lvm-image/
That explains how to mount an lvm image.

Alternatively you could look at https://github.com/unman/qubes-sync
where I outline how to rsync data over qrexec.
I've updated that to include sshfs over qrexec, but don't seem to have
pushed it up yet. That'll have to wait until the morning.
But the principle is simple - run sshd on the target
instead of rsyncd: use a forwarder, and then mount the remote directory
using sshfs on the client. That removes the need to copy files around,
and keeps a single archive accessible from other qubes.
That should give you idea of how to get started - if you need help let
me know and I'll try to help in the morning.

[heinrich...@googlemail.com]

sshfs sounds great since this probably allows me to also select the folders I want to share with the SMALL AppVM instead of revealing all files.

Thanks for taking the time to look into this and I'm looking forward to the push :)

[Ulrich Windl]

On 12/7/20 3:56 PM, 'heinrich...@googlemail.com' via qubes-users wrote:
> From one AppVM I need to temporarily access a large amount of files
> from another AppVM. Can this be done without copying the files around?
>

> _Background: _

> I have a large amount of files stored in AppVM "BIG". That's hundreds of
> GB in a separate pool on a spinning HDD.
> I also have a small AppVM "SMALL" running a program that needs to access
> files from "BIG". This AppVM resides on a small SSD.
>
> In the past I copied files from BIG to SMALL. But this takes time and I
> need to sort the files beforehand because there is not enough space on
> the SSD. I don't want to do that anymore. It would be okay to allow
> AppVM "SMALL" to access files from "BIG"'s private storage directly.
>
> Googling around tells me to mount "private.img", but I'm using LVM so
> that's not an option. But how can this be done? Can it be done? (Or is
> there even a better "file sharing" approach for this amount of data
> without having to revert to a NAS?)
>
> Any tips are appreciated.

Actually I have not done it, but it feels like you should have an NFS
server on BIG with a network only accessible from inside qubes, and
specifically from SMALL. Still it will have to transfer the file
contents, but you benefit from any application that only reads parts of
the files.

When not wanting to copy I guess you'll have to mount a snapshot of
BIG's data as the LV should be mounted only once (AFAIK).

Regards,
Ulrich

>
> (I'm on Qubes OS v4 latest)
>

> --
> You received this message because you are subscribed to the Google
> Groups "qubes-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to qubes-users...@googlegroups.com
> <mailto:qubes-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/qubes-users/5d42e07f-9170-4504-bbc4-d638d2403cfcn%40googlegroups.com
> <https://groups.google.com/d/msgid/qubes-users/5d42e07f-9170-4504-bbc4-d638d2403cfcn%40googlegroups.com?utm_medium=email&utm_source=footer>.

[unman]

On Mon, Dec 07, 2020 at 07:59:36AM -0800, 'heinrich...@googlemail.com' via qubes-users wrote:
> sshfs sounds great since this probably allows me to also select the folders
> I want to share with the SMALL AppVM instead of revealing all files.
>
> Thanks for taking the time to look into this and I'm looking forward to the
> push :)
>

Somewhat belatedly, pushed this morning.
Let me know how you get on.

[Stuart Perkins]

What I have done is...

created a large disk image (600g or so) on the mail dom0 drive space

written a script in dom0 to attach the image to whichever machine I want to access it from.

This script..
attaches the image to a loop device
mounts the image to the machine I desire
added /etc/hosts entries to each app vm to mount to a dedicated directory when doing "sudo mount -a" by uuid
script ends with an execution of "sudo mount -a" on the target vm

there is also a corresponding unmount script, so if it is mounted to "mail" I can easily unmount it and mount it to "money".


This lets me treat the 600g.img file like a removable media which can be attached to any app vm. I also back that drive image up on my network server.

Stuart

[heinrich...@googlemail.com]

Thank you all for describing the various approaches, this is great feedback and all sound feasible. I'll start testing soon. Ultimately I'll probably move all the files over to some network location if it proves fast enough. The BIG VM so far caused nothing but problems anyway. But currently it is there so I have to cope with it.