Improvement: check disk space before copy to VM

Robert Mittendorf

unread,

Nov 11, 2016, 3:39:34 PM11/11/16

to qubes-users

I just copied a file from dom0 to a AppVM via qvm-copy-to-vm.
The file transfer started until the private storage was full.
It would be better to check the free disk space size before executing
the copy command.

regards,
Robert

Andrew David Wong

unread,

Nov 11, 2016, 5:34:28 PM11/11/16

to Robert Mittendorf, qubes-users

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Good suggestion. Thank you!

https://github.com/QubesOS/qubes-issues/issues/2429

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJYJkdmAAoJENtN07w5UDAwjrIQAKcmYvgJB/pWzbrE008+SejT
mSJd1r3lUE8cXnXbP0iP3iw0Lt8VuItmmoIF22TGtiN6Yyu0GU/d752E9dBEY22w
W5Buvni0gU3We/s+2jvdkT8WcugZoYxH8V8hQ3qE/pyNoTADZNj5jEF+YRN29NVb
BFWzOZqqYxnzzgJxxM0RWht1CC67D79jq2BpQbQbyZdyquhPA7pHGUglvP31JEFO
Dp36VI+npJyTA/jvBP9VQ2dG5OVZnhrxcShUAPBJ8rbw92w7AfzeXqjChNIcmqwX
RxVkSygfvotkHvh5dii84MYbF1f0rc7tdX95bN5f4qDu1JtSU/hRvLGSYMUezeNN
nBJK+5vmW8YPptYZni3gX0P+R1D+TTBaxBhEgKefw2yWENELwVWkeIbaN8PEGqN2
dQMgOhU5L9YYqxT92hSI7U/iHbQi3GfPn4oVE/rFRIgbzqvuFGpj1smHxb5NMHR8
f0Fo6OAE57cqZImad9ItvQCtpefs4mBFQqsLWxMaAd4Iv2VjjJ2TC1Qsb5yDk5kJ
zMMfGU6Xd7Yr863ujsy3j1wFKNGtScTmzAUKoBZbhOW1RkkyHe2RtJRDtLETThHC
C0IweUjRwGJdoKwWRT4B2JlH6GiwTj7oVfpO8Tkmpe97BPvmKE6N0aH/PleAztYC
kKyJ5acIovRGQfMvIZs0
=KY90
-----END PGP SIGNATURE-----

Marek Marczykowski-Górecki

unread,

Nov 11, 2016, 5:57:12 PM11/11/16

to Andrew David Wong, Robert Mittendorf, qubes-users

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Fri, Nov 11, 2016 at 02:34:16PM -0800, Andrew David Wong wrote:
> On 2016-11-10 08:16, Robert Mittendorf wrote:
> > I just copied a file from dom0 to a AppVM via qvm-copy-to-vm.
> > The file transfer started until the private storage was full.
> > It would be better to check the free disk space size before executing
> > the copy command.
> >
> > regards,
> > Robert
> >
>
> Good suggestion. Thank you!
>
> https://github.com/QubesOS/qubes-issues/issues/2429

Actually I don't think it is a good idea. File copy protocol is
intentionally very simple, including being unidirectional. We don't want
to add any non-essential features there, to keep it as simple as
possible.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJYJkzCAAoJENuP0xzK19cs8Z8H/iWhG46WGSl/i2G6pzpyuUg5
dgAvz+UqgEWwQQVlYFA4tLqWcKbNaBhZB5n+MX8zFPU2MnBS6DCcbxx5DgpPc2Dn
SDGx/snbf+6GMBRFPXLtUdfgYTCK8l57NsKYY7nc+5P0IUBn2VV5swRe1WQ7rzMP
n5C9uLSVYpWfZs4FE65hRDCGLU/POCCnZvUkLXc0Mmk2VHHlOwkdCADECnSzpqTa
F2l0BsaIWYcIY+MCckj1oDZMLVsRa6MPs5O6mZ7nqzCs7dzEjJnRAR15T21u9BHh
SJWLW3o2ON+wsmRVi0EftfgOMJObPEIKTXtADRDN/u2wFC2EVH6THWJgXPWt+lM=
=8CN/
-----END PGP SIGNATURE-----

Marek Marczykowski-Górecki

unread,

Nov 11, 2016, 5:58:42 PM11/11/16

to Andrew David Wong, Robert Mittendorf, qubes-users

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Fri, Nov 11, 2016 at 11:57:07PM +0100, Marek Marczykowski-Górecki wrote:
> On Fri, Nov 11, 2016 at 02:34:16PM -0800, Andrew David Wong wrote:
> > On 2016-11-10 08:16, Robert Mittendorf wrote:
> > > I just copied a file from dom0 to a AppVM via qvm-copy-to-vm.
> > > The file transfer started until the private storage was full.
> > > It would be better to check the free disk space size before executing
> > > the copy command.
> > >
> > > regards,
> > > Robert
> > >
> >
> > Good suggestion. Thank you!
> >
> > https://github.com/QubesOS/qubes-issues/issues/2429
>
> Actually I don't think it is a good idea. File copy protocol is
> intentionally very simple, including being unidirectional. We don't want
> to add any non-essential features there, to keep it as simple as
> possible.

BTW None of file copying tools I know do that (cp, rsync, scp, ...).

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJYJk0cAAoJENuP0xzK19cs83gH/iV9H3AeWAWm9QUaRA/Q2pME
f04xhnqz5h54JGIQHvRfYy3XqkGsomC1QzFpSUMnIVAeAXNFfENfVzy5sHSEh/b6
V24tNY2RGl5sNeZMCyVvz3LW10n+nuBdpQ/8lrsYaRSqMAkcI4UyCkl8+Ve5MO/z
sOqyFP/T6UbG75JLvlDmAthtNrxQ55D3jYbJ01ZpCKqYfEBKtY2iMCGzJOZLLnA1
IdCqArR0RiQxll/qLuOnscyj5q6wy3KnQsEHulVjjEGP5rYngkfBgwWu3j2W6app
Gkl8BunTL6JK9HC2N/D8HpJygCTbI1DmAq5vh0AEEWgy8pGCUxEufvCB/i6j9iM=
=Fh8Q
-----END PGP SIGNATURE-----

Andrew David Wong

unread,

Nov 11, 2016, 6:25:41 PM11/11/16

to Marek Marczykowski-Górecki, Robert Mittendorf, qubes-users

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 2016-11-11 14:58, Marek Marczykowski-Górecki wrote:
> On Fri, Nov 11, 2016 at 11:57:07PM +0100, Marek Marczykowski-Górecki wrote:
>> On Fri, Nov 11, 2016 at 02:34:16PM -0800, Andrew David Wong wrote:
>>> On 2016-11-10 08:16, Robert Mittendorf wrote:
>>>> I just copied a file from dom0 to a AppVM via qvm-copy-to-vm.
>>>> The file transfer started until the private storage was full.
>>>> It would be better to check the free disk space size before executing
>>>> the copy command.
>>>>
>>>> regards,
>>>> Robert
>>>>
>>>
>>> Good suggestion. Thank you!
>>>
>>> https://github.com/QubesOS/qubes-issues/issues/2429
>
>> Actually I don't think it is a good idea. File copy protocol is
>> intentionally very simple, including being unidirectional. We don't want
>> to add any non-essential features there, to keep it as simple as
>> possible.
>
> BTW None of file copying tools I know do that (cp, rsync, scp, ...).
>

Oh, ok! Fair enough!

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJYJlNnAAoJENtN07w5UDAwx9wP/jRPsCXbukFCL77N57hQ4jiP
4VK8zEiVKtFeHl7NGp8QNBLecDHN+wHmw1PD2uKWiMJRRpqkrPHZS1NCcwFii+IA
0AowwU5OkpAM/48TeSgB0+ss9R9bV8/G8B/zrmaMQuE0EEA/xVFnKN3sCqPZL1aU
9kbLfgvITfbdrwl1V0frxZR18e1I5j+eNyvwjC3QFmc3aMrU486VsVwKD/0KBKgu
hUj2tXsgXGBaIJ2KJflt0qHQCuYljjFa7KElVZ/O9Gq1Md9+4roP2iMSqunQcdpH
fX0w8iHGU+zGPssCv2NZ7CJ1kfdhv6RI9YzuGF+DoKYICXS46imOS8aQGDnMMYIu
SoaxLRJklN4TwbV2GXn9TmzAb+Svv2IFj0f/zPhMjQAhLmKpvkmEJpzQvj54eR2g
Fsx8OA7ESpPEalw/cR5fBpmWWHVWwd00GI2JC1cU4xqRVgOIqTWei0TQuT1NRPUA
/7Bd7/9BVwNcTWOFBmn8XVBJvG0LtYAnwVPb8RAHYazE4GH89fNoNFzPJbYO6+yw
KDSv6I3YguwXCP3pv8jPYV/zr7HXKRnL80Xmq4hFs9pOwlioQZpq1Lp8EWzUkYLG
m18lxXicsKS/HMLL9g54l94zhYVC9SzTXBkK9KmoLRxkGumzLCeXFmngHNnDh0Cs
0zLdR1MVNLtf6kPge6oy
=6pmL
-----END PGP SIGNATURE-----

Robert Mittendorf

unread,

Nov 14, 2016, 4:31:25 AM11/14/16

to qubes-users

On 2016-11-11 14:58, Marek Marczykowski-Górecki wrote:
>
> >> Actually I don't think it is a good idea. File copy protocol is
> >> intentionally very simple, including being unidirectional. We don't
> want
> >> to add any non-essential features there, to keep it as simple as
> >> possible.
>
> > BTW None of file copying tools I know do that (cp, rsync, scp, ...).

Well, I somewhat understand the first argument, but not the second. To
have a bad usability and waste poeple's time just because other tools do
is not a good argument I think.

Obviously it is not unidirectional, otherwise the source would not know
"out of disk space". This does not have to be an interactive feature,
though.
Why not give the "out of disk space" error before accepting the
transfer? The communication from sink to source would be the same, but
less time would be wasted.

Sec Tester

unread,

Nov 14, 2016, 5:49:58 AM11/14/16

to qubes-users, mitte...@digitrace.de

Could open up a vulnerability if not done carefully.

VM could use it to query and identify other VMs in existence on the system.

But if it required a dom0 authorization before checking & transferring, should be ok.

Salmiakki

unread,

Nov 14, 2016, 7:03:42 AM11/14/16

to qubes-users, mitte...@digitrace.de

Maybe it could be done more general by just popping up a warning saying "this VM is low on disk space".
It would not work for cases where you transfer extremely large files (or at least it would only display the warning after transferring quite a large chunk).
However, it would also work for everything else!
For example I once had a problem because I decided to sync my entire imap folder for my mail VM and that is larger than 2 GB. The problem was actually kind of hard to spot. Ideally of course, the warning would be accompanied by an option to just extend the space (since it seems to be possible to do that while the VM is running anyway).

Andrew David Wong

unread,

Nov 14, 2016, 7:16:48 AM11/14/16

to Salmiakki, qubes-users, mitte...@digitrace.de

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

I've actually seen such a warning in 3.1 and earlier (haven't had cause to see it yet in 3.2), so it must already exist in some sense (or did exist). I never bothered to look into how the threshold is calculated, though.

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJYKascAAoJENtN07w5UDAwJv0P/jiO/pGfAyIGv+eENZ35Z1GY
+dR3udiEvJJvtMV3kPzaY3uOGwF72jwesjdwQo3DxgOZWpeMH5JzJQdHvx1WXoqp
dmpAJJ8VV0tI2JErI6gpxjKo47oIlog1g83u0p6iRksYE7wtqj4+4jsCsZiXO/B3
hsCFBV0f4UhlRPeBr9G23CJ93vn5sgcBvgnvN1whQRCBMZgVRATakmsmkpio0yIH
YbuqGjp35zZZyOGL4ZOybzxEPWiBoZPB+wXDHr1YLQGEhMaZfnKtPgt2HHu39WL/
nw8qotIjUaD3/bFOHncc90yIfpnQZKZaBwxk+hdW+1Rnecv/NiN4l33O+MiPNiRW
wrab84LMvoYJal9J35nVQrxAd+Hmpi/G96Lkr1TQTbnhsIA40+LkGLAxK4Gl9Gl/
8h+irOcYW+z7AQ76XqiRt/GLNGNhSVbPBj1MAfo48PUo9og31FizlDnoumQdrUYc
eAkZ/pjkcbsB2L73jrjYajHahIUkTbXmrUlZAU5mkjjCIjEJ7S7EzngGkSKayN4E
xvZzYZygOydXVcYCFWOewkhphpIUTPcihaXZZGfRKTpYivm7/6I9/0HX3eGl1JGQ
qNAIBaaLm/n8JYa5+3ADHZONLl+HHKZj2arWrmbNKQac5iCnpUMEZ69w04dbIYzy
nLrYKL4b+LA/hZrB6pt8
=MNnR
-----END PGP SIGNATURE-----

Unman

unread,

Nov 14, 2016, 8:35:39 AM11/14/16

to Andrew David Wong, Salmiakki, qubes-users, mitte...@digitrace.de

That message is thrown by the qube itself. It's a standard free space
check, not a Qubes thing.

On the question of "bad usability" it looks like horses for courses. I
personally don't want a warning if I've already started to clear out
space on the target. And if I'm transferring files as they are being
written, what use would the measure of free disk space be?

It isn't the TOOL here that will "waste people's time". They waste
their own time by not checking beforehand. There are some OS and tools
that try to fix this, but you see plenty of users baffled by the way
they work, and unsure why the warning arise. Best to keep it simple imo.

unman

Robert Mittendorf

unread,

Nov 14, 2016, 8:46:33 AM11/14/16

to qubes...@googlegroups.com

> It isn't the TOOL here that will "waste people's time". They waste
> their own time by not checking beforehand.

Why should a user always check the free disk space, if a tool can easily
do it? That's wasting time as well. Especially in the Qubes environment,
where you are easily handling tens of "disks" (at least one per Qube)
and you most likely do not always now which disk is how big and how full.

> There are some OS and tools
> that try to fix this, but you see plenty of users baffled by the way
> they work, and unsure why the warning arise. Best to keep it simple imo.
>
> unman

One basic principle of usability is to make it hard to make mistakes
(including destroying work/files).

As I stated before I think the protocol would not have to become "more
non-unidirectional" to improve on this.

Andrew David Wong

unread,

Nov 14, 2016, 8:54:02 AM11/14/16

to Unman, Salmiakki, qubes-users, mitte...@digitrace.de

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

> That message is thrown by the qube itself. It's a standard free space
> check, not a Qubes thing.
>

Oh, I know. But nonetheless, it's there, and it warns about low disk space in the qube.

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org

Unman

unread,

Nov 14, 2016, 9:58:53 AM11/14/16

to Robert Mittendorf, qubes...@googlegroups.com

On Mon, Nov 14, 2016 at 02:46:29PM +0100, Robert Mittendorf wrote:
>
> > It isn't the TOOL here that will "waste people's time". They waste
> > their own time by not checking beforehand.
> Why should a user always check the free disk space, if a tool can easily
> do it? That's wasting time as well. Especially in the Qubes environment,
> where you are easily handling tens of "disks" (at least one per Qube)
> and you most likely do not always now which disk is how big and how full.

The obvious answer is that they would check to avoid wasting time.
Can a tool do this? Obviously. And it could try to automagically clean up
and find free space.
Do I want that in Qubes? Not really.

> > There are some OS and tools
> > that try to fix this, but you see plenty of users baffled by the way
> > they work, and unsure why the warning arise. Best to keep it simple imo.
> >
> > unman
> One basic principle of usability is to make it hard to make mistakes
> (including destroying work/files).
>
> As I stated before I think the protocol would not have to become "more
> non-unidirectional" to improve on this.

There is, I think, no question of destroying work or files. The simplicity of the
tools militates against that.
Obviously, if you want to add this feature, you can. A simple wrapper
like this could be a start:

#!/bin/bash
DEST=$1
FILE=$2
SIZE=`du -B 1K $FILE|cut -f1`
FREE=`qvm-run $DEST df /rw --output=avail|tail -n1`
if [ $FREE -gt $SIZE ]
then
qvm-copy-to-vm $DEST $FILE
else
echo "Not enough space on $DEST!"
fi

Obviously this should be implemented using the qrexec framework

unman

Achim Patzner

unread,

Nov 14, 2016, 11:50:10 AM11/14/16

to qubes...@googlegroups.com

Am 14.11.2016 um 14:46 schrieb Robert Mittendorf:

> One basic principle of usability is to make it hard to make mistakes
> (including destroying work/files).

Imagine a guy dressed in an elaborate tin can standing behind you,
kicking you down some cliff shouting "THIS... IS... UINX...". Really, it
is. Failing to copy a file is nothing dramatic. Nothing is destroyed,
nothing erased. Let some air out of the elephant until you can recognize
the shape of the original mosquito, would you?

> As I stated before I think the protocol would not have to become "more
> non-unidirectional" to improve on this.

Why don't you just write a proof -of-concept and put it on github? If it
is working well and showin an improvement I'm sure someone will add it
to the Qubes repositories. They are not that dogmatic.

Achim

Jean-Philippe Ouellet

unread,

Nov 14, 2016, 2:42:48 PM11/14/16

to Sec Tester, qubes-users

On Mon, Nov 14, 2016 at 5:49 AM, Sec Tester <sectest...@gmail.com> wrote:
> Could open up a vulnerability if not done carefully.
>
> VM could use it to query and identify other VMs in existence on the system.

There are already several timing side-channel ways to do that.

Example:

AppVM$ time /usr/lib/qubes/qrexec-client-vm sys-net qubes.VMShell
Request refused
/usr/lib/qubes/qrexec-client-vm sys-net qubes.VMShell 0.00s user
0.00s system 1% cpu 0.180 total

AppVM$ time /usr/lib/qubes/qrexec-client-vm does-not-exist qubes.VMShell
Request refused
/usr/lib/qubes/qrexec-client-vm does-not-exist qubes.VMShell 0.00s
user 0.00s system 0% cpu 1.565 total

In this case the difference in time is quite obvious because it blocks
while an error dialog is open in dom0.

Jean-Philippe Ouellet

unread,

Nov 14, 2016, 3:18:49 PM11/14/16

to Sec Tester, qubes-users

On Mon, Nov 14, 2016 at 2:42 PM, Jean-Philippe Ouellet <j...@vt.edu> wrote:
> On Mon, Nov 14, 2016 at 5:49 AM, Sec Tester <sectest...@gmail.com> wrote:
>> Could open up a vulnerability if not done carefully.
>>
>> VM could use it to query and identify other VMs in existence on the system.
>
> There are already several timing side-channel ways to do that.
>

> Example: ...

Created an issue for this because it's really off-topic for this thread:
https://github.com/QubesOS/qubes-issues/issues/2436

Reply all

Reply to author

Forward