"Premature end of archive" when restoring backup from a USB VM

[Jeremy Rand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I recently needed to restore a backup of all my VM's and dom0
configuration. The backup was on a USB hard drive (created using a
USB VM). I inserted the hard drive into a USB port owned by a USB VM,
and verified that the drive was mounted and that the Qubes backup was
visible to the USB VM. I then attempted to restore the backup from
the Qubes VM Manager. When I gave it the path to the backup file, I
got this error:

> Premature end of archive while receiving backup header. Process
> output:

No other information was displayed. I was able to successfully
restore the backup by connecting the hard drive to dom0, so I'm
certain that the backup file was fine. (But obviously I'd prefer not
to connect stuff to dom0.)

The only possibly relevant info I can think of is that the path to the
backup file had some spaces in it (in both the USB VM and dom0). I
tried placing the entire thing in double-quote marks, which made no
difference. I doubt this is the issue.

Any idea what's wrong?

Thanks,
- -Jeremy Rand
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJWNq4cAAoJEAHN/EbZ1y06+noQALlbKZEXeIHjXTfl6dA+Lhw4
P+zoUGvDreMzERb+V4tcs/2bvNLhFqTA4sn9d+XwHd4tYQzu1zpX2vor+2qrPAlr
BPKMXveh3CpExrHNa1nwftVpFnPAtEM+ni7uuK4mcBXa0uV+2sAVn7kj4Bx5qcBW
xagD6TfTtyWGlwL0S5o6P7m3o4jKAMmoz9TwQ2HOSKj357iM7BwMQ+gn5Cb7Aiuj
TY72aAX8QEF9FkDyIgrXNc7fJxyKJj64Wps4qwMGbmagNUXBLp8wvX/vstf6Bby8
npt+KmJ50HPQqzTYhOBrnz4Z7T/4/+HTl6/BCAbr8LNLv59Ov0NnCOsXErIKRftw
S4ztNC3lDpGdcqmyJSAxL9wIG5YrpI7gnFs6zUmavlgs28Stpo+MkMUN2HbT+143
UoNzQeaHUj18G28jyt0MGtgvqO4+krVweq2Z0UmwqJJ4a4CrLp5d+LX/l1/8icA5
T+keUbxPhS4LWPePXzCaM69uoBAR1GofKWSekHzc7Y+YlJeq9XRiUEfkqBOAx2rZ
NwDLZ/+CWH9GNGMYz9LrHNyOXNiCAnpeav7oUj0nD5KFFUkZOP1tsIvCI1w6Pg3C
hvU8iKs+hCZHuJDGRmQYZW6EqhAC/W+7Fxy9Tyid+hiQ9j9x7QiMjE5LgXK/qmEG
cqijwwO5FKxXd2K95cgU
=GF2t
-----END PGP SIGNATURE-----

[Drew White]

If you are not wanting to have it connected to Dom0, then connect it to a VM and connect via that.

The resoration and backup utility allows connection to different VMs.

However, using directly through Dom0 is most secure and reliable.

The premature end could mean that the drive has a corrupted sector, or a bad block.
Is it an SSD or HDD?

I create backups of EACH virtual in separate backups. Rather than backing up one per.

On that note, I also TAR + BZip2 the directory itself to create my own backup on a network drive.

If things fail, I always have my own backup that is safely stored on a raided drive set. I just extract using tar, and I have my files there again.

I've had issues with the backup tool before, especially when the VM it used as a NetVM was no longer available.

Which is something that needs to be fixed in Qubes.

[Jeremy Rand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi Drew,

On 11/01/2015 06:42 PM, Drew White wrote:
> If you are not wanting to have it connected to Dom0, then connect
> it to a VM and connect via that.

If you had actually read my message, you would see that that's what I
tried first.

> The resoration and backup utility allows connection to different
> VMs.

I assume you mean Qubes VM Manager's restore utility? I specifically
said that's what I was using, and that I tried exactly that.

> However, using directly through Dom0 is most secure and reliable.

I gathered that from my experience. However, that's not useful advice,
since using dom0 for this has security risks that aren't present when
using a VM. Hence why I reported the issue.

> The premature end could mean that the drive has a corrupted sector,
> or a bad block.

That makes no logical sense. How would the drive have a corrupted
sector when restoring via dom0 worked with no trouble?

> Is it an SSD or HDD?

It's an HDD, but again, I don't think your speculation here makes any
sense.

> I create backups of EACH virtual in separate backups. Rather than
> backing up one per.

Good for you, but I have a large number of VM's, and I cannot be
bothered to back them up separately when I'm migrating systems. You
also are not explaining how that even plausibly has something to do
with the issue I reported.

> On that note, I also TAR + BZip2 the directory itself to create my
> own backup on a network drive.

It is obvious that if I backed up using a tool other than Qubes VM
Manager, I wouldn't be affected by a bug in Qubes VM Manager. That's
not remotely helpful in getting my issue fixed.

> If things fail, I always have my own backup that is safely stored
> on a raided drive set. I just extract using tar, and I have my
> files
there again.
>
> I've had issues with the backup tool before, especially when the VM
> it used as a NetVM was no longer available.
>
> Which is something that needs to be fixed in Qubes.

The USB VM in question didn't have any NetVM connected. Why would I
want to expose my USB controller to the network exactly?

I get the distinct impression that you didn't actually read or
understand the issue I reported. Also, please stop top-posting, it
makes reading your post unpleasant.

Cheers,

- -Jeremy Rand
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJWNrhLAAoJEAHN/EbZ1y06cgEQANFVNmdkdpWh7TKqc6l/ZkX5
3KFISExokKdbIcuAedhbgO6aZdYlxirHpKIt8gQZgAeny/nyKCLY2zoMJRLYnZq5
gs+syg5nSKig/VIf4MAxUniFAeR6KbsAc7kwLLPUdM9mdIOCmjf+ddxCfUH+b827
Otae5BqsWGSLP/PHcrR1GtUnltlRpIqeFEn4mfhvaCxi43jRBhMbVMvHzVfurbA+
nC2dk2oQhI4C3uWlEiXrGhqjWBjuJoAu8tWpiOJUyGNM9DTNX4/6pmHGw2mcta4A
3g+bjcmvuc10NgnuuC23O3rRFKxSLeXxj9Pn+lVGQjKYk6F9mI+Sxkb1x3BmZVjK
dELCVcUzlF1/JL7DAQ6Zi91sUpHxluORS039ep8M8cHU/8UQ448nxqcsn4vQENDF
WgY+ljXg1b/ZLG5nLWnRfbrj6WW9uVwdjZz8CmcN8A7tN+1fyRU45ZD1f9rOTiD5
MYPrWU2x9pl9/awLHL9EAcBx03kz1O4sKBsD0M0EdTfPm9+EO0EYY7jySAoSVdyw
u1G9PUBVWWoUVOasG/XGlcsEpsCO86ZSU+CV6jbf2xD8udeXwXxQlPLhxBeFOZ2R
U0JAiFxlzhYunuAjuKsU+AdOLTpyiMzv47oPyPxeQaAkCxjqSOSTYnrKCxm48e6n
0+ZV2fSwXa1w0w2ZMWB6
=t1Kp
-----END PGP SIGNATURE-----

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Sun, Nov 01, 2015 at 06:28:20PM -0600, Jeremy Rand wrote:
> I recently needed to restore a backup of all my VM's and dom0
> configuration. The backup was on a USB hard drive (created using a
> USB VM). I inserted the hard drive into a USB port owned by a USB VM,
> and verified that the drive was mounted and that the Qubes backup was
> visible to the USB VM. I then attempted to restore the backup from
> the Qubes VM Manager. When I gave it the path to the backup file, I
> got this error:
>
> > Premature end of archive while receiving backup header. Process
> > output:
>
> No other information was displayed. I was able to successfully
> restore the backup by connecting the hard drive to dom0, so I'm
> certain that the backup file was fine. (But obviously I'd prefer not
> to connect stuff to dom0.)
>
> The only possibly relevant info I can think of is that the path to the
> backup file had some spaces in it (in both the USB VM and dom0). I
> tried placing the entire thing in double-quote marks, which made no
> difference. I doubt this is the issue.
>
> Any idea what's wrong?

Actually I think having spaces in file name is an issue. Or rather there
is issue with handling such paths in restoration code.
Take a look at /etc/qubes-rpc/qubes.Restore in your USB VM, then change
the line:
/usr/lib/qubes/tar2qfile $TARGET $paths
to:
/usr/lib/qubes/tar2qfile "$TARGET" $paths

Yes, quotes over the first argument, but not the second.

I've also copied your report here:
https://github.com/QubesOS/qubes-issues/issues/1371

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBCAAGBQJWNrubAAoJENuP0xzK19csXU8IAIBOP59Uu+qgG505BtUF1TlM
LyF+pWezXqhhHa1RWALaOtufZq11JHFutvn1VAzKhPV7Jn9YN8DG+TrJrGtfutzq
bTxzSJqrMbNkH9jkFWxWm6MVORZdGsxqHuTHgyXX3hm7uWbsI9rxVGqWcDNTmEOl
oSQOGMpsrfvm3l5YinDQMjhtaIlVjhH/9P5BMCmJXIyZ5pe9xdEPNXaMbxAZWw8z
fQQOz1u2/SKcp7d1j/ROHNJNdV7DQIqlOx/yXbbrg/LyChfbN7jHqtXaEGYR/Njz
eYD5DYMMynzqKIX6RdFdTZCzPPocshfudlXa6iMSu/vLSYhMtIsU8rUwAvfBYtI=
=k9V3
-----END PGP SIGNATURE-----

[Jeremy Rand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Oh wow, yes, making that change does indeed fix the issue. Thank you
very much Marek!

- -Jeremy Rand
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJWNr36AAoJEAHN/EbZ1y06sLAP/Aq05jXC4sGBth8CxBGISMAV
p+NH3y6Lg1LvlnH2gyYNtbZTTe+58bbssQG44p0nVbtUG9wta/7mRoB7Vkh1XZcH
5QEtd2kC+o//JGHsMvLi1yZBQ26i4IewTy/OTKBM0RY1Z6hw8Z/+tA+PAOGUp2Ch
TLwQ/KnaxykwWJPk4y6wZvpY61/e5xHiz6VJeBADBKAlBnTrw5YMISwNtmMQG3C3
bPJfCLwz8CAqbx3z6gff+1kQGmaaXNnVOKpr2xxl0RTH/IHPGkwuWJ7RBC/M8usw
Q/V5lK2JvweHSxdR309nXpmpc94Rc45D2N5jpRRXXBMA9FL9toeRbRInaMmZ7/sa
Ac1MXqKkfzT4rTj6TdqbtiL2Vc2S1Z/Ep1dIDYm0lT6/3IgWUJdQixHVE9GI84ly
VWK5mCvBFvY8YQvZ/+fDyUxlhfjXFsUCf7bKU/VYybsnt9gbxf1SPdFVHcOukCiK
19cgLCUlaDO5FZdJgScfqWfQqUN0DezpdvRL3P5m5p5qViUhO29JhNXIZUNUKaxt
6kL1yrE+O572GxICnX5ZRu6TI5yj3t9YGlF23gmu4eSFXgt/tnAUs9RrJztmUEOj
RzdFusVzMkcBs5WawjWCEwNxBo8wTG+3ofuU5Re0S+mNa6gp+47+OUvHDujU+JyN
IGH6yvSO4zoQ7Oof8Ea2
=cdh6
-----END PGP SIGNATURE-----

[Drew White]

Jeremy, I am glad that yuo have got your question resolved and issue fixed.

But as for my post I was merely asking the questions and providing information.

All I was attempting to do was fin out the answers to simple questions as they have been issues before that I have experienced myself and had to develop a workaround.


You have misunderstood the information on the USB and NetVM. I was merely stating the issue that can occur.

I did read and reread and understood completely, but I merely required more information.

As for spaces in file names, why anyone with half a brain would do such a thing is unknown to me. Spaces in filenames should NEVER be used anyway, because it's only an issue creator.

[Jeremy Rand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 11/01/2015 08:19 PM, Drew White wrote:
> Jeremy, I am glad that yuo have got your question resolved and
> issue
fixed.
>
> But as for my post I was merely asking the questions and providing
> information.
>
> All I was attempting to do was fin out the answers to simple
> questions as they have been issues before that I have experienced
> myself and had to develop a workaround.
>
>
> You have misunderstood the information on the USB and NetVM. I was
> merely stating the issue that can occur.
>
> I did read and reread and understood completely, but I merely
> required more information.

None of the information you requested or provided had any relevance,
nor was remotely likely to have any relevance given the information
that I had already provided. Please remember that some people who read
this mailing list do not have infinite time, and consider accordingly
whether your messages are likely to be useful. Given that fact, I
won't be engaging you further on this and risk cluttering people's
inboxes.

> As for spaces in file names, why anyone with half a brain would do
> such a thing is unknown to me. Spaces in filenames should NEVER be
> used anyway, because it's only an issue creator.

Tell it to the developers of the USB mass storage device mounting code
in Nautilus. It put the spaces in the path, not me. (I also like the
implication that I don't have half a brain, but I'll assume that
implication was unintentional.)

(Sorry everyone for the spam... carry on with whatever you were doing
and ignore this exchange, which is over from my end anyway.)

- -Jeremy

PS: I also like how upon request to not top-post, you instead
completely left out my message, which makes it even more difficult for
people to follow the context of your message. Not everyone reads every
message on a mailing list, nor should they have to.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJWNsu/AAoJEAHN/EbZ1y06x9gP/2chXpDVkB0DnWeOxB4un1T3
rDNEPpX0ftexoN8OZDROCbeRnyFWx4KCcgBsHCODDz6dHXo7ttctJsKqIFt2hp9+
G6QbUb21NzrypUpXT+c9DKaHfCHGm9ksS6akOpOOpaY5Vzlbd22b2eFCpQ+1wUtu
wbQXtZPc1om5QOkaFUWNbfJZCdrPahaDIB8SybH7A+DGSw0VHU0oJ3JXcAaxtZBf
KO4yFoB7xFDCNcih3XTTyxmviHYcn9hebNp6RfH/y0ZYiRr+17BHl2S0O0PMm3GS
eI1iG4mdHir0VCW5+FVtpJC1U1yP9gMPs+V3B/ug1Hq9dP/uNcgzuCJpchxk4yaR
LmPES9MwMgYmgcq2eWAFhBiUGbJV7P45OKfRWznODGDUEYXgc2ExtGD3WL+h6lV7
wKy/SSmecHtQJl48Mol+BKAcBdwRL5jzh6oa1kGfyCo4q7zDw22mG84NU/qTUEeE
bhfvLNH87ZHLnvKo5Bxne+6A3xH1gS+uZSteSlNmo54gr+yHGO9nA5kN2U9x09Rk
9sOvc6cwB6ZIaW1LvbpS9wQCXOEUeStrUtYVkSRuNt72pykg6o+1k25GG4Mkbhed
31yUowA5hJqCFAITb6M22JLnVJ50S6Ge/4s0CQWfdMSptMj9v8p9veA/5PJV0zu/
nDlQDcRLIy26a6wEOnlR
=q2e1
-----END PGP SIGNATURE-----

[Drew White]

None of the information you requested or provided had any relevance,
nor was remotely likely to have any relevance given the information
that I had already provided. Please remember that some people who read
this mailing list do not have infinite time, and consider accordingly
whether your messages are likely to be useful. Given that fact, I
won't be engaging you further on this and risk cluttering people's
inboxes.

It did have relevance. But you are not able to see that because it did not come to fruition.

 

> As for spaces in file names, why anyone with half a brain would do
> such a thing is unknown to me. Spaces in filenames should NEVER be
> used anyway, because it's only an issue creator.

Tell it to the developers of the USB mass storage device mounting code
in Nautilus. It put the spaces in the path, not me. (I also like the
implication that I don't have half a brain, but I'll assume that
implication was unintentional.)

 It's not Nautilus that is the issue. I use Thunara myself, because it has things that the altered Qubes version of Nautilus doesn't have. (yes, the Qubes version has less than the standard Nautilus, but Thunara remains unaltered as faaar as I can see functionality wise.)

I apologise for the implication. It's not something that was directed at you, it's just something that I am frustrated with that people do. Personally, I use underscores to replace spaces, just to prevent ANY issues with file names. I also keep them short, not long. This provides remedy for all things. No matter the system, it will work.

Spaces and many dots are the bane of all systems, because "end users" started using them and doing all this stuff because they didn't think about the impact that it would have on the developers of these things. To have to take all this into consideration.

But such as life. I've spoken to many people about that issue and some related to it. Most of the people I spoke to completely agree with me on that subject.

anyhow. Your issue is resolved. If further discussion is required, it will be in another thread, or else on IRC in the challens that I am constantly spreading the word of the downfall of all this to get people to realise that spaces and periods and "special characters" should not be used.

[yaqu]

On Sun, 1 Nov 2015 19:10:15 -0800 (PST), Drew White
<mrdrew...@gmail.com> wrote:

> Spaces and many dots are the bane of all systems, because "end users"
> started using them and doing all this stuff because they didn't think
> about the impact that it would have on the developers of these
> things. To have to take all this into consideration.

Spaces, dots, unicode characters and even asterisk and question
mark (ok, not in windows) are perfectly legal in filenames. Please, do
not blame users for using them. It's a silly excuse for being unable
to work with valid filenames or for not sanitizing user input.

--
yaqu