Backup/Restore doesn't work
180 views
Skip to first unread message
Pete Howell
Nov 6, 2015, 12:52:48 AM11/6/15
to qubes-users
I rebuilt my system and the backup/restore did not work correctly -- at all. I had errors in the restore process and tried two different backups that I had made. If I hadn't copied the files in /var/lib/qubes as a precaution, I would have had to start from scratch.
I have to say that this process is very important and should work correctly. The fedora-21 template wouldn't restore because of the existing template and the only way to remove this template, was to remove the package using "yum remove"; not really what I wanted to do. The restore of the fedora-21 template created a fedora-21/fedora-21 folder, and luckily that sub-folder had the files. The AppVMs that failed, all failed with missing root.img errors and in the restored folder was personal/personal and work/work, but no root.img in those sub-folders.
In any case, I ended up with numerous AppVMs missing and sys-firewall was gone as well. The system turned the torvm proxy into the default network and didn't even run sys-net on startup. So, I've slowly recreated templates and missing AppVMs from the filesystem backup I made, but I'm now frustrated with the defaults. My fedora-21-net template is now the default (not fedora-21, which has been recreated), and I'd like to change the default NetVM, but I'm not sure where to do that.
How does one change the default template and netvm?
I have to say that this process is very important and should work correctly. The fedora-21 template wouldn't restore because of the existing template and the only way to remove this template, was to remove the package using "yum remove"; not really what I wanted to do. The restore of the fedora-21 template created a fedora-21/fedora-21 folder, and luckily that sub-folder had the files. The AppVMs that failed, all failed with missing root.img errors and in the restored folder was personal/personal and work/work, but no root.img in those sub-folders.
In any case, I ended up with numerous AppVMs missing and sys-firewall was gone as well. The system turned the torvm proxy into the default network and didn't even run sys-net on startup. So, I've slowly recreated templates and missing AppVMs from the filesystem backup I made, but I'm now frustrated with the defaults. My fedora-21-net template is now the default (not fedora-21, which has been recreated), and I'd like to change the default NetVM, but I'm not sure where to do that.
How does one change the default template and netvm?
Marek Marczykowski-Górecki
Nov 6, 2015, 1:28:30 PM11/6/15
to Pete Howell, qubes-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
On Thu, Nov 05, 2015 at 09:52:48PM -0800, Pete Howell wrote:
> I rebuilt my system and the backup/restore did not work correctly -- at
> all. I had errors in the restore process and tried two different backups
> that I had made. If I hadn't copied the files in /var/lib/qubes as a
> precaution, I would have had to start from scratch.
>
> I have to say that this process is very important and should work
> correctly. The fedora-21 template wouldn't restore because of the existing
> template and the only way to remove this template, was to remove the
> package using "yum remove"; not really what I wanted to do.
Generally good practice is to make any non-trivial modifications only in
custom template(s) (for example clone of the default one). Exactly for
that reason - overriding default template when restoring the backup
isn't easy at least.
If you want to only install few packages there, that's ok, but you'll
need to redo that after restoring the backup. Personally I keep a text
file with a list of additional packages installed, to ease this task.
> The restore of
> the fedora-21 template created a fedora-21/fedora-21 folder, and luckily
> that sub-folder had the files. The AppVMs that failed, all failed with
> missing root.img errors and in the restored folder was personal/personal
> and work/work, but no root.img in those sub-folders.
First of all, AppVMs don't have own root.img, only private.img. But
directory layout indeed looks strange. Can you provide some more details
on the situation?
1. What Qubes version was used to create the backup?
2. What was the settings (compression, encryption, using UsbVM or not?)
3. What Qubes version was used to restore the backup, did you installed
updated before restoring the backup?
4. Anything unusual? Some additional error messages?
> In any case, I ended up with numerous AppVMs missing and sys-firewall was
> gone as well. The system turned the torvm proxy into the default network
> and didn't even run sys-net on startup. So, I've slowly recreated
> templates and missing AppVMs from the filesystem backup I made, but I'm now
> frustrated with the defaults. My fedora-21-net template is now the default
> (not fedora-21, which has been recreated), and I'd like to change the
> default NetVM, but I'm not sure where to do that.
>
> How does one change the default template and netvm?
In Qubes manager, go to System menu, then Global settings.
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAEBCAAGBQJWPPFHAAoJENuP0xzK19csfr0H/3xJ/+1MBWdB1ASCv/FuoXNx
FMXUn6Ai3Lr5mjtqAmBKchF0xfMy2XV76gIVmcvYP6tUmgEz6ApS+/y6j+L8vpXk
sQ1TEDdJEPyjgUEAugd7eceQ7XVkvXchq8O9Zo/W20Gl13tXMjxP6ndw02+eOPEG
W10TIxYdSnuIGs4i11ASIlcrG1czmWUHfRxobk4K8AtGv1nKam2cpFDRYcqJal82
+MUNY9WUFJgD+5j7h3E1zBa1FrYe3aESiR1dV7qF4BR3LGwMFi0DyAvD/K+L3Ug5
jYBV64RtiGhInh9P2erjL2/vEQGOcDXcybHaMFajQ1NEkHRrA8dnPxjInuha+ug=
=huDD
-----END PGP SIGNATURE-----
Hash: SHA256
On Thu, Nov 05, 2015 at 09:52:48PM -0800, Pete Howell wrote:
> I rebuilt my system and the backup/restore did not work correctly -- at
> all. I had errors in the restore process and tried two different backups
> that I had made. If I hadn't copied the files in /var/lib/qubes as a
> precaution, I would have had to start from scratch.
>
> I have to say that this process is very important and should work
> correctly. The fedora-21 template wouldn't restore because of the existing
> template and the only way to remove this template, was to remove the
> package using "yum remove"; not really what I wanted to do.
custom template(s) (for example clone of the default one). Exactly for
that reason - overriding default template when restoring the backup
isn't easy at least.
If you want to only install few packages there, that's ok, but you'll
need to redo that after restoring the backup. Personally I keep a text
file with a list of additional packages installed, to ease this task.
> The restore of
> the fedora-21 template created a fedora-21/fedora-21 folder, and luckily
> that sub-folder had the files. The AppVMs that failed, all failed with
> missing root.img errors and in the restored folder was personal/personal
> and work/work, but no root.img in those sub-folders.
directory layout indeed looks strange. Can you provide some more details
on the situation?
1. What Qubes version was used to create the backup?
2. What was the settings (compression, encryption, using UsbVM or not?)
3. What Qubes version was used to restore the backup, did you installed
updated before restoring the backup?
4. Anything unusual? Some additional error messages?
> In any case, I ended up with numerous AppVMs missing and sys-firewall was
> gone as well. The system turned the torvm proxy into the default network
> and didn't even run sys-net on startup. So, I've slowly recreated
> templates and missing AppVMs from the filesystem backup I made, but I'm now
> frustrated with the defaults. My fedora-21-net template is now the default
> (not fedora-21, which has been recreated), and I'd like to change the
> default NetVM, but I'm not sure where to do that.
>
> How does one change the default template and netvm?
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAEBCAAGBQJWPPFHAAoJENuP0xzK19csfr0H/3xJ/+1MBWdB1ASCv/FuoXNx
FMXUn6Ai3Lr5mjtqAmBKchF0xfMy2XV76gIVmcvYP6tUmgEz6ApS+/y6j+L8vpXk
sQ1TEDdJEPyjgUEAugd7eceQ7XVkvXchq8O9Zo/W20Gl13tXMjxP6ndw02+eOPEG
W10TIxYdSnuIGs4i11ASIlcrG1czmWUHfRxobk4K8AtGv1nKam2cpFDRYcqJal82
+MUNY9WUFJgD+5j7h3E1zBa1FrYe3aESiR1dV7qF4BR3LGwMFi0DyAvD/K+L3Ug5
jYBV64RtiGhInh9P2erjL2/vEQGOcDXcybHaMFajQ1NEkHRrA8dnPxjInuha+ug=
=huDD
-----END PGP SIGNATURE-----
Pete Howell
Nov 6, 2015, 2:45:59 PM11/6/15
to qubes-users, w.peter...@gmail.com
I backed up and restored on the same version -- R3.0 to R3.0. No encryption options were selected for this as it wasn't for archival purposes, only transitioning to a new system. The restore system said that there were errors and root.img could not be restored, but as you say, there should not have been a root.img file; however, the restore process thought that there was one. All the AppVMs that had the root.img error in the restore did not restore and had to be recreated with qvm-create and then files were copied into the new VM, because they did not exist.
The fedora-21 template image also had restore issues and the actual files for the template ended up in /var/lib/qubes/vm-templates/fedora-21/fedora-21 and had to be relocated one folder up, so there is a definite restore problem somewhere.
As to the fedora-21 template, I would suggest that the backup process warn you that it won't be able to be restored, or the restore process should be fixed so that it can be restored into the existing fedora-21 template on the new system. It's not obvious that it won't be restorable without removing the RPM and, as you said, it's problematic and shouldn't be attempted.
One additional thing is that the restore does not follow symbolic links, so .img files that were actually on different storage devices were not included in the backup. A warning message during the backup process, or just following the link and backing up the file by following the symbolic link would be nice.
The fedora-21 template image also had restore issues and the actual files for the template ended up in /var/lib/qubes/vm-templates/fedora-21/fedora-21 and had to be relocated one folder up, so there is a definite restore problem somewhere.
As to the fedora-21 template, I would suggest that the backup process warn you that it won't be able to be restored, or the restore process should be fixed so that it can be restored into the existing fedora-21 template on the new system. It's not obvious that it won't be restorable without removing the RPM and, as you said, it's problematic and shouldn't be attempted.
One additional thing is that the restore does not follow symbolic links, so .img files that were actually on different storage devices were not included in the backup. A warning message during the backup process, or just following the link and backing up the file by following the symbolic link would be nice.
Axon
Nov 6, 2015, 3:00:52 PM11/6/15
to Pete Howell, qubes-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Pete Howell:
be able to back them up, one options is to symlink the whole
directory, as explained here:
https://www.qubes-os.org/doc/secondary-storage/
-----BEGIN PGP SIGNATURE-----
iQIcBAEBCgAGBQJWPQbUAAoJEJh4Btx1RPV8ui4QAO8ESk4A4HqVYWdvkxJOWmlo
5uJ9cDEWg8YpKnGfHierJ/xMoqsdp3m/GOrkFM548/UMslsaVu8g5a/nX01F8Ey2
9vVr6h0b3NVLHW+dj3Sa8gz5E6ekRWOFVGtaE4V6+yy8pshNZADJ8Rn0mQzABp9l
X/WST2jBiH0/VqIIrSRak7GDgHPIvaNHTWiZXoYP855lq7aI+l21kY86JqPCXmEj
MOgtGrH+oZDT10FX7nF3SnctEuxg3SJxdgYglEtfLWRPUVseke+OPSEC4h4kur8W
Js1fmz85dBD/XRN4dkFfuPZfsF9QdeTN10DN5XeuPmOHfLWtOpfe41KoQ6J15T7P
wGMw1cwO08Sq5qMeHe4yvctzgUS0ketIzhJXjGiLowExAIg3Q8MAmGrX19PdnqNs
Fh72baMoiXsjOqQ9bKgNCBTaPtxGScHtTwExCp5P4WIarUVcf3X+mcNZTf0n7ODr
+qHjpXWiER4GABsNqscCe4qMmI519XQLBma0vwtzhg+FexZs/Of+Bto08hgAqII/
H6uANHnuZtDf27E2GSd5ACedkNsfMiPNm6orcGudIFwBOkuSyp61WUHOxUJVhp7R
hbQg2mQWYLeDza1TG0iRxXRSxxzaWDXzjCh7puTbjm/j4ckKUJv4VfHw9yc42jUR
eNT/d+BH0QvNO0+26HRd
=2PT0
-----END PGP SIGNATURE-----
Hash: SHA512
Pete Howell:
> I backed up and restored on the same version -- R3.0 to R3.0. No
> encryption options were selected for this as it wasn't for archival
> purposes, only transitioning to a new system. The restore system
> said that there were errors and root.img could not be restored, but
> as you say, there should not have been a root.img file; however,
> the restore process thought that there was one. All the AppVMs
> that had the root.img error in the restore did not restore and had
> to be recreated with qvm-create and then files were copied into the
> new VM, because they did not exist.
>
> The fedora-21 template image also had restore issues and the actual
> files for the template ended up in
> /var/lib/qubes/vm-templates/fedora-21/fedora-21 and had to be
> relocated one folder up, so there is a definite restore problem
> somewhere.
>
> As to the fedora-21 template, I would suggest that the backup
> process warn you that it won't be able to be restored, or the
> restore process should be fixed so that it can be restored into the
> existing fedora-21 template on the new system. It's not obvious
> that it won't be restorable without removing the RPM and, as you
> said, it's problematic and shouldn't be attempted.
>
> One additional thing is that the restore does not follow symbolic
> links, so .img files that were actually on different storage
> devices were not included in the backup.
If you'd like to be able to store VMs on a secondary drive but still
> encryption options were selected for this as it wasn't for archival
> purposes, only transitioning to a new system. The restore system
> said that there were errors and root.img could not be restored, but
> as you say, there should not have been a root.img file; however,
> the restore process thought that there was one. All the AppVMs
> that had the root.img error in the restore did not restore and had
> to be recreated with qvm-create and then files were copied into the
> new VM, because they did not exist.
>
> The fedora-21 template image also had restore issues and the actual
> files for the template ended up in
> /var/lib/qubes/vm-templates/fedora-21/fedora-21 and had to be
> relocated one folder up, so there is a definite restore problem
> somewhere.
>
> As to the fedora-21 template, I would suggest that the backup
> process warn you that it won't be able to be restored, or the
> restore process should be fixed so that it can be restored into the
> existing fedora-21 template on the new system. It's not obvious
> that it won't be restorable without removing the RPM and, as you
> said, it's problematic and shouldn't be attempted.
>
> One additional thing is that the restore does not follow symbolic
> links, so .img files that were actually on different storage
> devices were not included in the backup.
be able to back them up, one options is to symlink the whole
directory, as explained here:
https://www.qubes-os.org/doc/secondary-storage/
iQIcBAEBCgAGBQJWPQbUAAoJEJh4Btx1RPV8ui4QAO8ESk4A4HqVYWdvkxJOWmlo
5uJ9cDEWg8YpKnGfHierJ/xMoqsdp3m/GOrkFM548/UMslsaVu8g5a/nX01F8Ey2
9vVr6h0b3NVLHW+dj3Sa8gz5E6ekRWOFVGtaE4V6+yy8pshNZADJ8Rn0mQzABp9l
X/WST2jBiH0/VqIIrSRak7GDgHPIvaNHTWiZXoYP855lq7aI+l21kY86JqPCXmEj
MOgtGrH+oZDT10FX7nF3SnctEuxg3SJxdgYglEtfLWRPUVseke+OPSEC4h4kur8W
Js1fmz85dBD/XRN4dkFfuPZfsF9QdeTN10DN5XeuPmOHfLWtOpfe41KoQ6J15T7P
wGMw1cwO08Sq5qMeHe4yvctzgUS0ketIzhJXjGiLowExAIg3Q8MAmGrX19PdnqNs
Fh72baMoiXsjOqQ9bKgNCBTaPtxGScHtTwExCp5P4WIarUVcf3X+mcNZTf0n7ODr
+qHjpXWiER4GABsNqscCe4qMmI519XQLBma0vwtzhg+FexZs/Of+Bto08hgAqII/
H6uANHnuZtDf27E2GSd5ACedkNsfMiPNm6orcGudIFwBOkuSyp61WUHOxUJVhp7R
hbQg2mQWYLeDza1TG0iRxXRSxxzaWDXzjCh7puTbjm/j4ckKUJv4VfHw9yc42jUR
eNT/d+BH0QvNO0+26HRd
=2PT0
-----END PGP SIGNATURE-----
Pete Howell
Nov 6, 2015, 3:59:02 PM11/6/15
to qubes-users, w.peter...@gmail.com, ax...@openmailbox.org
I have the root.img running off ssd storage and private running off a larger and slower disk, so I can't symlink the entire directory.
Marek Marczykowski-Górecki
Nov 6, 2015, 4:05:15 PM11/6/15
to Pete Howell, qubes-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hash: SHA256
On Fri, Nov 06, 2015 at 11:45:58AM -0800, Pete Howell wrote:
> I backed up and restored on the same version -- R3.0 to R3.0. No
> encryption options were selected for this as it wasn't for archival
> purposes, only transitioning to a new system. The restore system said that
> there were errors and root.img could not be restored, but as you say, there
> should not have been a root.img file; however, the restore process thought
> that there was one. All the AppVMs that had the root.img error in the
> restore did not restore and had to be recreated with qvm-create and then
> files were copied into the new VM, because they did not exist.
>
> The fedora-21 template image also had restore issues and the actual files
> for the template ended up in
> /var/lib/qubes/vm-templates/fedora-21/fedora-21 and had to be relocated one
> folder up, so there is a definite restore problem somewhere.
I think the problem is here, in that directory structure. And when
> I backed up and restored on the same version -- R3.0 to R3.0. No
> encryption options were selected for this as it wasn't for archival
> purposes, only transitioning to a new system. The restore system said that
> there were errors and root.img could not be restored, but as you say, there
> should not have been a root.img file; however, the restore process thought
> that there was one. All the AppVMs that had the root.img error in the
> restore did not restore and had to be recreated with qvm-create and then
> files were copied into the new VM, because they did not exist.
>
> The fedora-21 template image also had restore issues and the actual files
> for the template ended up in
> /var/lib/qubes/vm-templates/fedora-21/fedora-21 and had to be relocated one
> folder up, so there is a definite restore problem somewhere.
template restored that way (so effectively restore failed), all the VMs
based on that template also failed to restore.
I think this was because directory /var/lib/qubes/vm-templates/fedora-21
already was there. If you'd remove/move it before restoring (or not
exclude 'fedora-21' from restoring), it would just work. Anyway this is
clearly a bug.
https://github.com/QubesOS/qubes-issues/issues/1386
> As to the fedora-21 template, I would suggest that the backup process warn
> you that it won't be able to be restored, or the restore process should be
> fixed so that it can be restored into the existing fedora-21 template on
> the new system. It's not obvious that it won't be restorable without
> removing the RPM and, as you said, it's problematic and shouldn't be
> attempted.
useful.
https://github.com/QubesOS/qubes-issues/issues/1385
> One additional thing is that the restore does not follow symbolic links, so
> .img files that were actually on different storage devices were not
> included in the backup. A warning message during the backup process, or
> just following the link and backing up the file by following the symbolic
> link would be nice.
https://github.com/QubesOS/qubes-issues/issues/1384
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
6pNmcO+a3WwIRGkBSenjuskXtk4MFmcJ3Jt87WUJ8dFO3ZyO54cW7awsGk4ULRyT
9hvE3rt0rFSHMzhE27DTVEc/gfvIgUir7PzLi69M91q5d2CMB4WzQgOTkavvxnYQ
n5B+eMFTGc4kJDQjsyWHCkCQpqu9iCwHEu5jiF5jPpJQxjbwzCsf7K/e35Sp7T8r
IdxBmCtQGKW1s9iH7yqjUG5Em7UsH3WJ6ufeURTrpFfiBIFCi0FA9tf+jZfXCuoJ
tHcI4wJQiLQaZMJ5ukpJbMjPFWGGLhwhnVrfi/gJySkzGBL0O6B3beo8lkGXxzs=
=+t1t
-----END PGP SIGNATURE-----
Axon
Nov 6, 2015, 7:39:31 PM11/6/15
to Pete Howell, qubes-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Pete Howell:
Hash: SHA512
Pete Howell:
> I have the root.img running off ssd storage and private running off
> a larger and slower disk, so I can't symlink the entire directory.
>
If it's a TemplateBasedVM, you can, since as Marek mentioned,
> a larger and slower disk, so I can't symlink the entire directory.
>
TemplateBasedVMs don't have their own root.img.
(PS - Please don't top post.)
zqkKjn32eUqb59+rRnoHOHGzEbI4j1PXWzkErBD5DXp0V7O9jwME7/tL6/eON3zV
ZgbcGFPU1UamcworJCuku9oHQN5Rw24B2MsGMTB89bXYY6QogStpbourdgvpX2bM
+eKyotk5VqWY5g/jAA0xKGTm76UySrEIliThpG1KD/jLJ69fpQKd74WaFOuyao2P
37VEl8dpGzcNy4Dm1/48YPVPdtBNuQGQxMsIylGSE6330B6gvszMWIEhKvFlMwwC
QKos2dwGlw1ImBaqdkBv2fjuTlGO4YTbFNRlLCfRbhg4U05yJXyiia/+M6IwyQOi
FDnFI4FuZwmJxeTQqBi8k0G9Av8eV8f1VkARmxebLXXJAroWrXeAjQMLLZhWF//m
jJuaLkwuuJoTFIN/oyQOGHsvbDGmLNkAylyIHj8paIezGEzRF/OLx5DitXsZmj71
kO9OlCAaG8AcM7nMpLDOdz2bkWg86EFR4ptrSXYgzYdzkd2/z7u96MaclHAFRsYT
6+rZ1MTzS3ihKGacoyYiL2SbAYUrBi/HKbaRkmEY/7KH53r4Ukl42d4XGpeTUMZi
UoQpMir5ZORSMkZhOEVWc2DIUOL38nhJmDuGqh5uC6jU9H2L4HDehgs4lNF0M61Z
gay4uJXAOslC6DsEEP3u
=EerQ
-----END PGP SIGNATURE-----
Reply all
Reply to author
Forward
0 new messages