Need docs for qvm-backup-restore

[cprise]

> $ qvm-backup-restore --help
> Usage: qvm-backup-restore [options] <backup-dir> [vms-to-be-restored ...]



The above, plus the options list, is all I have for documentation of
this command. The wiki entry is empty and there are no examples of
usage. In particular, I can't figure out what "backup-dir" means. I
think the path to the backup file should go there, but...?

Also, I need to do a separate restore of dom0 without disturbing the
contents of the new dom0. But from reading the source code, I think that
is the opposite of what actually happens...?

[Jason M]

try: qvm-backup-restore --help.  You can select which appvms to restore.  I do not think any templates VMs ever get backed up even though they are selected to be in backup for some reason, at least not for me; I have a script that's backs those up as well as complete root.

The restore only restores your dom0 home directory and the files in /var/lib/qubes I believe; so move any appvms with same name out of the way (clone them; remove old copy I would think). A snapshot is a good idea; don't have any experience with LVM though.

qvm-backup-restore --encrypted /location/of/file

I use btrfs as my filesystem, so I just create a subdirectory `/.snapshots` then `btrfs su sn -r / /.snapshots/root-date-of-snap`, and in 1 second I have a complete snapshot of system I can restore to.

You could also consider to make a copy of the `/var/lib/qubes` directory of you can not do snapshot.

Good luck.

[cprise]

On 03/15/15 19:26, Jason M wrote:
>
>
> On Sunday, 15 March 2015 14:44:22 UTC-4, cprise wrote:
>
> > $ qvm-backup-restore --help
> > Usage: qvm-backup-restore [options] <backup-dir>
> [vms-to-be-restored ...]
>
>
>
> The above, plus the options list, is all I have for documentation of
> this command. The wiki entry is empty and there are no examples of
> usage. In particular, I can't figure out what "backup-dir" means. I
> think the path to the backup file should go there, but...?
>
> Also, I need to do a separate restore of dom0 without disturbing the
> contents of the new dom0. But from reading the source code, I think
> that
> is the opposite of what actually happens...?
>
>
> |
> try:qvm-backup-restore --help.Youcan selectwhich appvms to restore. I

> donotthink any templates VMsever getbacked up even though they are
> selected to be inbackup forsome reason,at least notforme;I have a script

> that's backs those up as well as complete root.
>
> The restore only restores your dom0 home directory and the files in
> /var/lib/qubes I believe; so move any appvms with same name out of the
> way (clone them; remove old copy I would think). A snapshot is a good

> idea; don't have any experience withLVM though.
>
> qvm-backup-restore --encrypted /location/of/file
>
> I usebtrfs asmyfilesystem,so I just create a subdirectory

> `/.snapshots`then`btrfs su sn -r /

> /.snapshots/root-date-of-snap`,andin1second I have a complete snapshot

> of system I can restore to.
>

> Youcould also consider to make a copy of the `/var/lib/qubes`directory

> of you can notdosnapshot.
>
> Goodluck.
>
>

Since I chose btrfs when I reinstalled, I could try this also. However,
the --help output appears to be incorrect or incomplete. <backup-dir>
isn't really expecting a directory... is it?

And there is no explanation how to point to a source file within a
backup vm.

I guess I'll just create a snapshot then use the GUI to restore.

[nrgaway]

I think when you back up, your choose a directly to save to, but when restoring you select the file and use the qvm-backup-restore script.  I do backups every few weeks and only restore a couple times a year and do mostly use the GUI except not lately since I been using Release 3 and the Qubes manager was not complete at the time.

btrfs su sn -r / /dir_of_snapshot/filename, or in full

btrfs subvolume snapshot -r (read-only) <source> (/ root dir) <destination>

The nice thing about btrfs is that the snapshots are instant and it doesn't not really take up any more space since the COW and only stores the differences.

Try to keep the disk usage to under 80% for best performance and at some point in future, I can give you more tips to optimize your experience with btrfs.

[cprise]

On 03/15/15 20:48, nrgaway wrote:
>
> On 15 March 2015 at 20:21, cprise <cpr...@gmail.com

> /.snapshots/root-date-of-snap`__,andin1second I have a complete

Thanks.

BTW, do you have a recipe for cloning a template using only a reflink
copy for the root image?

I'm thinking of doing a cp --reflink of the whole template dir, then
changing the templateXX.conf references inside the new template dir, and
finally doing a qvm-add-template.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

When you specify "-d VMNAME", the path to the backup (indeed file path
in place of <backup-dir>...) is relative to that VM.

Actually if you restore from a VM, you can use some command instead of
path to retrieve the backup for example from some NAS. Make sure that
command is quoted so it is a single parameter. Such command should
output the backup to its stdout. Example command would be "curl
https://my.backup.server/my-qubes.backup".

You're right, documentation of this tool needs some work...

> I guess I'll just create a snapshot then use the GUI to restore.
>
>

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJVBjyTAAoJENuP0xzK19csUFcH/AgbipKpVdmDBs7Xh9VJMb4d
MNtB/+K70uLmGp2XDnAfdmZ+LYLMH5KwJjL9aJAds49cPkkxKkqMVse5rUC5gFLE
sJK5XGoJnL0Tcog3Hl1EYU+roaHLIwy65Kc/HJYbrGx3phmtx3cwa+sOXEyIUiyJ
EEHbSgXNm8/BkussaEF0bmp+hmMNkyIV98IR/EquEBWCqVWawLLcIZ0mQTwc+Z0O
4mT6bimxdIAFm2K+f2Bitdg54taJawydXgMxOl4XKUHmMHquq5PFhB62r5JgIpNS
c6ADSIE2Pg8PcOmBH6AR9l3y6TrFoGa2hQj7Wdt0JEdNqne+d/bfOE9dh/EdhBM=
=yorT
-----END PGP SIGNATURE-----

[cprise]

Seems you've got the terms and descriptions used for backup in the
restore tool. I.e. "The Appvm to send backups to" instead of "The Appvm
to restore from". In place of <backup-dir>, <backup-file> or
<backup-path> would make more sense.

[Axon]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Jason M wrote:
>
>
> On Sunday, 15 March 2015 14:44:22 UTC-4, cprise wrote:
>>
>>> $ qvm-backup-restore --help Usage: qvm-backup-restore [options]
>>> <backup-dir> [vms-to-be-restored
>> ...]
>>
>>
>>
>> The above, plus the options list, is all I have for documentation
>> of this command. The wiki entry is empty and there are no
>> examples of usage. In particular, I can't figure out what
>> "backup-dir" means. I think the path to the backup file should go
>> there, but...?
>>
>> Also, I need to do a separate restore of dom0 without disturbing
>> the contents of the new dom0. But from reading the source code, I
>> think that is the opposite of what actually happens...?
>>

By default, restoring dom0's home dir from a backup should first move
the existing contents of dom0's home dir to a new appropriately-named
subdirectory.

Note there's also a "--skip-dom0-home" option, in case you want to
handle it separately.

>
> try: qvm-backup-restore --help. You can select which appvms to
> restore. I do not think any templates VMs ever get backed up even
> though they are selected to be in backup for some reason, at least
> not for me; I have a script that's backs those up as well as
> complete root.
>

This only applies to "system-managed" templates (e.g., fedora-20-x64
and fedora-20-x64-minimal). This is by design, since these templates
may periodically be updated in ways that might break user
customizations in the template. For this reason, users are advised to
clone these templates before making any changes.

> The restore only restores your dom0 home directory and the files in
> /var/lib/qubes I believe; so move any appvms with same name out of
> the way (clone them; remove old copy I would think).

Renaming the existing same-name AppVMs also works.

> A snapshot is a good idea; don't have any experience with LVM
> though.
>
> qvm-backup-restore --encrypted /location/of/file
>
> I use btrfs as my filesystem, so I just create a subdirectory
> `/.snapshots` then `btrfs su sn -r /
> /.snapshots/root-date-of-snap`, and in 1 second I have a complete
> snapshot of system I can restore to.
>
> You could also consider to make a copy of the `/var/lib/qubes`
> directory of you can not do snapshot.
>
> Good luck.
>

-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJVBpnlAAoJEJh4Btx1RPV8LAAQAJRyy45z8Kfe84r1Zax6y7Zm
GzYYeGtwGwXJoQcirtbc/Eu8+C+pn5RAJQduE4hKPiIkaqteDJAaPDHe6Ph0maPz
RndqW3ofEydu1cVq+SGrFzH2xSwuoBAsDChvDSfT3IAtBluFn0I5DOB1UcCiO5if
JJvdNZMzUf/6npHCH7Txbbn+2TzASXqXi5uB6TN/u6ZosgQg2Vp0wbqOjroT52Z2
UtkTEeUkB1X+itks/wDSNYLhjTw8P36uhpcm+El7NnCriBfjhDto4m/Nk+l+icVJ
KzowW1CpZp6gvB0XM9nxtaLU/KsfqCsTEvDd7+nwFaNeH5i9gJxSwO1iwGByvr/E
NsWn+rUBHuIuw6CzryzeSaM4OaDNCctr2xB3GXx2ydDCOOKtUzfFHNkhnggrBCnH
a5VUZrlW1lee9yFjLky7w248vZylr3863twqjbJm1qPWVry1Z1AXlw49fnEv5yhA
SFVComk3cqXeAmKMsZG2iegx1oD+fN50Ic8NDpmy0p3ouguwRIO/KZEMFpWM0yVu
Qj++B1yuC14WJNSwaDzO3Tx2zanCg2whxc+XvYQEhLtKVtVQni9a/FpR3hiDBXbC
FgDir3cyTxFEMUHkiLT0DTnY4H/pPMLWvSHCIoPPfgvAP08N5cUQTPbd8IOvjrNq
tu/0ECc3InHKDH7sf/qa
=GLnf
-----END PGP SIGNATURE-----

[nrgaway]

Do you know what the definition of 'system-managed' is?  Is there some setting in qvm-prefs or some other utility that would indicate the template is system managed.  I really think there should be a better indication for end users that certain templates will not be backed up, as within Qubes manager, I have them all selected to include in backup.

I have had non-system templates also not backup and it would be nice to know which ones will not.  Again, I think there should be a clear indication that a template will not back up in Qubes manager, so instead of having the option to 'include in backup', it could state something to the effect of 'This template is exempt from backup'.

Personally I wrote my own backup script I run before a Qubes backup that does a backup of everything in the /var/lib/qubes directory that does not get backed up, along with separate boot and root backups which are placed in dom0 home directory so they will be included with the system backups.  When doing a system backup, I do one for Dom0, one for a standalone backupVM that contains scripts to help restore and the remainder in the rest.

When restoring (complete restore), I restore the Backup VM, followed by dom0 so I can have my templates in place and restore my root partition to previous state.  Then finally the remainder.

[Axon]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

I guess the best definition is probably "any template obtained directly
via RPM." In other words, any template you did not clone yourself from
an existing template.

>> Is there some setting in qvm-prefs or some other utility that
>> would indicate the template is system managed.

Not that I'm aware of.

>> I really think there should be a better indication for end users
>> that certain templates will not be backed up, as within Qubes
>> manager, I have them all selected to include in backup.
>

It would probably make sense to disable the "backupable" attribute for
system-managed templates.

>> I have had non-system templates also not backup and it would be
>> nice to know which ones will not.

OK, so this sounds like something different. I've never had a
non-system-managed template fail to get included in one of my backups
when I selected it to be backed up. Was it a non-Fedora template? In
any case, you may want to check if there was an incorrect setting
somewhere.

>> Again, I think there should be a clear indication that a template
>> will not back up in Qubes manager, so instead of having the
>> option to 'include in backup', it could state something to the
>> effect of 'This template is exempt from backup'.
>

That sounds reasonable to me.

iQIcBAEBCgAGBQJVBtJmAAoJEJh4Btx1RPV8INUQAOSqt0x3/HGBFeFwZuSA5TC2
v/Ao7MsBHvMhD5DyKOKcbNRfSlZX3Bh9d7hxnSwnTtBIWtCUw6cTVaa02yBmws2z
vNRiMfiSlBKTt6A+68AWmRFeEL6NeCvcq8BNiJRzpS8qLgJiWvhDw/eWlykIyU7b
DFEsngeSnYaQtBgRCV5R6ss8/nU9ozByzNIhhLy1RpF8Bku4XB1bmBmAfQuNqoEW
7hzEGkgd67Ecx9N2AqPWruGJunqKIUDmn47pN4mbqhz8gyRotumqbrIQqHh3a+jC
YR/KBvFhfvBx+yibwOLc/eXrFN8KW2+PaQA4MYohxPrVu0Fob6MENNUV4pB9GwE+
KGYWXpHZrQwwLc2GCULq83PDMEdsd4uDDUrw04CW355ycOCOE3axLiHR7w+/VFfI
AoDaiwh0dN4hSSAFVZzRUOPsWIEErL8JYltU2UfSSeog7J2pZg56+6taqm4aqZQ8
MAz3Z/3wf9YFbXlMVRAioIWqi6GdQRx7xPUKAyKOT20ojUCw7ixKkK0vzz0fRwao
l/Qinsr0FnePLv/TglLWREThjIcEzUMDb8k6BfFBSEeC74xzixnm+SNCPMXRO6WQ
tNq4Inr2IvJQc2WQ8GkV/FaQ3e/iJF3NmyNWjtbt/lZ2cAQlS8vhPafEn4so6/FD
jj7tLMMPwe8BRFV9fRhD
=xcyo
-----END PGP SIGNATURE-----

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

There is one: qvm-prefs shows "installed_by_rpm" flag.

> >> I really think there should be a better indication for end users
> >> that certain templates will not be backed up, as within Qubes
> >> manager, I have them all selected to include in backup.
> >
>
> It would probably make sense to disable the "backupable" attribute for
> system-managed templates.

Actually you can backup "system-managed" templates when you select them
manually in GUI, or include in qvm-backup cmdline. They are not included
by default, that's all.
But restore of such templates would be tricky (assuming full reinstall)
- - you first need to remove original template, but this will probably
mean that you need to remove all the VMs (including the one used to
access the backup...), or clone that template first. So the procedure
would look somehow like:
1. clone default template
2. switch all the VMs to the cloned template
3. remove default template
4. restore data from the backup

This applies to full reinstall. You don't need such procedure to restore
just a single VM of course.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJVBuobAAoJENuP0xzK19csO+cH/3ZobDnpSc9ExPR7f1PlhCNz
cIR8EZgL6Enon4F+txw4J8yxPU3sMlxD9nM/fCpf8NQ8q9dLniQ9dJ1vB1JhAutr
tjL+3LTBfpwB0ZkQX9FYK1BY3nvLxmlGhNNU1oQF7SJLy3Y4zrOiDoO6273V5H9g
MVUO7OKjRFHl7s/tCpmf4zCio5wefthi/Jsm/yORSgWkZAbYElMuSeEDoKbArmIg
lKR8e3/h4cjI2JSJ/CTJu5BJh6UgYMtYMMVJ8Zh6FYFFRvUqkH/t66B723zYyY3U
Y8fMOvPU8Y0U9SV3/IAuh+iqbaMGldq1qKhwlVk7DWsUYaQpOAUGPVQ/eC52Hys=
=mWib
-----END PGP SIGNATURE-----

[nrgaway]

That explains it.  I install most of my template by RPM :)
 

> >> I really think there should be a better indication for end users
> >> that certain templates will not be backed up, as within Qubes
> >> manager, I have them all selected to include in backup.
> >
>
> It would probably make sense to disable the "backupable" attribute for
> system-managed templates.

Actually you can backup "system-managed" templates when you select them
manually in GUI, or include in qvm-backup cmdline. They are not included
by default, that's all.
But restore of such templates would be tricky (assuming full reinstall)
- - you first need to remove original template, but this will probably
mean that you need to remove all the VMs (including the one used to
access the backup...), or clone that template first. So the procedure
would look somehow like:
1. clone default template
2. switch all the VMs to the cloned template
3. remove default template
4. restore data from the backup

This applies to full reinstall. You don't need such procedure to restore
just a single VM of course.

This is the reason I have a separate standalone backupvm.  When I do a full restore the first thing I do is restore it, then remove everything else so it can be restored.

--
You received this message because you are subscribed to a topic in the Google Groups "qubes-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qubes-users/uQEUpv4THsY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qubes-users...@googlegroups.com.
To post to this group, send email to qubes...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/20150316143507.GU2321%40mail-itl.

For more options, visit https://groups.google.com/d/optout.

[Axon]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Ok, thanks.

>>>> I really think there should be a better indication for end
>>>> users that certain templates will not be backed up, as within
>>>> Qubes manager, I have them all selected to include in
>>>> backup.
>>>
>
>> It would probably make sense to disable the "backupable"
>> attribute for system-managed templates.
>
> Actually you can backup "system-managed" templates when you select
> them manually in GUI, or include in qvm-backup cmdline. They are
> not included by default, that's all.

Ah, I see. But is the "include in backups by default" flag respected
if the user checks that box in QVMM?

> But restore of such templates would be tricky (assuming full

> reinstall) - you first need to remove original template, but this

> will probably mean that you need to remove all the VMs (including
> the one used to access the backup...), or clone that template
> first. So the procedure would look somehow like: 1. clone default
> template 2. switch all the VMs to the cloned template 3. remove
> default template 4. restore data from the backup
>
> This applies to full reinstall. You don't need such procedure to
> restore just a single VM of course.
>

Or perhaps just choose the "don't preinstall any VMs" option when
reinstalling?

-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJVBu8vAAoJEJh4Btx1RPV8R9QQALAHvGajEhmpEkdHy/PzY1yU
Jmngbzbv3yEyWWPUAN7Ktj1f50pEe9p2RzzP9LdYJjgF8nspjW739sAQzRm1G+3X
7RfQ5QUOGDisV/5/F6YoUCPjKDoA95MqxTgRQEB16qL3RHFi+ZWl4COainob34eW
AE8kCoysnIGjdzMymQJMTmbq3ma08NwBV32c34DPs4072s2gwx2TnutTcNh5iZSU
zBQcVX4GBN+Wm2hgP8lXy3o8An4xmsLaQgyq7ir1e4zIg8+OJp5txG6Ocrq67cVE
SLlVBSEhiEBpQRK0S7g0n0zuDreKKJ8KQqFVUnsg6EUmcOU65oCncoMIqPdxY9OR
AFOw2T0vk0/qYbJAjT7i+ZjgTq8x6jdkxNCbEJxg3N1UivR7WocWLF6Qzb26VVBi
IzCzR2yRa1IiDSjWbd5Ke+jNoQJxhcF6CYByOg00gl6VkXDvH+GNq0bDVonw/AWI
CmDqLA0tB3TEpAQuOEV7RGLWWYbtk/ITnMuic/OkNHK0diZSOPedAHL++gG5rJGV
85/0sUB/aqZD5dZPgaYaxHTG9JwlQOhK/x3ifj7AXuIC+2rzM7tCSaVAsKUtcwvM
6uNn6y5rcDLs+EU7vWc0x6X6gVGxZMr4CpYMuFSLhHS+Wo6YEkDdE5xFR0w7kywj
E2rqRputGnh91oC7uaK3
=EiuZ
-----END PGP SIGNATURE-----

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

No. And I think this is a bug.

> > But restore of such templates would be tricky (assuming full
> > reinstall) - you first need to remove original template, but this
> > will probably mean that you need to remove all the VMs (including
> > the one used to access the backup...), or clone that template
> > first. So the procedure would look somehow like: 1. clone default
> > template 2. switch all the VMs to the cloned template 3. remove
> > default template 4. restore data from the backup
> >
> > This applies to full reinstall. You don't need such procedure to
> > restore just a single VM of course.
> >
>
> Or perhaps just choose the "don't preinstall any VMs" option when
> reinstalling?

But you'll probably still want some VM (UsbVM/NetVM) to retrieve the
backup from.
You can use nrgaway's approach to use a standalone VM for that.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJVBvUBAAoJENuP0xzK19csJ7AH/2c7miRr/DHYTGz0ObNGYKpx
esnCh/ypWB9ilJrytQSb39rV9uldhoj0zpkfWUZRDF2gTg668c2qkwVxfcwM4Ejj
t6KK5IlqUB6MuZy6MMZgp43u4QXTtptztSsf/rUQSSK9YWHKnlAdlZgW3Tl1cKhw
2z+Rg1dVF9cxTc6XLCZbFtqWkuEHXcXTgCSzmbIvhmEI/Pt+2TLt8a+zX635GyVz
zmFi7RGccXUlit6y/b1/v+TmjGTZNeT+iPjt1RWAdGCMtFKlg5hIhx19+Z+NROGv
qaJuavDcdRt9LsNrLPNwlfSDghr32uQ0lNbnQCLJTGQZOvsmEpY6+pqUahh5yo8=
=XiYm
-----END PGP SIGNATURE-----

[Axon]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Good point.

-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJVBviHAAoJEJh4Btx1RPV85aYP/20ivimDRU6qyT91Vy7/A3Q8
Cz5Iy+cPhQ5rI6ldh6TesG7r2YzWNH6Rmi0NOhlokpgyzNonqo4F01qqzFGN+xoU
LaUSXP9cmz0efp1GvouXz4btfIPn2JQAQj94gBkjL/BwjJ37s1DGNEN/VBkgC9UM
r0DlpPAvn2HuEo3o1mjOWzLHmWayYeIbUP303UH/hUiuDTpaPztGvTuqiKAPJP4y
RQEBgYhAdnGrjpGedRRVGWFeOsB2zwAprY9x3YgQM+VdckL4ru/clv32ira8Rou5
A6Ct0u60lI7lsL91tvY+q714BbiLzgT+f+TODZc/QPHU9aRKCmEsz8tyj0+BF3lG
y1Vd77URYd516bK7LmgfRGz0PPMsrINZvwoQTVeQKqVu5RSybZo9y+qHHIFbJmrM
9l4T8ZZQ1rNUD6Xr44lFAwPajAuZfHFVzxGuzsGcnjaA1yDLCK3Ndmuld/CkKI+n
A4CAB9FDPid2ztwQ5v9ajgp05icSx8/li3xu0iXLLxs9Vf6Q+HdhOWn4TfTX3obT
elN4E6Eky8HSb9JEtPjZXz3LaObTmAm3RNLDpbfPrGiDyGbkk7cdDC/HsDyvEnnX
EwD54tYqLZ46sMtv2IC3dcfx4Uuh4bpYHuzs8ZmH+Wf8JS7LxGNEL0og/QzyxOzo
Euc9wmKOiTD1mJqb36uw
=9NU5
-----END PGP SIGNATURE-----