Disk image backup - dd / partclone / clonezilla?

[General]

My use case is a 500GB machine with a few dozen frequently changing VMs.
Qubes backup takes more than four hours. In a restore scenario, I'd have
to maintain Qubes install media, run an install, then run a restore.
Certain Dom0 settings such as the split gpg configs etc would not be
preserved.

I have a fast SSD in a SATA>USB3 external enclosure. I'd like to

1) make a drive image from the host to the external enclosure regularly; and

2) be able to plug the external drive into a spare host to boot into my
system immediately in the case of a hardware failure on my primary host


I haven't got dd or CloneZilla/ partclone to work in this way, what do I
need to be aware of?

[General]

After doing a dd of the partition table and a clonezilla copy of /boot
and / volumes I receive, after Starting Plymouth Boot Screen

EXT-fs error (device dm-4) __ext4_get_inode-loc:4700 ... : cp,, (lightdm)...

ext4_journal_check_start:61 Detected aborted journal


and the system hangs,

[Chris Laprise]

On 2/11/20 2:53 PM, General wrote:
> My use case is a 500GB machine with a few dozen frequently changing VMs.
> Qubes backup takes more than four hours. In a restore scenario, I'd have
> to maintain Qubes install media, run an install, then run a restore.
> Certain Dom0 settings such as the split gpg configs etc would not be
> preserved.
>
> I have a fast SSD in a SATA>USB3 external enclosure. I'd like to
>
> 1) make a drive image from the host to the external enclosure regularly; and
>
> 2) be able to plug the external drive into a spare host to boot into my
> system immediately in the case of a hardware failure on my primary host

Moving boot partitions to other drives and getting them to boot can be
dicey. You have to know where drive paths and UUIDs are specified in the
bootloader and configs for various storage layers like encryption. You
also have to configure Qubes not to disable USB at boot time if your
boot drive is USB.

However, it might not be too hard to do, since the relevant paths and
IDs are standard for Linux... existing guides for cloning Linux LVM
systems should work. The rest of the volume-oriented stuff in Qubes is
based on named volume groups and volume names within the LVM config,
which makes handling all the post-boot storage details quite easy (just
keep the vg/lv names the same).

>
>
> I haven't got dd or CloneZilla/ partclone to work in this way, what do I
> need to be aware of?

If backup speed is the main issue, take a look at Wyng:

https://github.com/tasket/wyng-backup

Its made to backup LVM storage (what Qubes uses by default) and can find
changes instantly, which makes it possible to update a set of large
volumes in under a minute. It creates archives not bootable drives
(there is an issue open for a differential mode to update cloned
partitions, which will probably make it into a future release).

--
Chris Laprise, tas...@posteo.net
https://github.com/tasket
https://twitter.com/ttaskett
PGP: BEE2 20C5 356E 764A 73EB 4AB3 1DC4 D106 F07F 1886

[General]

Procedure to restore from Wyng backup to a new install of Qubes.

These are the steps I followed.


1. Install Qubes 4.0.3 from ISO image.

2. In a disposable qube, download Chris' public key and git
verify-commit the wyng repo.

3. Copy wyng to Dom0 using qvm-run --pass-io and cat

4. Create a "backup" qube and mount the external drive containing the
wyng backup

5. Restore the images as follows:

a) list all the folders in the wyng repo

b) import each using --from and renaming to restored-${VMNAME}

c) for each lvrename the original volume (if present) to ${VMNAME}-old

d) lvrename each restored-${VMNAME} to ${VMNAME}

6. Restore the root snapshot to root-restored

7. Edit the xen.cfg in /boot replacing /dev/qubes_dom0/root with
/dev/qubes_dom0/root-restored

8. Umount everything and reboot


Unfortunately this did not lead to a working Qubes system for me. On
reboot, I enter the luks passphrase and the system does not load beyond
some Dracut messages. Perhaps it is wrong to use the root volume from a
different hardware configuration?

If so, what would be the Qubes/wyng way to restore a full system backup
to a new host?

[General]

Many thanks for your helpful reply Chris.

> > You also have to configure Qubes not to disable USB at boot time if
> your boot drive is USB.

Ah, this is why it was not working for me! I shall try installing the OS
directly onto the USB drive and then make the clones from there. It
occurs that the USB isolation in Qubes is valuable and therefore running
a USB boot drive may not be sensible.

> If backup speed is the main issue, take a look at Wyng

I have installed this in Dom0 and made a backup, also using your helpful
instruction:


file: /lib/systemd/system-shutdown/10_root_snapshot.shutdown
```
#!/bin/sh
/usr/sbin/lvremove --noudevsync --force -An qubes_dom0/root-autosnap || true
/usr/sbin/lvcreate --noudevsync --ignoremonitoring -An -kn -pr -s
qubes_dom0/root -n root-autosnap
```

to enable snapshots of the root volume.

Incremental backups are indeed very fast and this is what I was looking
for. I need to build another box to run a test restore procedure.
Perhaps this could be scripted now.

Thanks again.

[General]

Many thanks for your helpful reply Chris.

You also have to configure Qubes not to disable USB at boot time if

your boot drive is USB. 

Ah, this is why it was not working for me! I shall try installing the OS
directly onto the USB drive and then make the clones from there. It
occurs that the USB isolation in Qubes is valuable and therefore running
a USB boot drive may not be sensible.

If backup speed is the main issue, take a look at Wyng

[Chris Laprise]

Glad I could help!

FWIW, the root-autosnap is not strictly necessary since Linux does allow
snapshots to be taken of live LVs (e.g. you could point wyng to 'root'
and it would make its own snapshot of 'root' while it is live) but I
guess the above method is a bit safer.

Since Wyng doesn't yet handle Qubes config for individual VMs, restoring
a full Qubes system state with Wyng would require at least restoring
Qubes configuration (such as qubes.xml and firewall.xml) but that is
covered if you backup/restore the 'root' volume. OTOH, if your Qubes VM
settings are default or you made notes about them, then the xml files
aren't needed and you can restore the data volumes into freshly created
VMs (e.g. after reinstalling Qubes).