What happened to "paranoid mode"?

[tetra...@danwin1210.me]

From back in the 3.2 era:

https://www.qubes-os.org/news/2017/04/26/qubes-compromise-recovery/
$ qvm-backup-restore --paranoid-mode

On my 4.0 install this option does not appear. Is it no longer
considered necessary?

[awokd]

tetrahedra via qubes-users:

Looks like it is coming back:
https://github.com/QubesOS/qubes-issues/issues/5310.

--
- don't top post
Mailing list etiquette:
- trim quoted reply to only relevant portions
- when possible, copy and paste text instead of screenshots

[Anil Eklavya]

A better way to backup and restore will make Qubes much more usable, preferably at some point using something like rsync, as they have already been considering. This is one crucial feature that makes it difficult for common users to use this OS as a default.

> On 04-Jan-2020, at 3:08 PM, 'awokd' via qubes-users <qubes...@googlegroups.com> wrote:
>
> tetrahedra via qubes-users:

> --
> You received this message because you are subscribed to the Google Groups "qubes-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/8255dbcc-7e09-eb18-cce3-17aa54dcfa82%40danwin1210.me.

[dhorf-hfre...@hashmail.org]

On Sat, Jan 04, 2020 at 03:56:49PM +0530, Anil Eklavya wrote:
> A better way to backup and restore will make Qubes much more usable,
> preferably at some point using something like rsync, as they have
> already been considering. This is one crucial feature that makes it
> difficult for common users to use this OS as a default.

idgi, why dont these people use whatever backup solution they prefer?

i have been using "borgbackup" to make automated, hourly, incremental
backups of my qubes machines since qubes3, even did the move
from qubes3 to qubes4 by "reinstall, restore" or use "backup on machine
x, restore on machine y" to move appvms or even templates around.

"restic" looks to have a very similar featureset and i heard good things
about it, might have used it if i had found it before borg.

in comparison, "qubes-backup" is about as useful as considering "tar"
to be the actual default backup solution of a plain linux.
dont get me wrong, i am not uninstalling tar or qubes-backup, but
i cant remember the last time i was desperate enough to actualy
use either for a backup.

three important points wrt backups in the real world:
- backups have to be automated + background, or they wont happen.
- backups have to be incremental, or they wont happen frequently.
- and unless you have a restore process you are familiar with and
use at least sometimes ...
... you dont have a (real) backup you should rely on.

[Anil Eklavya]

I wasn’t aware of these options. Thanks for pointing out. I will certainly try them out.

I agree with all your points about backup being practical and usable. The password is the weak link here’ as everywhere’ just like private keys. Perhaps something like YubiKey plus KeePass can solve this problem. Or someone may have a better suggestion.

I had once tried setting up YubiKey on Qubes, but there was some problem. I will get around to solve them as YubiKey is known to work with Qubes. I made the mistake of trying it out on MacOS Catalina, but all hell broke lose. I do use it on Windows, Google etc.

Regards,

Anil

> On 04-Jan-2020, at 4:41 PM, dhorf-hfre...@hashmail.org wrote:

[dhorf-hfre...@hashmail.org]

On Sat, Jan 04, 2020 at 05:05:01PM +0530, Anil Eklavya wrote:

(please dont TOFU)

> I wasn’t aware of these options. Thanks for pointing out. I will
> certainly try them out.

this is all "some assembly required" stuff, but i will try to describe
a working borg setup with some variations and try to explain some of
the thinking behind it.

there are some example scripts here:
https://github.com/xaki23/rzqubes/tree/master/borg

most of these are not commented, userfriendly or have proper
separation of "code" and "config", but otoh we are talking
about something you set up just once for each system.

the complex parts there are bsnap.sh (which is the hourly
cronjob that does the actual borg-snapping) and bsync.sh
(which is optionally called at the end of bsnap and does
the syncing of backup to external target(s), if desired).
the *wraps are just thin wrappers as crude ways to use
remote-capable tooling (here: borg and rsync) over qubes-rpc.

bsnap.sh has a bit of config at the beginning (lines 3-5),
storing a password like that is certainly not ideal, but otoh
doesnt matter (to me) since the script is inside dom0 which
already has access to all my data and if it is compromised
its pretty much gameover anyways.

lines 7-13 are leftover from qubes3 days (or for people using
qubes pools of type "file" with q4).

lines 15+16 are a sample of how to use the remote-wrapped variant.
basicly that means your dom0 still does all the reading, chunking,
encryption, but the actual storage backend process is running
on a remote host (or in a qubes appvm). this can be very useful
if you are backing up a stationary desktop to a bulk storage
host on the same lan.

lines 20-30 are three "backup groups". private volumes at rest,
unsynchronized private volumes of running vms, and dom0 as "files".
the FLP/FLS parts (lines 20+24) select which VMs are backed up
in that way, you can play around with the ls+grep on the commandline
until it matches whatever you want to back up. the examples there
are of the "everything, except what the grep throws away".

lines 21+25+29 delete old backup snapshots that are outside the
specified keep-range. 30/30/30/30 is _a_ _lot_ ...

lines 34-43 are the call to sync out the backup to external
storage, with crude locking. the locking (even when less crude)
is mainly a policy question. if you dont use locking for the sync,
and your sync takes longer than your backup frequency, you might
end up with the sync always just doing half a sync, never completing.
that can be very bad.
otoh, if you do locking, and the (locked) sync stalls out and you
dont have stale-lock detection or a timed hard limit, that stalled
sync job will block all newer sync attempts forever.
thats also very bad.

the called-under-lock bsync.sh tries to level the field (lines 4-8)
by killing/removing anything that might be leftover from older
syncs, creates a lvm snapshot of the local lvm backup volume,
attaches it to a sync-vm and runs a target-specific script inside
that vm.
this is just as setup-specific as it is modular.
doesnt matter to the backup at all whether the sync is done
over webdav, nfs, cifs or rsync-over-ssh.

the modularity isnt limited to the "sync" phase either.
want to make it respect the per-vm include_in_backup pref?
just add a filter stage based on qvm-ls/qvm-prefs early in bsnap.
want a more paranoid handling of the borg-pw?
use a detached borg repo header, with a different PW in dom0
than in your secret-stash masterkey backup repo, and no repo
header whatsoever in the remote bulk copy.


when restoring localy i tend to use "borg list" to see what snaps
are avail, then "borg list ::snap-12345" to see whats inside,
then "borg extract --sparse ::snap-12345 dev/whatevs".
these steps are done in dom0 when restoring from a local repo,
or in the sync-vm if restoring from remote repo.
the restored file is copied to the right blockdevice in dom0 by
"dd if=restored/blah of=/dev/mapper/something conv=sparse".
if the vm/volume used as a restore target isnt fresh/clean/unused,
make really, really sure to blkdiscard the target volume first.
(filesystems can get really upset if blocks they expect to be zeroed
are not)


there is a lot of flexibility and options in all this.
a lot of the options depends on your personal threat model and prefs.
this also means it is not suitable for users who dont know how
to delete a file without a mouse.
but those will need help with setting up qubes anyways.


feedback, questions, suggestions?
go ahead, either here, or on #qubes on freenode, or pullreqs ...

[Chris Laprise]

On 1/4/20 6:35 AM, Anil Eklavya wrote:
> I wasn’t aware of these options. Thanks for pointing out. I will certainly try them out.

I've also been working on an incremental backup tool that is more
efficient than borg, as it doesn't have to scan the entire volume before
backing it up.

https://github.com/tasket/sparsebak

It uses LVM metadata to instantly find where volumes have changed and
processes only those chunks. This is similar to the way Apple's Time
Machine used update information from its 'sparsebundle' volumes (giving
Time Machine a major speed boost).

Coding progress had slowed over the last few months, but I'm spending
more time on it again and hope to have a beta release this month.

--

Chris Laprise, tas...@posteo.net
https://github.com/tasket
https://twitter.com/ttaskett
PGP: BEE2 20C5 356E 764A 73EB 4AB3 1DC4 D106 F07F 1886

[Anil]

At that time I didn't notice, but what in the world is TOFU? I even
looked it up on Google, in Urban Dictionary, but still couldn't decide
in which sense it was being used and for what.

I have been exploring the backup options suggested by you and others.

Once reason 'those people' (I will still call them common people,
although they will actually be power users at least, if they have come
to this point in using OSs) may think twice about something like borg,
is that, in Qubes, they will be wary of installing (or putting)
something in dom0: something that can be executed. Perhaps that is
being overcautious, but borg, for example, requires installation of
several other dependencies.

There aren't just two kinds of people: Developers/power-users and
those who don't know how to delete a file without using a mouse.
Unless I am extremely wrong, most Linux users will fall in somewhere
between these two extremes and may of them won't even be power users.
Linux is actually being used now and you can manage to do quite a lot
with the GUI, so becoming more and more like Windows in that sense.

I am sure you, of all people, know that, but my long term peeve
(expressed before on this forum) has been that this large section of
people in the middle is largely ignored. Even the documentation, as I
pointed out earlier, is either too technical or too noob-friedly,
although I understand that there aren't enough people to do
documentation and this is open source and free software. They - we -
have to go to something like reddit for finding clues.

That aside, if one does decide to install bog, it seems to be a pretty
good option.

--
अनिल एकलव्य
(Anil Eklavya)

[Sven Semmler]

On Tue, Feb 25, 2020 at 12:42:42AM +0530, Anil wrote:
> At that time I didn't notice, but what in the world is TOFU? I even
> looked it up on Google, in Urban Dictionary, but still couldn't decide
> in which sense it was being used and for what.

In top-posting style, the original message is included verbatim, with the reply above it. It is sometimes referred to by the acronym TOFU ("text over, fullquote under"). It has also been colloquially referred to as Jeopardy! reply style: as in the game show's signature clue/response format, the answers precede the question.

https://en.wikipedia.org/wiki/Posting_style

/Sven

[Eva Star]

Don't forget about instructions for dummies how to use it with Qubes, please!!!

[Chris Laprise]

On 3/1/20 4:00 PM, Eva Star wrote:
> Don't forget about instructions for dummies how to use it with Qubes,
> please!!!

A Howto doc for backing up Qubes with Wyng is on the way. Here is a
Linux Howto referencing the new name & URL:

https://github.com/tasket/wyng-backup/wiki/Making-incredibly-fast-LVM-backups-with-Wyng