backup of files in a qube without networking to an internet service

[lik...@gmx.de]

Hi,

assume there are files stored in a qube without networking. Furthermore assume there's a secured backup server located in the internet. This server is only a storage of client-side (before data is sent over the wire) encrypted files.  What options do you imagine to backup those files (skip the client-side encryption) to the server?

I can imagine the following options:
1. enable temporary the network with firewall restricted to the server for  the (previously offline) qube
     Advantage: no inter-vm copying of files.
    Disadvantage: firewall rules must be setup correctly to avoid to bypass any other traffic like icmp/dns etc. I can imaging a potential information leakage due to enabling network access.
2. copy files temporary to another qube (dvm?) with a firewalled internet connection
    Advantage: files not being backed up can stay secured in the non-network cube. Leakage of data is reduced in comparison to 1.
    Disadvantage: can take time and needs additional disk ressources

I've learned that you should always find at least 3 options, otherwise you haven't thought hard enough. Which options am I missing?

Which option would you prefer and why?

Best, Pete

[Chris Laprise]

Another disadvantage of #1 is that connecting the net to the source qube
exposes it to attack.

Had you thought about using qvm-backup? Also, I'm working on a fast
incremental backup tool that's suitable for Qubes:

https://github.com/tasket/sparsebak

--

Chris Laprise, tas...@posteo.net
https://github.com/tasket
https://twitter.com/ttaskett
PGP: BEE2 20C5 356E 764A 73EB 4AB3 1DC4 D106 F07F 1886

[unman]

3. Create encrypted (compressed) backup in offline qube.
qvm-copy backup to online disposableVM.
Copy encrypted file to backup server.

Advantage: All files secured in non-network qube.
Disadvantage: ???

Is inter-vm copying of files really an issue? Free space such an issue?
Using compressed backups should help mitigate this as a serious issue,
but that problem would extend to *all* your Qubes use.

unman

[lik...@gmx.de]

On 2/19/19 6:22 PM, Chris Laprise wrote:

> On 2/19/19 10:41 AM, liked2-Mm...@public.gmane.org wrote:
>> Hi,
>>
>> assume there are files stored in a qube without networking. Furthermore assume there's a secured backup server located in the internet. This server is only a storage of client-side (before data is sent over the wire) encrypted files.  What options do you imagine to backup those files (skip the client-side encryption) to the server?
>>
>> I can imagine the following options:
>> 1. enable temporary the network with firewall restricted to the server for  the (previously offline) qube
>>       Advantage: no inter-vm copying of files.
>>      Disadvantage: firewall rules must be setup correctly to avoid to bypass any other traffic like icmp/dns etc. I can imaging a potential information leakage due to enabling network access.
>> 2. copy files temporary to another qube (dvm?) with a firewalled internet connection
>>      Advantage: files not being backed up can stay secured in the non-network cube. Leakage of data is reduced in comparison to 1.
>>      Disadvantage: can take time and needs additional disk ressources
>>
>> I've learned that you should always find at least 3 options, otherwise you haven't thought hard enough. Which options am I missing?
>>
>> Which option would you prefer and why?
>
> Another disadvantage of #1 is that connecting the net to the source qube exposes it to attack.
>
> Had you thought about using qvm-backup? Also, I'm working on a fast incremental backup tool that's suitable for Qubes:
>
> https://github.com/tasket/sparsebak
>

I've checked qvm-backup. It's an appropriate solution if you don't care about disk space and bandwitdth of the network connection. Saving of a subset of files due to remote space and bandwidth resouces will not work well with qvm-backup.

Also incremental backup is not really possible with qvm-backup.

Regarding the solution you're working on: If I get it right, it's meant to be a disk->disk backup? What I'm looking for is an incremental client-side encrypted backup, similar to the tool duplicati. Also a poor man solution with git+rsync+veracrypt would be possible.
Can you imagine how sparsebak could be combined with truecrypt? Is there compression support?

[Stuart Perkins]

My backup routine is this.

Important files are kept in a LUKS encrypted container on specific appVM's, mounted when needed.

A similar LUKS encrypted container is maintained on my home network server. I mount the container on my home server over an ssh connection then use rsync.

Different appVM's use different paths within the container, so they all backup to the same home network based container...which is unmounted when not actively being used.

If I completely lost my current machine setup, it would only take a couple of hours to setup a new one...and quite some time to rsync all of the important files, but solid and repeatable.

My encrypted backup contains 30 years of e-mails as well as financial and client information. I also keep another backup of the same information on a completely different physical machine, which while not Qubes can actually run enough software against the encrypted container to continue with my necessary day-to-day tasks.

I clone my templates before major upgrades or just if there are questionable upgrades happening so if there is an issue I can set all of the machines which use the template to use the backup while I sort it out. I may not make a fresh backup of a template if the only upgrades are chrome or vivaldi or something else non-critical.

It is hard to beat rsync...just work a method which is appropriate for the different appVMs as if they were discrete machines...which they are from a logical standpoint.

Stuart

[Chris Laprise]

On 2/20/19 5:38 AM, lik...@gmx.de wrote:
> Regarding the solution you're working on: If I get it right, it's meant to be a disk->disk backup? What I'm looking for is an incremental client-side encrypted backup, similar to the tool duplicati. Also a poor man solution with git+rsync+veracrypt would be possible.

Actually, sparsebak is not direct disk-to-disk but disk-to-archive. An
archive is a directory with a collection of chunk files named for their
address within the source volume.

> Can you imagine how sparsebak could be combined with truecrypt? Is there compression support?

Gzip compression is used.

Some of us are backing up to LUKS containers. On the new4 branch there
is a simple overview of the process under 'Encryption options' in the
README; This can be easily adapted by substituting Veracrypt for
cryptsetup(LUKS).

Be advised that example is rather network intensive as the filesystem is
being run over the network (although still quite usable on a local
server via wifi). If you had intended to run veracrypt on the server
instead (e.g. you trust the server) then this performance issue goes
away (in this case, you would need to configure the .ini with a
qubes-ssh: destination instead of internal: and have a VM setup for ssh
key access to the destination server.)

[Chris Laprise]

This isn't _terribly_ hard to beat. :) For one, there is a trusted
server being accessed by different appVMs. Second, there is a trusted
server in the first place. Third, this is only efficient for the "lots
of small files" usage model, otherwise rsync tends to bog down.

A good backup solution for security-focused systems should run in the
Admin scope, handle disk volumes without getting involved in guest
filesystems, and instantly know which parts of each volume have changed
without data scans or regard to file sizes. The user should also feel
free to consider any volumes for backup without thinking about
additional security logistics (which guests can be trusted, etc).

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

It depends on how much data you have, but for most users, qvm-backup on
a subset of VMs with bzip2 compression is likely to result in a
reasonably small backup file that is suitable for daily or weekly
backups.

IMHO: qvm-backup is an underrated, underutilized tool. It's currently
the most secure way to do backups in Qubes OS (and has been for
years), because it performs authentication and encryption/decryption
in dom0 without having to install any additional programs in dom0.
It's time-tested, reliable, and secure: exactly what I want from a
backup tool. Really the only major thing it's missing is, as you say,
incremental backup functionality.

> [...]

>
> A good backup solution for security-focused systems should run in
> the Admin scope, handle disk volumes without getting involved in
> guest filesystems, and instantly know which parts of each volume
> have changed without data scans or regard to file sizes. The user
> should also feel free to consider any volumes for backup without
> thinking about additional security logistics (which guests can be
> trusted, etc).
>

Agreed. qvm-backup ticks all of these boxes except incremental backups.

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEZQ7rCYX0j3henGH1203TvDlQMDAFAlxuDFoACgkQ203TvDlQ
MDB5HA//a4d7y9PqTmqtyxd2aXXtlf02CHkAM/i6OYJSFQJ0IITq1jA/1nFbD+Y5
9/qY9CZGyKeJkkKPJncFw29ae4OQKsK0f4AUtVbYb9dDQX2trcK55lMOIE3kW2uv
lTMmQPsSRNxJF7Cj0qyBZyDR+jXy/a7q5AgSFykYljvvalBjg7RMTJoIpBZ8zw+L
B0nZAHe8j6Cv4i0kIyLYQJXWl9zm6zzxEHP3QHK3kU8zIjgY334cHeeiTYGgCdtN
24jpQPsCKlaXgS8UgMT3rTnMBHOmPw4EZ1OuruLYhaQBfjbjTEoyjpREaCsk0tso
+bkAlMekyPJbZLgaihimCeJwBse5oeXyWJYjIJBevl1Xocr5z89IKxX4HnTacbI4
ls3lZN2IvWv/PWVPcKBzyPLj5tIfRHSjjBH8iSC6DB8IxXqUfT/c+8ZU/qoPub1x
zDKn4KcLOIROmQJKSJf9UhoCfWp8e96PvMZbuPkhLIwE63v87kkWkG4wB+rF38M3
ogu/pvUYazLV9Fil3o8BN5Dzp//G04NvLYMF7QcZ+H/zbL+QXCf52E5d+p2TnnTX
13L0y1T/kz/18XOduqKXuWs+/h73BrdvtlckW5LXZVwTG0JQwATENXA4Y3G59YrU
96ApVXYSpWfgJ87SihqJFH05NiIPc25vRj9xr9kd6G7B+BLPrZM=
=a9F/
-----END PGP SIGNATURE-----

[Chris Laprise]

Backup is an underutilized category in general, so its not surprising
qvm-backup is also.

OTOH, we could consider Apple's high profile promotion of Time Machine
to be a successful promotion of backups on that platform. Their general
idea is to use the best filesystem features to make backups so efficient
as to be barely noticeable even when performed every hour. Sparsebak is
built on similar technical principles and use case as Time Machine...
people moving between coffee shops and hotels and work should be able to
securely get their backups done without long delays, battery drain, etc.
-- even when you've got lots of data on a Qubes system.

And the archive set shouldn't have to be maintained by the user after
setting it up. Just keep pushing backups at it and the system can
automatically prune old versions to make room for the new. Most backup
systems make you manually curate files (qvm-backup) or wait through a
grinding space reclamation process as it re-writes data.

>
>> [...]
>>
>> A good backup solution for security-focused systems should run in
>> the Admin scope, handle disk volumes without getting involved in
>> guest filesystems, and instantly know which parts of each volume
>> have changed without data scans or regard to file sizes. The user
>> should also feel free to consider any volumes for backup without
>> thinking about additional security logistics (which guests can be
>> trusted, etc).
>>
>
> Agreed. qvm-backup ticks all of these boxes except incremental backups.

Incremental & efficient is what the OP stressed, however. And I'm not
sure why they would resort to bzip compression when that is such a drain
on CPU and time. I chose gzip level 4 as a nice compromise between speed
and disk space... although users will be able to choose in the future,
this is significantly faster than the default level 6 with little
difference in compression ratio.

[Stuart Perkins]

You point out the deficiencies in my setup, and I understand those. You say it isn't hard to beat. You describe a better system, but you don't identify it. Does it exist?

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

True.

> OTOH, we could consider Apple's high profile promotion of Time
> Machine to be a successful promotion of backups on that platform.
> Their general idea is to use the best filesystem features to make
> backups so efficient as to be barely noticeable even when performed
> every hour. Sparsebak is built on similar technical principles and
> use case as Time Machine... people moving between coffee shops and
> hotels and work should be able to securely get their backups done
> without long delays, battery drain, etc. -- even when you've got
> lots of data on a Qubes system.
>
> And the archive set shouldn't have to be maintained by the user
> after setting it up. Just keep pushing backups at it and the system
> can automatically prune old versions to make room for the new. Most
> backup systems make you manually curate files (qvm-backup) or wait
> through a grinding space reclamation process as it re-writes data.
>

Yes, those would be significant quality-of-life improvements.

>>
>>> [...]
>>>
>>> A good backup solution for security-focused systems should run
>>> in the Admin scope, handle disk volumes without getting
>>> involved in guest filesystems, and instantly know which parts
>>> of each volume have changed without data scans or regard to
>>> file sizes. The user should also feel free to consider any
>>> volumes for backup without thinking about additional security
>>> logistics (which guests can be trusted, etc).
>>>
>>
>> Agreed. qvm-backup ticks all of these boxes except incremental
>> backups.
>
> Incremental & efficient is what the OP stressed, however. And I'm
> not sure why they would resort to bzip compression when that is
> such a drain on CPU and time. I chose gzip level 4 as a nice
> compromise between speed and disk space... although users will be
> able to choose in the future, this is significantly faster than the
> default level 6 with little difference in compression ratio.
>

Well, two of the main complaints were about disk space and network
bandwidth. A higher compression ratio can mitigate these factors by
producing a smaller backup. The increased time and CPU usage needn't
pose a problem if the compression is done while the machine is
not in use.

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEZQ7rCYX0j3henGH1203TvDlQMDAFAlxvWC0ACgkQ203TvDlQ
MDBi9RAAyiOZp+23F6jfdq8AoMtj/CPnsLs1HYyy0I+CXaxHa8sFLmNo1K6yIVIh
z5+sd84PybF2VHxb2147gQS6IwFLKHHOBas4DWpiFgSRQe7jA5seQIna4V4seazV
T1XINhshpOgyybsQXNYW7V3tUYwTOGsBIva+rvUSv6SEHPM4FwMD91FhfuhYEDqM
2gFGGrTv6EDtWAtKKoyZY4hB1DLXizsRFoaV8i+9Hvw+x5wkui0BaM2QSs2+5b5L
8Pr/3OghFx0wnX9mPkZw0TbGu3p0v9gLmg5jHIudg6OcmDTTATJi4j+Y6UgiYD0i
q2IGo1xu5fKSeUrNr2DVcJLEc0AUp32S11AE1XCMyxvfuIvr+yVxSVVBHWyQ+2jT
mxEClqTnlAjmtMfvBH0jlI551V8OqyhG7ddcKVh5c4znqg43JFsXC0eobuIIusHE
audsRNJmziuUQVrQ8krZoKsUgKXvObytCDLsezXcm3IxHeXwKChSzcCovEUqAZQk
uHRuT208IxLfK8DZuvD5TMzrGj/rBSP0fRM8Y8FjxEdCysazJce62tJBElsYXavJ
wT0DZAePusQemrB4/cBzIHJyGDbI8rA5+qJflhGf99FR1XQ6aW2AYiUwICJbaoXq
cMMSrW6eCZhbCA+zQwrZmL8qVb9w+BZwJ+64CrwMkHjVCs0dXBI=
=A4xg
-----END PGP SIGNATURE-----