Disaster recovery, bextract, and encryption
266 views
Skip to first unread message
Christian Svensson
Jun 16, 2021, 1:40:14 PM6/16/21
to bareos-users
Hi Bareos list,
I am trying to wrap my head around how I can create a good disaster
recovery procedure.
The documentation has some ideas, but specifically what I'd like to do
is extract the BackupCatalog job's SQL database dump and configuration
files with the bare minimal configuration and steps.
I am using both drive encryption (AME) and Bareos data encryption for
my normal backups, and currently that's also true for my BackupCatalog
jobs.
Now, for doing this disaster recovery I would of course need copies of
the applicable decryption keys. For BackupCatalog, which contains the
AME volumes keys, it seems reasonable to skip using AME encryption for
these jobs to avoid a circular dependency.
I plan on creating a tape pool that is not AME encrypted and use it
solely for BackupCatalog jobs.
So far so good.
What I now need is to simply call `bextract` on my BackupCatalog
volumes and decrypt it with my master key, right?
Well, that's the problem - the documentation says that bextract does
not support restore of encrypted files!
That's a bummer.
What would be the best way to tackle this? I am considering
implementing my own encryption (like, taring it up and GPG encrypting
it) so that from Bareos's PoV it is simply plain data.
Then I could use bextract to get the `catalog.tar.gz.gpg`, decrypt it,
and restore my Bareos system from that.
Thoughts?
Regards,
I am trying to wrap my head around how I can create a good disaster
recovery procedure.
The documentation has some ideas, but specifically what I'd like to do
is extract the BackupCatalog job's SQL database dump and configuration
files with the bare minimal configuration and steps.
I am using both drive encryption (AME) and Bareos data encryption for
my normal backups, and currently that's also true for my BackupCatalog
jobs.
Now, for doing this disaster recovery I would of course need copies of
the applicable decryption keys. For BackupCatalog, which contains the
AME volumes keys, it seems reasonable to skip using AME encryption for
these jobs to avoid a circular dependency.
I plan on creating a tape pool that is not AME encrypted and use it
solely for BackupCatalog jobs.
So far so good.
What I now need is to simply call `bextract` on my BackupCatalog
volumes and decrypt it with my master key, right?
Well, that's the problem - the documentation says that bextract does
not support restore of encrypted files!
That's a bummer.
What would be the best way to tackle this? I am considering
implementing my own encryption (like, taring it up and GPG encrypting
it) so that from Bareos's PoV it is simply plain data.
Then I could use bextract to get the `catalog.tar.gz.gpg`, decrypt it,
and restore my Bareos system from that.
Thoughts?
Regards,
Philipp Storz
Jun 17, 2021, 4:54:02 AM6/17/21
to Christian Svensson, bareos-users
Hello,
Am 16.06.21 um 19:40 schrieb 'Christian Svensson' via bareos-users:
In general, I think that disaster recovery and encryption are no good friends.
In case of a real disaster, it is always good to be able to get things working again as simple as
possible.
Encryption makes things more complex and enlarges the probability that you are not able to recover
in a reasonable time.
To be prepared for a disaster, it is advisable to document and test the procedure on a regular basis
to be sure that the disaster recovery really works.
Best regards,
--
Mit freundlichen Grüßen
Philipp Storz philip...@bareos.com
Bareos GmbH & Co. KG Phone: +49 221 63 06 93-92
http://www.bareos.com Fax: +49 221 63 06 93-10
Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Geschäftsführer: Stephan Dühr, M. Außendorf,
J. Steffens, P. Storz
Am 16.06.21 um 19:40 schrieb 'Christian Svensson' via bareos-users:
> Hi Bareos list,
>
> I am trying to wrap my head around how I can create a good disaster
> recovery procedure.
> The documentation has some ideas, but specifically what I'd like to do
> is extract the BackupCatalog job's SQL database dump and configuration
> files with the bare minimal configuration and steps.
>
> I am using both drive encryption (AME) and Bareos data encryption for
> my normal backups, and currently that's also true for my BackupCatalog
> jobs.
> Now, for doing this disaster recovery I would of course need copies of
> the applicable decryption keys. For BackupCatalog, which contains the
> AME volumes keys, it seems reasonable to skip using AME encryption for
> these jobs to avoid a circular dependency.
> I plan on creating a tape pool that is not AME encrypted and use it
> solely for BackupCatalog jobs.
That makes definitely sense
>
> I am trying to wrap my head around how I can create a good disaster
> recovery procedure.
> The documentation has some ideas, but specifically what I'd like to do
> is extract the BackupCatalog job's SQL database dump and configuration
> files with the bare minimal configuration and steps.
>
> I am using both drive encryption (AME) and Bareos data encryption for
> my normal backups, and currently that's also true for my BackupCatalog
> jobs.
> Now, for doing this disaster recovery I would of course need copies of
> the applicable decryption keys. For BackupCatalog, which contains the
> AME volumes keys, it seems reasonable to skip using AME encryption for
> these jobs to avoid a circular dependency.
> I plan on creating a tape pool that is not AME encrypted and use it
> solely for BackupCatalog jobs.
> So far so good.
>
> What I now need is to simply call `bextract` on my BackupCatalog
> volumes and decrypt it with my master key, right?
> Well, that's the problem - the documentation says that bextract does
> not support restore of encrypted files!
> That's a bummer.
>
> What would be the best way to tackle this? I am considering
> implementing my own encryption (like, taring it up and GPG encrypting
> it) so that from Bareos's PoV it is simply plain data.
> Then I could use bextract to get the `catalog.tar.gz.gpg`, decrypt it,
> and restore my Bareos system from that.
That is definitely a possible solution.
>
> What I now need is to simply call `bextract` on my BackupCatalog
> volumes and decrypt it with my master key, right?
> Well, that's the problem - the documentation says that bextract does
> not support restore of encrypted files!
> That's a bummer.
>
> What would be the best way to tackle this? I am considering
> implementing my own encryption (like, taring it up and GPG encrypting
> it) so that from Bareos's PoV it is simply plain data.
> Then I could use bextract to get the `catalog.tar.gz.gpg`, decrypt it,
> and restore my Bareos system from that.
In general, I think that disaster recovery and encryption are no good friends.
In case of a real disaster, it is always good to be able to get things working again as simple as
possible.
Encryption makes things more complex and enlarges the probability that you are not able to recover
in a reasonable time.
To be prepared for a disaster, it is advisable to document and test the procedure on a regular basis
to be sure that the disaster recovery really works.
Best regards,
--
Mit freundlichen Grüßen
Philipp Storz philip...@bareos.com
Bareos GmbH & Co. KG Phone: +49 221 63 06 93-92
http://www.bareos.com Fax: +49 221 63 06 93-10
Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Geschäftsführer: Stephan Dühr, M. Außendorf,
J. Steffens, P. Storz
Christian Svensson
Jun 17, 2021, 12:29:38 PM6/17/21
to Philipp Storz, bareos-users
Many thanks for sanity-checking my thoughts.
On Thu, Jun 17, 2021 at 10:54 AM Philipp Storz <philip...@bareos.com> wrote:
> In general, I think that disaster recovery and encryption are no good friends.
True as that may be, storing data unencrypted is not an option for me :-).
It certainly makes it harder though, but I appreciate a good challenge.
> In case of a real disaster, it is always good to be able to get things working again as simple as
> possible.
>
> Encryption makes things more complex and enlarges the probability that you are not able to recover
> in a reasonable time.
Indeed, it's a cost.
That's also why I wanted to use bextract - to reduce complexity
getting a new Bareos setup up and running. The alternative of
bootstrapping a new using bscan etc. seems to be much more complex in
my eyes.
> To be prepared for a disaster, it is advisable to document and test the procedure on a regular basis
> to be sure that the disaster recovery really works.
No arguments from me there.
Regards,
On Thu, Jun 17, 2021 at 10:54 AM Philipp Storz <philip...@bareos.com> wrote:
> In general, I think that disaster recovery and encryption are no good friends.
It certainly makes it harder though, but I appreciate a good challenge.
> In case of a real disaster, it is always good to be able to get things working again as simple as
> possible.
>
> Encryption makes things more complex and enlarges the probability that you are not able to recover
> in a reasonable time.
That's also why I wanted to use bextract - to reduce complexity
getting a new Bareos setup up and running. The alternative of
bootstrapping a new using bscan etc. seems to be much more complex in
my eyes.
> To be prepared for a disaster, it is advisable to document and test the procedure on a regular basis
> to be sure that the disaster recovery really works.
Regards,
Brock Palen
Jun 17, 2021, 12:45:40 PM6/17/21
to Christian Svensson, Philipp Storz, bareos-users
Just my $0.02
I have saved copies of the tape key and all other keys and bareos config files in an encrypted vault like Vault, 1Password, Cyberark, so I’ll have all me keys replicated in another location but in an encrypted way.
I use as part of my catalog backup after script rclone the catalog volumes to a cloud bucket, in addition to local tape/disk copies on the server.
I also modified so that the SQL file is a dump to disk and then backed up, and not deleted as part of the backup job. They may not be an option for really large catalogs, but points is I have the catalog in a SQL dump on disk, in cloud (encrypted one way) on disk volume (not encrypted except the disk volume with key in replicated key storage) , and tape volume.
Main thing is have those keys so you can pull one of these catalog backups.
Brock Palen
bro...@mlds-networks.com
www.mlds-networks.com
Websites, Linux, Hosting, Joomla, Consulting
> --
> You received this message because you are subscribed to the Google Groups "bareos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/CADiuDAQGUyhbS5ruFmU%2B0fCYpBnwE43mRsSfxFjV26J59KB6-g%40mail.gmail.com.
I have saved copies of the tape key and all other keys and bareos config files in an encrypted vault like Vault, 1Password, Cyberark, so I’ll have all me keys replicated in another location but in an encrypted way.
I use as part of my catalog backup after script rclone the catalog volumes to a cloud bucket, in addition to local tape/disk copies on the server.
I also modified so that the SQL file is a dump to disk and then backed up, and not deleted as part of the backup job. They may not be an option for really large catalogs, but points is I have the catalog in a SQL dump on disk, in cloud (encrypted one way) on disk volume (not encrypted except the disk volume with key in replicated key storage) , and tape volume.
Main thing is have those keys so you can pull one of these catalog backups.
Brock Palen
bro...@mlds-networks.com
www.mlds-networks.com
Websites, Linux, Hosting, Joomla, Consulting
> You received this message because you are subscribed to the Google Groups "bareos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/CADiuDAQGUyhbS5ruFmU%2B0fCYpBnwE43mRsSfxFjV26J59KB6-g%40mail.gmail.com.
Fabio Violino
Jun 22, 2021, 7:48:51 AM6/22/21
to bareos-users
Hi all.
I just learned the hard lesson and lost all my backup: I have no alternative copy of the catalog and no alternative copy of the configuration files, so I can't bextract since I was using PKI encryption.
bextract "sees" the files, recreates them on target disk, but zero-length, complaining with lots and lots of warnings "bextract JobId 0: Error: Unknown stream=20 ignored. This shouldn't happen!".
Luckily enough this happened in a non-disaster situation. It was easy, though, to dump all the bareos server with just a single "apt purge postgresql-10" command and a couple of dreadful confirmations. bareos, which was installed along with postgres, got flushed in seconds down with it.
All my fault, but I was quite sure there was a B-plan to restore from volumes. This is not true: if you use PKI and you want to restore something, then you need a fully operational bareos server with fully working catalog and fully working config file already in place. No B-plan.
So now I have to rebuild all the (almost perfect) setup I had for Always Incremental, AND THEN create a cronjob to make a dump of the catalog (as simple as a pgdump in my case) and make a gz of /etc/bareos folder. Then store it in a safe place, together with all other most precious files.
Just to share this nice moment with you, my friends.
Cheers!
Christian Svensson
Jul 5, 2021, 6:59:47 PM7/5/21
to Fabio Violino, bareos-users
Hello,
I have now implemented what I described for my personal backups.
I have uploaded the relevant parts of my configuration and scripts to my github, here:
For example, here is the backup script that creates the encrypted tar archive:
And here is the job definition:
I ended up creating a new FileDaemon called "catalog-fd" since I had to disable the PKI encryption that my normal client has enabled.
It seems that this is the easiest (only?) way to do that.
I will try to do a full DR recovery soon, hopefully things work out!
Regards,
Reply all
Reply to author
Forward
0 new messages