Filesystem snapshot support
162 views
Skip to first unread message
Christian Svensson
Jul 6, 2021, 8:27:19 AM7/6/21
to bareos-users
Hello,
I am curious about the state of snapshot management in Bareos (and Bacula).
It seems that in the past Bacula at least had ZFS/LVM/BTRFS snapshot
support[1] but that seems to have been removed at some point.
To me, the ability of taking backup of a complex filesystem using e.g.
BTRFS incremental snapshots[2] seems like a very nice feature to have
in Bearos.
It seems it would be quite trivial to add support for implementing
Full, Incremental, and even Differential backups using these
snapshots.
Why would I need this? Consistency. I would like to snapshot the
filesystem in a point-of-time, not relying on that the file daemon is
able to work fast enough for the backup to not diverge too much from
when it started to when it finished.
I can see two operation modes:
1) File-level:
Ideally the file daemon would sense that "hey, this is a btrfs volume,
I will take a snapshot to read all the files in the fileset from" and
it would be transparent to the sysadmin setting things up.
When the backup is complete, the snapshot is removed.
Changes are detected using the normal scanning of mtime etc.
2) Filesystem-level:
This is more involved but handles complete restores of the full FS
using "btrfs send" and "btrfs receive" but changes are instead handled
by btrfs, ensuring that all changed data is backed up regardless of
timestamps.
Thoughts?
If I wanted to implement (2), what would be a good way to do that - as
a python plugin?
I am curious about the state of snapshot management in Bareos (and Bacula).
It seems that in the past Bacula at least had ZFS/LVM/BTRFS snapshot
support[1] but that seems to have been removed at some point.
To me, the ability of taking backup of a complex filesystem using e.g.
BTRFS incremental snapshots[2] seems like a very nice feature to have
in Bearos.
It seems it would be quite trivial to add support for implementing
Full, Incremental, and even Differential backups using these
snapshots.
Why would I need this? Consistency. I would like to snapshot the
filesystem in a point-of-time, not relying on that the file daemon is
able to work fast enough for the backup to not diverge too much from
when it started to when it finished.
I can see two operation modes:
1) File-level:
Ideally the file daemon would sense that "hey, this is a btrfs volume,
I will take a snapshot to read all the files in the fileset from" and
it would be transparent to the sysadmin setting things up.
When the backup is complete, the snapshot is removed.
Changes are detected using the normal scanning of mtime etc.
2) Filesystem-level:
This is more involved but handles complete restores of the full FS
using "btrfs send" and "btrfs receive" but changes are instead handled
by btrfs, ensuring that all changed data is backed up regardless of
timestamps.
Thoughts?
If I wanted to implement (2), what would be a good way to do that - as
a python plugin?
Spadajspadaj
Jul 6, 2021, 8:33:11 AM7/6/21
to bareos...@googlegroups.com
In case of filesystems I can think of as interesting for myself:
1) Windows FD has VSS support
2) In case of ZFS/LVM you can run a pre/post scripts creating a snapshot
and mounting it for reading then unmounting and removing snapshot after
backup.
I suppose you can pack it into a python plugin but on first glance it
seems as a bit of an overkill.
1) Windows FD has VSS support
2) In case of ZFS/LVM you can run a pre/post scripts creating a snapshot
and mounting it for reading then unmounting and removing snapshot after
backup.
I suppose you can pack it into a python plugin but on first glance it
seems as a bit of an overkill.
Christian Svensson
Jul 6, 2021, 8:39:59 AM7/6/21
to Spadajspadaj, bareos-users
Hello,
On Tue, Jul 6, 2021 at 2:33 PM Spadajspadaj <spadaj...@gmail.com> wrote:
> 1) Windows FD has VSS support
That's interesting. It would be cool to have feature parity I suppose.
> 2) In case of ZFS/LVM you can run a pre/post scripts creating a snapshot
> and mounting it for reading then unmounting and removing snapshot after
> backup.
>
> I suppose you can pack it into a python plugin but on first glance it
> seems as a bit of an overkill.
I am trying to limit the amount of power the director has over my -fds
by allowing only backup and restores.
The "runscript" is a bit scary I think so I would prefer it if it can
be avoided by e.g. having a plugin.
But that's just my own threat model, it is OK if it is not shared by
other folks :-).
I guess I could always have a /.snapshot/ directory in every supported
filesystem with a cron that refreshes it, and configure my FileSets to
only copy those files.
That would be an easy way to do it.
Regards,
On Tue, Jul 6, 2021 at 2:33 PM Spadajspadaj <spadaj...@gmail.com> wrote:
> 1) Windows FD has VSS support
> 2) In case of ZFS/LVM you can run a pre/post scripts creating a snapshot
> and mounting it for reading then unmounting and removing snapshot after
> backup.
>
> I suppose you can pack it into a python plugin but on first glance it
> seems as a bit of an overkill.
by allowing only backup and restores.
The "runscript" is a bit scary I think so I would prefer it if it can
be avoided by e.g. having a plugin.
But that's just my own threat model, it is OK if it is not shared by
other folks :-).
I guess I could always have a /.snapshot/ directory in every supported
filesystem with a cron that refreshes it, and configure my FileSets to
only copy those files.
That would be an easy way to do it.
Regards,
Spadajspadaj
Jul 6, 2021, 8:50:18 AM7/6/21
to bareos-users
On 06.07.2021 14:39, Christian Svensson wrote:
> Hello,
>
> On Tue, Jul 6, 2021 at 2:33 PM Spadajspadaj <spadaj...@gmail.com> wrote:
>> 1) Windows FD has VSS support
> That's interesting. It would be cool to have feature parity I suppose.
>> 2) In case of ZFS/LVM you can run a pre/post scripts creating a snapshot
>> and mounting it for reading then unmounting and removing snapshot after
>> backup.
>>
>> I suppose you can pack it into a python plugin but on first glance it
>> seems as a bit of an overkill.
> I am trying to limit the amount of power the director has over my -fds
> by allowing only backup and restores.
> The "runscript" is a bit scary I think so I would prefer it if it can
> be avoided by e.g. having a plugin.
> But that's just my own threat model, it is OK if it is not shared by
> other folks :-).
of privileges on the box for the backup job.
And - more importantly - even as a plugin you'd have to give the FD
enough powers to run the snapshot. So you might simply prepare the
script yourself.
> I guess I could always have a /.snapshot/ directory in every supported
> filesystem with a cron that refreshes it, and configure my FileSets to
> only copy those files.
> That would be an easy way to do it.
>
Of course you have to remember of all the caveats that arise from the
asynchronicity between backup jobs and snapshot preparation.
Reply all
Reply to author
Forward
0 new messages