[Help] Bareos + rclone + google drive

[Dobri Dobrev]

Hey guys,

So.. I've found this post (bacula-related):

https://www.bacula.lat/community/bacula-storage-in-any-cloud-with-rclone-and-rclone-changer/?lang=en

I've configured rclone to properly work with google drive, however...

I'm trying to place the appropriate configs in their respective locations, as I see them in the post, but.. for some weird reason they have an "Autochanger {}" placed in the director? (there's 2 Autochanger {}s) Either that or I don't understand how to properly place each config block in its appropriate location in Bareos, to have my daily backups placed directly in google drive.

Has anyone done this type of configuration in Bareos, and if so - would it be possible to show me example configs?

Regards,

D

[Dobri Dobrev]

So, I figured out the config and adapted to Bareos, however, I'm getting the following errors:
- when a file is supplied to SD's device -- /mnt/vtapes/tape
04-Jun 22:11 BackupServer_StorageDaemon JobId 1: Error: stored/dev.cc:176 /mnt/vtapes/tape is an unknown device type. Must be tape or directory, st_mode=0644

When it is a directory --/mnt/vtapes/tapes
*label barcodes storage=Google_Drive pool=GoogleDrive-Full
Connecting to Storage daemon Google_Drive at backup-storage.local.com:9103 ...
Connecting to Storage daemon Google_Drive at backup-storage.local.com:9103 ...
3306 Issuing autochanger "slots" command.
Device "gdrive" has 0 slots.
No slots in changer to scan.

*label
Automatically selected Storage: Google_Drive
Enter new Volume name: Test
Enter slot (0 or Enter for none): 1
Defined Pools:
     1: Scratch
     2: GoogleDrive-Full
Select the Pool (1-2): 2
Connecting to Storage daemon Google_Drive at backup-storage.local.com:9103 ...
Sending label command for Volume "Test" Slot 1 ...
3304 Issuing autochanger "load slot 1, drive 0" command.
3992 Bad autochanger "load slot 1, drive 0": ERR=Child exited with code 1.
Results=Traceback (most recent call last):
  File "/usr/sbin/rclone-changer", line 270, in <module>
    dispatch(config.command, commandOpts, state)
  File "/usr/sbin/rclone-changer", line 233, in dispatch
    dispatch[command](commandOpts, state)
  File "/usr/sbin/rclone-changer", line 186, in load
    os.unlink(opts.archive)
IsADirectoryError: [Errno 21] Is a directory: '/mnt/vtapes/tapes'

Any assistance would be great ...

If necessary - I'll post all configs in here.

[Dobri Dobrev]

I have some progress, the backups get written in google drive, however, the latest backup is not written, i.e. it stays locally in the file.

It only gets written when a new job starts.

I've tried using "volume only once" (i.e. 1 job per volume), however, it doesn't seem to solve anything.

Any ideas, or, is this a dead group and I'm talking to myself?

[Dobri Dobrev]

Anyone on this????

[Chad William Seys]

Hi Dobri,

> I have some progress, the backups get written in google drive,
> however, the latest backup is not written, i.e. it stays locally in
> the file. It only gets written when a new job starts.

Have you checked the status of the volume in Bareos? It might be that
it isn't yet safe to send the volume to the cloud.


We have a setup rclone (to Google Drive) and bareos. It is completely
different than the rclone changer you found.
If 'rclone mount' works flawlessly these days, the only thing you need
to do is write file volumes into the 'rclone mount'ed google drive
directory.
However, since this didn't work a few years ago (and maybe still does
not) we had to work around the 'rclone mount' shortcomings.
What we did is use mergerfs to create a kind of overlay filesystem
where bareos reads and writes file volumes. Behind the scenes 'rclone
mount' AND 'rclone move' puts the volumes on google drive and makes the
available for reading.
First Bareos is configured to write to file volumes. The volumes are
limited to 10G in size. The file volumes are written to a directory,
say '/bareos/volumes'
In other directory '/bareos/remote' google drive is mounted with
rclone. This is where files are read from google drive.
Finally there is a directory '/bareos/local' which is a local cache
for mergerfs.

mergerfs is mounted like this:
/bareos/remote=RO:/bareos/local=RW /bareos/volumes

/bareos/volumes is the merged mount point.

/bareos/remote=RO is rclone mounted google drive. RO means mergerfs
only reads from it.
The RO and RW options to mergerfs mean that files can only be read from

/bareos/local=RW means that new files written to /bareos/volumes are
created in /bareos/local .

Finally, there is a script which queries bareos volumes to see what
their status is. If they are 'Full' that means they won't be written to
again. This triggers the script to 'rclone move' the volume from
/bareos/volumes to google drive. (Directly to google drive, not by
copying to /bareos/remote).

The script also looks for 'Purged' volumes, which it removes from google
drive. The bareos Pool has the option 'Recycle = no' so that bareos
doesn't attempt to reuse file volumes (which are uploaded to google drive).

So bareos writes and appends to volumes to /bareos/volumes, which
mergerfs places in /bareos/local. When bareos is done writing it
changes the status of the volume to 'Full'. The script notices this and
'rclone move's the volume to google drive. (In the background the
volume switches from being in /bareos/local to /bareos/remote) If
bareos needs to read the volume, it does so through mergerfs, which find
the file in /bareos/remote and causes 'rclone mount' to download the
volume for bareos to read. Restores are so quick that I suspect 'rclone
mount' has the ability to skip around in the volume as bareos requests
(instead of downloading the whole file).

Good luck!
C.