On the matter of Tapes.

[Christian Reiss]

Hey folks.

First off, I am a long time b* user, starting back in the day with
Bacula, then migrated to Bareos several years ago. I deployed several
installations spanning geographically distinct data centres et all. Most
issues I can solve in my sleep 😉 So I have some b* knowledge.

What I do not have is Tape experience. And I would like your input on
this matter.

I got my hands on an LTO-5 drive along with 20 brand new Tapes. Complete
with HBA, so that works. I was able to do some "mt" commands and use tar
for what it is actually intended. Which I never did before, frankly. 😄

What I am aiming for is a permanent cold storage for hand picked stuff.
Dont worry about filelists or backup jobs. Here are my specific questions:

Tape as Volumes
I know that b* actually meant devices to be tapes. (Wow). Now I am
having a manual loader, so no barcodes et all. Lets assume that physical
tape 1 is empty and mounted. Bareos now writes a backup job on that.

1. Do I need to supply the non-rewind or the rewind device?
2. How does Bareos handle end of bareos volume on the tape, eof markers
on tape?
3. How does Bareos write more data, ie backup job 2? Does it require
tape 1, forwards to the eof marker (or end of job?) and continues from
there?
4. How is "tape full" handled? Will the remainder of the backup job
spill over to tape 2?

Setup Questions
I am planing of doing a single, new installation of bareos on my own
pc. Full stack, ie dir, sd and fd. No scheduled run. In my mind I run a
backup job via bconsole by hand, selecting directories and run it.
Bareos will prompt me to insert appendable tape or new tape and do the
usual stuff from there.

My setup:
1. I have a NAS with ~12tb of changing storage. I am placing files there
but old files do vanish. This is my backup source.
2. My PC with the full stack, having mounted all dirs from my NAS to
local. This is where the fd runs "from local".
Goal: The Backup should be a growing Archiv of all files that were on
the NAS, growing with ever more tapes.

My Setup Questions:
1. As this is a cold storage, I would disable accurate flag so deleted
files don't vanish. Because...
2. .. I will only do one initial full backup and then forever
incrementals? Better idea?
3. Restoring of a file (most will never change) will likely to require
only one tape (ignoring spanning of volumes across tapes).
4. I would totally need to backup my local postgresql and conf directory
/ bootstraps or I am in a lot of trouble?

Thanks for your ideas :*)

--
with kind regards,
mit freundlichen Gruessen,

Christian Reiss

[Stoat]

On 12/10/2022 08:36, 'Christian Reiss' via bareos-users wrote:

> What I am aiming for is a permanent cold storage for hand picked stuff.
> Dont worry about filelists or backup jobs. Here are my specific questions:

Don't use B* - LTFS (tape as filesystem) is a better choice.

B* isn't really an archival tool, but it's a good backup/IDS one

> 1. Do I need to supply the non-rewind or the rewind device?

/dev/nstN (or /dev/IBMTAPE/ if you're using those drivers. Be aware that
the IBMtape driver sends a "Status 0, OK" error message after BSF/FSF
which used to upset Bacula and probably still breaks Bareos (the latest
version of the driver has a ST compatibility section but it doesn't
provide the /dev/sg interface you need if you're multipathing in a
fabric environment)

> 2. How does Bareos handle end of bareos volume on the tape, eof markers
> on tape?

Yes

Make sure you test using btape before trying anything else

> 3. How does Bareos write more data, ie backup job 2? Does it require
> tape 1, forwards to the eof marker (or end of job?) and continues from
> there?

It knows where it was. When tape 1 ends it marks it as full and moved to
tape2. When restoring it will zoom to the _exact_ locations needed to
extract blocks

> 4. How is "tape full" handled? Will the remainder of the backup job
> spill over to tape 2?

Yes - -but it's not like tar. You can put dozens of differemt jobs on
one tape and access them all (think of it as a very slow retrieval system)

> My setup:
> 1. I have a NAS with ~12tb of changing storage. I am placing files there
> but old files do vanish. This is my backup source.

Put the tape drive there if you can. LTO5 runs at upwards of 150MB/s and
you don't want network bottlenecks (ie: Gigabit ethernet isn't fast
enough).

In addition for these tape drive speeds, a SSD spool drive should be
considered mandatory along with enlarging the file block size from 10GB
to something considerably larger

The last thing you want is shoe-shining (which is where spooling comes
in) and the drive stops/starts momentarily every time it writes a EOF
marker. minimising these events speeds up backups and reduces tape stress

LTO verify on the fly and rewrite if the read fails so verification
passes aren't necessary on a day to day basis but PAY ATTENTION to error
messages and write errors in particular. Tapes are highly sensitive to
dust - biological (skin) is more or less ok, but hard particles (eg
smoke and geological dust) will destroy individual tape heads - each one
is finer than a human hair (If you hear the drive starting to shoeshine
on a backup when feeding it fast enough, then there's probably a rewrite
occuring.

sg_read_attr is your friend along with various drive message
interpretation tools. One of the last things I did before breaking off
contact in frustration with B*systems was to work with Kern adding
better interpretation of LTO messages (hard vs soft errors, etc). I
don't know if that's been ported to Bareos

> 2. My PC with the full stack, having mounted all dirs from my NAS to
> local. This is where the fd runs "from local".
> Goal: The Backup should be a growing Archiv of all files that were on
> the NAS, growing with ever more tapes.

NFS/SMB is _SLOOOOW_. Put the FD on the NAS if you can

> My Setup Questions:
> 1. As this is a cold storage, I would disable accurate flag so deleted
> files don't vanish. Because...
> 2. .. I will only do one initial full backup and then forever
> incrementals? Better idea?
> 3. Restoring of a file (most will never change) will likely to require
> only one tape (ignoring spanning of volumes across tapes).
> 4. I would totally need to backup my local postgresql and conf directory
> / bootstraps or I am in a lot of trouble?

See comment at top about using LTFS. You can treat the tape like a giant
appendable CDrom that way - directory-structured navigation and no
databases required (the directory lives on the first partition of the
tape and is loaded when the tape is mounted)