Tape to tape copy performance

[Jon Schewe]

I have 2 LTO-8 drives and I'm executing a copy job from one to the other. I'm using hardware encryption and compression. I have spooling turned off because it's tape to tape, so I woudl expect one could just read from one tape and write to the other. The source tape has 16TB of raw data. I would expect a tape to tape copy to be reasonably fast, however I'm finding that after 24 hours I've only seen 12TB copied between the 2 tapes.

Is this just how slow the tapes are? Or is there something I can reconfigure in bareos to speed this up?

[Bruno Friedmann (bruno-at-bareos)]

Hi Jon,

The encryption key is set by drive/tape so of course it is not just copy that block from here to here, it has to be deciphered uncompressed and then goes the inverse.

From my maths (if correct) it look like you have 142MB/s read/write which is ~50% of native speed.

Maybe the hba is somewhat limited and can handle more than the native 300MB/s one way?

[Jon Schewe]

Bruno,

I understand it's a decrypt and decompress, transfer data and then do it over again. I would hope that the hardware could do encryption and compression at line rate.

So you're thinking it's hardware and there aren't things I should change in bareos? I'm assuming that tape to tape copies should not spool, is that a reasonable assumption?

I'm seeing about 140MB/s on a newer system as well. So wasn't sure if it's the hba or Linux or bareos or something else.

[Andreas Rogge]

Hi Jon,

what's your Maximum File Size setting on the tape device?
The default of 1 GB will not perform very well with recent tape drives,
as it is way too low and flushes the drive buffer far too often.

For encryption/decryption the drives can definitely do this at
line-speed. I haven't seen any performance impact with LTO encryption yet.

Best Regards,
Andreas

Am 10.04.25 um 18:07 schrieb Jon Schewe:

> --
> You received this message because you are subscribed to the Google
> Groups "bareos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to bareos-users...@googlegroups.com <mailto:bareos-
> users+un...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/bareos-
> users/22318893-035d-4d10-9f52-e45906bed9e3n%40googlegroups.com <https://
> groups.google.com/d/msgid/bareos-users/22318893-035d-4d10-9f52-
> e45906bed9e3n%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
Andreas Rogge andrea...@bareos.com
Bareos GmbH & Co. KG Phone: +49 221-630693-86
http://www.bareos.com

Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Komplementär: Bareos Verwaltungs-GmbH
Geschäftsführer: Stephan Dühr, Jörg Steffens, Philipp Storz

[Jon Schewe]

I have not set the Maximum File Size for my tape devices as I assumed this was only for file based backups. I now see https://docs.bareos.org/bareos-19.2/TasksAndConcepts/AutochangerSupport.html#tapespeed-and-blocksizes points to changing this value. Is there a recommendation for LTO-8 and LTO-9 devices?

It looks like changing the Maximum Block Size can be a challenge to keep compatibility with older backups. Does changing this make a large difference in performance and make it worth the risks?

[Andreas Rogge]

Hi Jon,

Am 14.04.25 um 19:36 schrieb Jon Schewe:

> I have not set the Maximum File Size for my tape devices as I assumed
> this was only for file based backups. I now see https://docs.bareos.org/
> bareos-19.2/TasksAndConcepts/AutochangerSupport.html#tapespeed-and-
> blocksizes points to changing this value. Is there a recommendation for
> LTO-8 and LTO-9 devices?

My personal recommendation for newer LTO generations (i.e. LTO-6 and
newer) is to set Maximum File Size to 250 GB.

Just to add a bit of background information:
As you already know tapes basically contain a stream of blocks. However,
historically (i.e. when using tar/cpio directly) you would put multiple
files onto a tape. Therefore end-of-file marks are placed on the tape
and the drives can fast-forward to these.
Bareos makes use of this by putting end-of-file marks onto the tape at
least every Maximum File Size bytes.
When putting such an end-of-file mark into the tape, there is also a
guarantee that the data including the mark was actually persisted to the
tape (and is not just in the drive's buffer).

The downside of this is, of course, that the tape buffer is flushed
every time a mark is written. As modern drives have pretty large caches
(1 GB and maybe even more), the drive will never come up to speed as
you're basically synchronously flushing the buffer every time before it
gets full.

> It looks like changing the Maximum Block Size can be a challenge to keep
> compatibility with older backups. Does changing this make a large
> difference in performance and make it worth the risks?

Please do not mix up Maximum File Size and Maximum Block Size. These are
totally different things.
Maximum Block Size nowadays defaults to 1 MB. Outside of testing there
should be no need to ever touch this parameter.
Read compatibility was also added to Bareos 23: When the SD encounters a
block that is larger than its configured Maximum Block Size, it will
automatically retry with 1,2,4,8 and 16 MB block size.
So while the warnings in the whitepaper about changing the Maximum Block
Size are still valid, restore should still work even with a
misconfigured SD.

Best Regards,
Andreas

[Jon Schewe]

Andreas,

Thank you for that helpful explanation. I recognize that Maximum File Size and Maximum Block Size are different. Glad to hear that I can leave Maximum Block Size alone. For Maximum File Size, I'll try making that change when I can restart my storage daemon next. Initially I was confused as I figured this would relate to the files from the client that are being written, however it appears this is more about how bareos does buffering of data to the tapes.