Transferspeed

[Patrick Winter]

Hello,

I have a LTO - Tape system with 3 drives.

the tapes are able to write with 280mb/s, which i can verify.

i tested this with a dataset of 500gb of small files, and another test with 1 single 200gb file. (from the same, and a different server, to be sure the network connection doesnt bottle neck.)

Also with rsync i get the same speed (around 250-280mb/s)

both filesets (the 500gb of small files, and the single 200gb file) are backuped with bareos, with only 100-120 mb/s

I already tried to set maximum_block_size to 1M, as i found some hints that this could help.

my current device configuration looks like this:

Device {
  Name = Drive-0
  DriveIndex = 0
  MediaType = LTO
  ArchiveDevice = /dev/nst1
  AutomaticMount = yes
  RemovableMedia = yes
  RandomAccess = no
  Autochanger = yes
  Maximum Changer Wait = 600
  Maximum Open Wait = 600
  Maximum Block Size = 1048576
}

bareos-dir --version

25.0.2~pre13.add249aa9

Any tips or recommendations are very welcomed.

best regards

Patrick 

[Bruno Friedmann (bruno-at-bareos)]

Did you ever check and adjust the Maximum File Size parameter

https://docs.bareos.org/Configuration/StorageDaemon.html#config-Sd_Device_MaximumFileSize

it should be at least bigger than the buffer of your drive 

MaximumFileSize = 100GB

There's also an ongoing discussion about performance and how to measure it correctly

https://github.com/bareos/bareos/discussions/2497

[Patrick Winter]

Hi Bruno,

I actually missed this setting, and tried setting it up with 200G, as we use LTO-9, and this already increased the speed a lot.

so i will keep experimenting like this, to bring the best performance possible. Thank you very much!

[Bruno Friedmann (bruno-at-bareos)]

There's a number of factor that can greatly affect speed:

In fileset the choice of Signature Algorithm is curcial, we recommend to use xxh128 being far most quicker than md5 

having any compression done by the FD will also greatly influence the speed of transfert.

Of course having tuned PG correctly and not using the default 131MB of share memory will help increasing the needed time to insert file record in the db.

etc etc...

[Brock Palen]

Sorry to interject Bruno this might be useful for that github thread at least for my SAS attached drives (does not work with ibm lintape which is less common)

tapestat 1

Will print the read/write performance of your drives each second. Useful to see what the drive performance is ‘at that exact moment’ as Baroes does report total time (effective speed), which might including spooling, catalogs updates etc.

sg_logs -a /dev/sgX

Where X is the scsi generic device number from lsscsi -g

This gives a lot of details about your drive and the currently mounted media, such as errors, compression ratios etc.
I find much of the data impenetrable but it was useful for confirming behavior.


Brock Palen
bro...@mlds-networks.com
www.mlds-networks.com
Websites, Linux, Hosting, Joomla, Consulting

> --
> You received this message because you are subscribed to the Google Groups "bareos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/bareos-users/bb690c37-cb2a-480c-88c2-743045f9e614n%40googlegroups.com.

[Bruno Friedmann (bruno-at-bareos)]

No worry, this is useful information

I don't know if you saw that discussion about the lin tape

https://github.com/bareos/bareos/discussions/2186#discussioncomment-12262004

If you are willing to test it, it might become supported ;-) 

I believe I will give the tapestat 1 a try and maybe add some information in the tape chapter documentation.

Of course a community PR might accelerate the doc inclusion :-)