Bareos bootstrap file records too few

[Simon Cox]

Hi, I noticed since upgrading to Bareos v 23 that only one file record per tape is written to the boot strap file. This makes restore really slow, since bareos has to search fro mthe beginning of the tape to find a file. 

I have Max File Size = 50GB set in the device config. I though it was this that governed how many file markers are recorded?

Below is an example from the bootstrap file and my tape config

Example from bootstrap file

........

Slot=62
VolSessionId=3
VolSessionTime=1691585094
VolAddr=1-1172526073781
FileIndex=103931-573370
Volume="117L8"
MediaType="LTO-8"
Slot=63
VolSessionId=3
VolSessionTime=1691585094
VolAddr=1-1194000921914
FileIndex=573370-1003055
Volume="116L8"
MediaType="LTO-8"
.........

Device config for LTO drive

......

Device {

Name = "drive-1"
DeviceType = tape
DriveIndex = 0
ArchiveDevice = /dev/tape/by-id/scsi-35000e1117342c05b-nst
MediaType = LTO-8
AutoChanger = yes
AutomaticMount = yes
Maximum File Size = 50GB
Maximum Network Buffer Size = 65536
Maximum Block Size = 2097152
AlwaysOpen = yes
RemovableMedia = yes
RandomAccess = no
Spool Directory = /mnt/spool/drive-1
Maximum Spool Size = 1600G
Maximum Concurrent Jobs = 10
}

........

[Sebastian Sura]

Hi Simon

Im not quite sure what the problem is with that bsr.  Could you post a pre-23 bsr and a post-23 bsr of the same job,
so its easier to see where the problem lies ?

If bareos is noticably slower for you after upgrading to 23, then this is a bug.
Please report it at https://github.com/bareos/bareos/issues

Kind Regards
Sebastian Sura

Am 23.05.24 um 13:31 schrieb 'Simon Cox' via bareos-users:

--
You received this message because you are subscribed to the Google Groups "bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/3f83731b-c70e-47d4-97ef-2f7927ec30e1n%40googlegroups.com.

-- 
 Sebastian Sura                  sebasti...@bareos.com
 Bareos GmbH & Co. KG            Phone: +49 221 630693-0
 https://www.bareos.com
 Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
 Komplementär: Bareos Verwaltungs-GmbH
 Geschäftsführer: Stephan Dühr, Jörg Steffens, Philipp Storz

[Sebastian Sura]

Hi Simon,

If I understand you  correctly, the problem is not the generation of the
bootstrap record,
but the fact that bareos chooses to write to multiple different tapes
during the backup.

Its hard to say why this is happening.  Did you change your
configuration at all ?
Do you now have multiple jobs that try to write to the same volume ?
Could you maybe post your configuration and the joblog for the offending
job(s) ?
What does bareos say about the status of the volumes ?

Kind Regards
Sebastian Sura

[Simon Cox]

Hi,

Here's an example of a  job's bsr pre-23 a, the path is different for the jobs, but otherwise, it's an identical configuration. We only do one backup per job as we use  Bareos as an archive repository rather than backup and recovery. Ads you can see in the examples. Pre-23.txt has lots of markers per tape and post-23 has only one per tape, making walking through a 13TB LTO8 a slow process.

Thanks for any insights!

[Simon Cox]

 Hi, not sure attachments are allowed, here are examples of pre and post 23 bsr files It is not exactly the same job, the path is different every time, since we use the system as an archive solution rather than backup. What you can see is the the pre 23 records are quite granular and there are many per lto tape, but post-23 bsr files have only one record per tape.

Grateful for any insights!

Pre 23

# 19-Jan-2022 22:04:39 - xxx.2022-01-10_12.30.00_46 - Full (upgraded from Incremental)

Volume="xxx074L8"

MediaType="LTO-8"

Slot=39

VolSessionId=146

VolSessionTime=1629983495

VolAddr=575525633702-575525641505

FileIndex=1-1589

Volume="xxx074L8"

MediaType="LTO-8"

Slot=39

VolSessionId=146

VolSessionTime=1629983495

VolAddr=579820584960-579820608800

FileIndex=1589-14096

Volume="xxx074L8"

MediaType="LTO-8"

Slot=39

VolSessionId=146

VolSessionTime=1629983495

VolAddr=584115552256-584115576096

FileIndex=14096-16309

Volume="xxx074L8"

MediaType="LTO-8"

Slot=39

VolSessionId=146

VolSessionTime=1629983495

VolAddr=588410519552-588410543392

FileIndex=16309-16827

Post 23

# 29-Aug-2023 09:50:05 - xxx.2023-08-11_13.42.25_50 - Full (upgraded from Incremental)
Volume="xxx125L8"
MediaType="LTO-8"
Slot=61
VolSessionId=3
VolSessionTime=1691585094
VolAddr=798863925657-1108101577830
FileIndex=1-103931
Volume="xxx133L8"
MediaType="LTO-8"

Slot=62
VolSessionId=3
VolSessionTime=1691585094
VolAddr=1-1172526073781
FileIndex=103931-573370

Volume="xxx117L8"

MediaType="LTO-8"
Slot=63
VolSessionId=3
VolSessionTime=1691585094
VolAddr=1-1194000921914
FileIndex=573370-1003055

Volume="xxx116L8"
MediaType="LTO-8"
Slot=64
VolSessionId=3
VolSessionTime=1691585094
VolAddr=1-1103806615636
FileIndex=1003055-1791185

On Monday 17 June 2024 at 13:57:19 UTC+2 Sebastian Sura wrote:

[Simon Cox]

Hi, attached is an example of pre and post 23 bsr files. It is not exactly the same job, the path is different every time, since we use the system as an archive solution rather than backup. Whast you can see is the the pre 23 records are quite granular and there are many per lto tape, but post-23 bsr files have only one record per tape.

Slot=62
VolSessionId=3
VolSessionTime=1691585094
VolAddr=1-1172526073781
FileIndex=103931-573370

Volume="xxx117L8"

MediaType="LTO-8"
Slot=63
VolSessionId=3
VolSessionTime=1691585094
VolAddr=1-1194000921914
FileIndex=573370-1003055

Volume="xxx116L8"
MediaType="LTO-8"
Slot=64
VolSessionId=3
VolSessionTime=1691585094
VolAddr=1-1103806615636
FileIndex=1003055-1791185

To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/5f701dea-d8db-4168-a93c-e008ea344b8a%40bareos.com.

--

Simon Cox

Important Looking Pirates - ILP
Mobile: +46 708 665 814

Visit our website: www.ilpvfx.com
Join us on Facebook: http://www.facebook.com/ImportantLookingPirates

[Sebastian Sura]

What are the contents of the tapes ? e.g. we have

> Slot=68
> VolSessionId=3
> VolSessionTime=1691585094
> VolAddr=1-1168231119062
> FileIndex=3799000-4654766
> Volume="xxx123L8"
> MediaType="LTO-8"

Is it possible that the first TB of this tape just contains the given files and nothing else ? 

What is the result of this query:

select * from jobmedia where jobid=<jobid> and mediaid=<mediaid>

where <jobid> is the job with the session time and session id from above and <mediaid> is the mediaid of the volume above ? Bareos should create a new jobmedia record for every file mark.

If you can, try to do a bscan with option `-r/--list-records` on a volume from that list and check what kind of records are on that tape (and from which job they come).

In Bareos 23 we changed the default settings on devices such that only 1 job can write a given device at a time which would cause a lot of continuous blocks on a tape to belong to the same job which may cause this.

Kind Regards
Sebastian Sura

Am 17.06.24 um 15:20 schrieb 'Simon Cox' via bareos-users:

To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/23c2902b-9024-43ea-9ef2-554311648ab1n%40googlegroups.com.