Director Bug-Report

Zhavoronok

unread,

Apr 1, 2024, 8:10:52 AM4/1/24

to bareos-users

Good afternoon !
You have an error in the director.
It appears when restoring files when the source volumes (Full, Diff, Inc) are located in different folders in the storage

The error is present in all versions (18-22).
Please fix it and update all versions.
Thank you.

================= SD ====================
Device {
Name = Local-Full-Storage-
Media Type = File
Device Type = File
Archive Device = /mnt/drive/Full
Count = 5
LabelMedia = yes;
Random Access = yes;
AutomaticMount = yes;
AlwaysOpen = no;
RemovableMedia = no;
Maximum Open Volumes = 1
Maximum Concurrent Jobs = 1
Volume Poll Interval = 60
Maximum Network Buffer Size = 65536
SpoolDirectory = /data/spool
Maximum Job Spool Size = 20GB
}

Device {
Name = Local-Diff-Storage-
Media Type = File
Device Type = File
Archive Device = /mnt/drive/Diff
Count = 5
LabelMedia = yes;
Random Access = yes;
AutomaticMount = yes;
AlwaysOpen = no;
RemovableMedia = no;
Maximum Concurrent Jobs = 1
Maximum Open Volumes = 1
Volume Poll Interval = 60
Maximum Network Buffer Size = 65536
}

Device {
Name = Local-Inc-Storage-
Media Type = File
Device Type = File
Archive Device = /mnt/drive/Inc
Count = 5
LabelMedia = yes;
Random Access = yes;
AutomaticMount = yes;
AlwaysOpen = no;
RemovableMedia = no;
Maximum Concurrent Jobs = 1
Maximum Open Volumes = 1
Volume Poll Interval = 60
Maximum Network Buffer Size = 65536
}

================= DIR-Storage ====================
Storage {
Name = Local-Full-Storage
Device = Local-Full-Storage-0001
Device = Local-Full-Storage-0002
Device = Local-Full-Storage-0003
Device = Local-Full-Storage-0004
Device = Local-Full-Storage-0005
Media Type = File
Maximum Concurrent Jobs = 5
Maximum Concurrent Read Jobs = 5
Address = backup1.master-plan.nl
Password = "tvMuY3v8f1"

}

Storage {
Name = Local-Diff-Storage
Device = Local-Diff-Storage-0001
Device = Local-Diff-Storage-0002
Device = Local-Diff-Storage-0003
Device = Local-Diff-Storage-0004
Device = Local-Diff-Storage-0005
Media Type = File
Maximum Concurrent Jobs = 5
Maximum Concurrent Read Jobs = 5
Address = backup1.master-plan.nl
Password = "tvMuY3v8f1"

}

Storage {
Name = Local-Inc-Storage
Device = Local-Inc-Storage-0001
Device = Local-Inc-Storage-0002
Device = Local-Inc-Storage-0003
Device = Local-Inc-Storage-0004
Device = Local-Inc-Storage-0005
Media Type = File
Maximum Concurrent Jobs = 5
Maximum Concurrent Read Jobs = 5
Address = backup1.master-plan.nl
Password = "tvMuY3v8f1"
}

================= DIR-Pool ====================

Pool {
Name = Full-6M-Pool
Description = "Test-Full-Pool"
Pool Type = Backup
Maximum Volume Jobs = 1
Label Format = "${Job}-${NumVols}"
Storage = Local-Full-Storage
Recycle = Yes
AutoPrune = Yes
Job Retention = 6 Month
File Retention = 6 Month
Volume Retention = 6 Month
Maximum Volume Bytes = 500GB
ActionOnPurge = Truncate
}

Pool {
Name = Diff-3M-Pool
Description = "Diff-Pool"
Recycle = Yes
Maximum Volume Jobs = 1
Label Format = "${Job}-${Level}-${NumVols}"
Storage = Local-Diff-Storage
AutoPrune = Yes
Job Retention = 3 Month
File Retention = 3 Month
Volume Retention = 3 Month
Maximum Volume Bytes = 500GB
ActionOnPurge = Truncate
}

Pool {
Name = Inc-32D-Pool
Description = "Inc-Pool-32D"
Recycle = Yes
Maximum Volume Jobs = 1
Label Format = "${Job}-${Level}-${NumVols}"
Storage = Local-Inc-Storage
AutoPrune = Yes
Job Retention = 32 Days
File Retention = 32 Days
Volume Retention = 32 Days
Maximum Volume Bytes = 500GB
ActionOnPurge = Truncate
}

================= DIR-job ====================

Job {
Name = test-dmp
JobDefs = DefaultJob
Type = Backup
Client = test-dmp-fd
FileSet = test-dmp
Schedule = Fri_W01
Pool = Inc-32D-Pool
Full Backup Pool = Full-6M-Pool
Differential Backup Pool = Diff-3M-Pool
Incremental Backup Pool = Inc-32D-Pool
Rerun Failed Levels = No
Enabled = Yes

}

================= Bconsole Restore files ====================

restore client=test-dmp-fd restoreclient=bareos-fd where="/mnt/1/" select current all done yes

Using Catalog "MyCatalog"
Automatically selected FileSet: test-dmp
+-------+-------+----------+---------------+---------------------+-----------------------+
| jobid | level | jobfiles | jobbytes | starttime | volumename |
+-------+-------+----------+---------------+---------------------+-----------------------+
| 19 | F | 88,136 | 2,601,570,843 | 2024-03-29 19:02:00 | test-dmp-1 |
| 23 | I | 1 | 173,683,469 | 2024-03-30 19:02:00 | test-dmp-Incremental-4 |
| 26 | I | 1 | 173,801,641 | 2024-03-31 19:02:00 | test-dmp-Incremental-6 |
+-------+-------+----------+---------------+---------------------+-----------------------+
You have selected the following JobIds: 19,23,26

Building directory tree for JobId(s) 19,23,26 ... ++++++++++++++++++++++++++++++++++++++++++++
78,474 files inserted into the tree and marked for extraction.
Bootstrap records written to /var/lib/bareos/bareos-dir.restore.3.bsr

The job will require the following
Volume(s) Storage(s) SD Device(s)
===========================================================================

test-dmp-1 Local-Full-Storage Local-Full-Storage-0001
test-dmp-Incremental-6 Local-Inc-Storage Local-Inc-Storage-0001

Volumes marked with "*" are online.

*list joblog jobid=28
2024-04-01 12:22:46 bareos-dir JobId 28: Start Restore Job RestoreFiles.2024-04-01_12.22.44_34
2024-04-01 12:22:46 bareos-dir JobId 28: Connected Storage daemon at backup1.master-plan.nl:9103, encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3
2024-04-01 12:22:46 bareos-dir JobId 28: Encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3
2024-04-01 12:22:46 bareos-dir JobId 28: Using Device "Local-Full-Storage-0001" to read.
2024-04-01 12:22:46 bareos-dir JobId 28: Probing client protocol... (result will be saved until config reload)
2024-04-01 12:22:46 bareos-dir JobId 28: Connected Client: bareos-fd at localhost:9102, encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3
2024-04-01 12:22:46 bareos-dir JobId 28: Handshake: Immediate TLS
2024-04-01 12:22:46 bareos-dir JobId 28: Encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3
2024-04-01 12:22:48 backup1.master-plan.nl-fd JobId 28: Connected Storage daemon at backup1.master-plan.nl:9103, encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3
2024-04-01 12:22:48 backup1.master-plan.nl-fd JobId 28: Encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3
2024-04-01 12:22:48 bareos-sd JobId 28: Ready to read from volume "test-dmp-1" on device "Local-Full-Storage-0001" (/mnt/drive/Full).
2024-04-01 12:22:48 bareos-sd JobId 28: Forward spacing Volume "test-dmp-1" to file:block 0:211.
2024-04-01 12:22:59 bareos-sd JobId 28: End of Volume at file 0 on device "Local-Full-Storage-0001" (/mnt/drive/Full), Volume "test-dmp-1"
2024-04-01 12:22:59 bareos-sd JobId 28: Warning: stored/acquire.cc:278 Read open device "Local-Full-Storage-0001" (/mnt/drive/Full) Volume "test-dmp-Incremental-6" failed: ERR=stored/dev.cc:605 Could not open: /mnt/drive/Full/test-dmp-Incremental-6, ERR=No such file or directory

2024-04-01 12:22:59 bareos-sd JobId 28: Please mount read Volume "test-dmp-Incremental-6" for:
Job: RestoreFiles.2024-04-01_12.22.44_34
Storage: "Local-Full-Storage-0001" (/mnt/drive/Full)
Pool: Incremental
Media type: File
2024-04-01 13:22:59 bareos-sd JobId 28: Please mount read Volume "test-dmp-Incremental-6" for:
Job: RestoreFiles.2024-04-01_12.22.44_34
Storage: "Local-Full-Storage-0001" (/mnt/drive/Full)
Pool: Incremental
Media type: File

Can't find (test-dmp-Incremental-6) because it is in another directory.

[bareos-dir.d]# ls -l /mnt/drive/Inc/test-dmp-Incremental-6
-rw-r----- 1 bareos bareos 173931275 Mar 31 19:02 /mnt/drive/Inc/test-dmp-Incremental-6

If you restore it like this:

restore client=test-dmp-fd restoreclient=bareos-fd where="/mnt/1/" jobid=19 all done yes
restore client=test-dmp-fd restoreclient=bareos-fd where="/mnt/1/" jobid=23 all done yes
restore client=test-dmp-fd restoreclient=bareos-fd where="/mnt/1/" jobid=26 all done yes

Everything is restored normally, but this is not convenient with a large number of volumes.

Andreas Rogge

unread,

Apr 2, 2024, 3:49:55 AM4/2/24

to bareos...@googlegroups.com

Hi,
that behaviour is documented, but regularly overlooked.
The last paragraph of the Media Type[1] parameter states the following:

"""In addition, if you are using disk storage, each Device resource will
generally have a different mount point or directory. In order for Bareos
to select the correct Device resource, each one must have a unique Media
Type."""

While there probably is room for improvement (feel free to file a PR to
clarify the documentation), distinct storage locations (i.e. mount
points / directories) require distinct media types.

Best Regards,
Andreas

[1]
https://docs.bareos.org/Configuration/StorageDaemon.html#config-Sd_Device_MediaType

--
Andreas Rogge andrea...@bareos.com
Bareos GmbH & Co. KG Phone: +49 221-630693-86
http://www.bareos.com

Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Komplementär: Bareos Verwaltungs-GmbH
Geschäftsführer: Stephan Dühr, Jörg Steffens, Philipp Storz

OpenPGP_0x00314758866BD59E.asc

OpenPGP_signature.asc

Zhavoronok

unread,

Apr 2, 2024, 5:15:04 AM4/2/24

to bareos-users

Good afternoon !

Thanks for your help, I made a different Media Type for each folder.

Tell me how to change the Media Type for previously created volumes.

I need to get them back to work. Thank you very much for your help.

Pool: Inc-32D-Pool
+---------+---------------------------+-----------+---------+-------------+----------+--------------+---------+------+-----------+-------------+---------------------+-------------------+
| mediaid | volumename | volstatus | enabled | volbytes | volfiles | volretention | recycle | slot | inchanger | mediatype | lastwritten | storage |
+---------+---------------------------+-----------+---------+-------------+----------+--------------+---------+------+-----------+-------------+---------------------+-------------------+
| 5 | test-front1-Incremental-0 | Used | 1 | 195,603,360 | 0 | 2,764,800 | 1 | 0 | 0 | File | 2024-03-26 19:02:51 | Local-Inc-Storage |
| 6 | test-front1-Incremental-1 | Used | 1 | 191,850,710 | 0 | 2,764,800 | 1 | 0 | 0 | File | 2024-03-27 19:03:05 | Local-Inc-Storage |
| 7 | test-front1-Incremental-2 | Used | 1 | 193,452,106 | 0 | 2,764,800 | 1 | 0 | 0 | File | 2024-03-28 19:02:50 | Local-Inc-Storage |
| 9 | test-front1-Incremental-3 | Used | 1 | 192,004,056 | 0 | 2,764,800 | 1 | 0 | 0 | File | 2024-03-29 19:02:54 | Local-Inc-Storage |
| 10 | test-dmp-Incremental-4 | Used | 1 | 173,813,007 | 0 | 2,764,800 | 1 | 0 | 0 | File | 2024-03-30 19:02:03 | Local-Inc-Storage |
| 11 | test-front1-Incremental-5 | Used | 1 | 1,583 | 0 | 2,764,800 | 1 | 0 | 0 | File | 2024-03-30 19:02:38 | Local-Inc-Storage |
| 12 | test-dmp-Incremental-6 | Used | 1 | 173,931,275 | 0 | 2,764,800 | 1 | 0 | 0 | File | 2024-03-31 19:02:03 | Local-Inc-Storage |
| 13 | test-front1-Incremental-7 | Used | 1 | 191,838,574 | 0 | 2,764,800 | 1 | 0 | 0 | File | 2024-03-31 19:02:52 | Local-Inc-Storage |
| 14 | test-dmp-Incremental-8 | Used | 1 | 174,214,813 | 0 | 2,764,800 | 1 | 0 | 0 | File | 2024-04-01 19:02:05 | Local-Inc-Storage |
| 15 | test-dmp-Incremental-9 | Used | 1 | 174,150,016 | 0 | 2,764,800 | 1 | 0 | 0 | Storage-Inc | 2024-04-02 12:01:54 | Local-Inc-Storage |
+---------+---------------------------+-----------+---------+-------------+----------+--------------+---------+------+-----------+-------------+---------------------+-------------------+

вторник, 2 апреля 2024 г. в 10:49:55 UTC+3, Andreas Rogge:

Zhavoronok

unread,

Apr 4, 2024, 4:55:47 AM4/4/24

to bareos-users

Good afternoon ! I don’t understand why files located in different folders should have different “Media Type”? This is somehow not logical... These are still the same files... I don't think it's that hard to make changes to your wonderful Bareos program. https://docs.bareos.org/bareos-22/DeveloperGuide/catalog.html In your "Media (Volume)" table you also have the fields "PoolId" and "StorageId" by which you can accurately determine which "Storage Device" you should contact for the next volume. Please correct me, I think it won't take much time. Thank you.