Re: [bareos-users] some newbie questions

56 views
Skip to first unread message

Jon SCHEWE

unread,
Dec 23, 2020, 9:16:18 AM12/23/20
to bareos-users, Frank Cherry
This may not be exactly what you want, but I spool my data to a disk and then stream to the tape drive. This keeps up a constant stream of data to the tape.

In /etc/bareos/bareos-sd.d/device/tape.conf:
Device {
...
  Maximum Spool Size = 1500Gb
  Spool Directory = /mnt/bacula-spool
}

In /etc/bareos/bareos-dir.d/jobdefs/LTO-Backup.conf:
JobDefs {
...
  Spool Data = yes
}

This spools up to 1.5TB of data to /mnt/bacula-spool on the bareos server. When that limit is reached or the job finishes, then the data is despooled to the tape drive.

Jon Schewe

Research Scientist


C: +1 612.263.2718

O: +1 952.545.5720

jon.s...@rtx.com


Raytheon Technologies

Raytheon Intelligence & Space

5775 Wayzata Blvd. Suite 630

St. Louis Park, MN 55416


RTX.com | LinkedIn | Twitter | Instagram



From: 'Frank Cherry' via bareos-users <bareos...@googlegroups.com>
Sent: Tuesday, December 22, 2020 23:59
To: bareos-users <bareos...@googlegroups.com>
Subject: [External] [bareos-users] some newbie questions
 

Hello there,
I'm Frank, changing from Bacula to Bareos because didn't get it running.

Now Bareos is working well after installation, the test backup and restore of /usr/sbin was successful with "file" as storage on a dedicated hard disk.

Now my goal is the following scenario:

1. using the Quantum Ultrium LTO-7 tape drive:
- I can read an write to tape, using the devices /dev/st0 or /dev/nst0 or /tape
Have add in /etc/bareos/bareos-sd/device a file with this content:

Device {
  Name = TapeStorage
  Media Type = LTO-7
  Archive Device = /dev/st0
  LabelMedia = no;                   # lets Bareos label unlabeled media
  Random Access = no;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = yes;
  AlwaysOpen = yes;
  Description = "LTO 7 Tape."
}

With this file, bareos SD daemon won't start.
What's wrong?

2. My backup plan is, to collect the data from several servers by NFS / GulsterFS connection, store it on a dedicated local disk on the Bareos server and than stream it to the tape inside the bareos server too. Can anyone give me please some hints, which configuration steps I have to do for this scenario?

Thans for any help, Frank

--
You received this message because you are subscribed to the Google Groups "bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/e53797d1-7bbd-48a5-a1bc-debb6991450fn%40googlegroups.com.

DUCARROZ Birgit

unread,
Dec 23, 2020, 9:29:36 AM12/23/20
to Frank Cherry, bareos-users
Hi,

please read the mail exchange in subtitle "Bareos 20 with Ubuntu server
20.04" of this present list; you will find several ideas and commands
/scripts to test and debug your system.

Regards,
Birgit

On 23/12/20 15:15, 'Jon SCHEWE' via bareos-users wrote:
> This may not be exactly what you want, but I spool my data to a disk and
> then stream to the tape drive. This keeps up a constant stream of data
> to the tape.
>
> In /etc/bareos/bareos-sd.d/device/tape.conf:
> Device {
> ...
>   Maximum Spool Size = 1500Gb
>   Spool Directory = /mnt/bacula-spool
> }
>
> In /etc/bareos/bareos-dir.d/jobdefs/LTO-Backup.conf:
> JobDefs {
> ...
>   Spool Data = yes
> }
>
> This spools up to 1.5TB of data to /mnt/bacula-spool on the bareos
> server. When that limit is reached or the job finishes, then the data is
> despooled to the tape drive.
>
> *Jon Schewe*
>
> Research Scientist
>
> *
> *C: +1 612.263.2718
>
> O: +1 952.545.5720
>
> _jon....@rtx.com__<mailto:na...@rtx.com>_
>
> *
> **Raytheon Technologies*
>
> Raytheon Intelligence & Space
>
> 5775 Wayzata Blvd. Suite 630
>
> St. Louis Park, MN 55416
>
>
> _RTX.com <https://www.rtx.com/>_|_LinkedIn
> <https://www.linkedin.com/company/raytheontechnologies>_|_Twitter
> <https://twitter.com/raytheontech>_|_Instagram
> <https://www.instagram.com/raytheontechnologies>_
>
>
> ------------------------------------------------------------------------
> *From:* 'Frank Cherry' via bareos-users <bareos...@googlegroups.com>
> *Sent:* Tuesday, December 22, 2020 23:59
> *To:* bareos-users <bareos...@googlegroups.com>
> *Subject:* [External] [bareos-users] some newbie questions
> <mailto:bareos-users...@googlegroups.com>.
> <https://groups.google.com/d/msgid/bareos-users/e53797d1-7bbd-48a5-a1bc-debb6991450fn%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "bareos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to bareos-users...@googlegroups.com
> <mailto:bareos-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/bareos-users/SN1P110MB025354C113D02DB5AE241D5AEDDE0%40SN1P110MB0253.NAMP110.PROD.OUTLOOK.COM
> <https://groups.google.com/d/msgid/bareos-users/SN1P110MB025354C113D02DB5AE241D5AEDDE0%40SN1P110MB0253.NAMP110.PROD.OUTLOOK.COM?utm_medium=email&utm_source=footer>.

laur...@qsplace.co.uk

unread,
Dec 23, 2020, 9:30:19 AM12/23/20
to bareos...@googlegroups.com
Hi Frank,

1) Have you checked that the bareos user is able to access these
devices? the permissions may be too restrictive (i.e. to root only)

To test, try chmod 777 on the devices, if that works then change the
permissions to suit your environment (I personally use udev rules to
configure this)

I would also recommend you avoid using the /dev/sr devices and instead
look for the persistent named devices (normally /dev/tape/by-*) which
can again be controlled using udev rules.

2) As Jon has highlighted, you can spool your backup data to disk and
then despool to tape; the disadvantage is that the data is not recorded
in the catalogue until the job has completed; this may or may not be of
concern depending on your back up size and speed.

Alternatively if you have disk space, perform a full backup to disk
(which will record items in the catalogue as they are backed up) and
then have another job to migrate this backup onto tape, again this has
its disadvantages too.

-- Lauz
> RTX.com [1] | LinkedIn [2] | Twitter [3] | Instagram [4]
>
> -------------------------
> [5].
>
> --
> You received this message because you are subscribed to the Google
> Groups "bareos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to bareos-users...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/bareos-users/SN1P110MB025354C113D02DB5AE241D5AEDDE0%40SN1P110MB0253.NAMP110.PROD.OUTLOOK.COM
> [6].
>
>
> Links:
> ------
> [1] https://www.rtx.com/
> [2] https://www.linkedin.com/company/raytheontechnologies
> [3] https://twitter.com/raytheontech
> [4] https://www.instagram.com/raytheontechnologies
> [5]
> https://groups.google.com/d/msgid/bareos-users/e53797d1-7bbd-48a5-a1bc-debb6991450fn%40googlegroups.com?utm_medium=email&amp;utm_source=footer
> [6]
> https://groups.google.com/d/msgid/bareos-users/SN1P110MB025354C113D02DB5AE241D5AEDDE0%40SN1P110MB0253.NAMP110.PROD.OUTLOOK.COM?utm_medium=email&utm_source=footer

Frank Cherry

unread,
Dec 23, 2020, 12:14:58 PM12/23/20
to bareos-users
Thanks, have insert it into my configs and start a test but this was terminated by an error. Here is the lof of the job:


2020-12-23 16:32:25

storage1-fd JobId 9: Fatal error: lib/bsock_tcp.cc:131 Unable to connect to Storage daemon on 127.0.0.1:9103. ERR=Verbindungsaufbau abgelehnt

2020-12-23 16:32:25

storage1-fd JobId 9: Fatal error: Failed to connect to Storage daemon: 127.0.0.1:9103

2020-12-23 16:32:25

bareos-dir JobId 9: Fatal error: Bad response to Storage command: wanted 2000 OK storage
, got 2902 Bad storage


2020-12-23 16:32:25

bareos-dir JobId 9: Error: Bareos bareos-dir 19.2.7 (16Apr20):
Build OS: Linux-3.10.0-1062.18.1.el7.x86_64 redhat Red Hat Enterprise Linux release 8.1 (Ootpa)
JobId: 9
Job: BackupStorage1.2020-12-23_16.02.22_04
Backup Level: Full
Client: "Storage1" 19.2.7 (16Apr20) Linux-3.10.0-1062.18.1.el7.x86_64,redhat,CentOS release 6.10 (Final)
FileSet: "Storage1Data" 2020-12-23 14:53:23
Pool: "Full" (From command line)
Catalog: "MyCatalog" (From Client resource)
Storage: "Tape" (From Job resource)
Scheduled time: 23-Dec-2020 16:02:22
Start time: 23-Dec-2020 16:02:24
End time: 23-Dec-2020 16:32:25
Elapsed time: 30 mins 1 sec
Priority: 10
FD Files Written: 0
SD Files Written: 0
FD Bytes Written: 0 (0 B)
SD Bytes Written: 0 (0 B)
Rate: 0.0 KB/s
Software Compression: None
VSS: no
Encryption: no
Accurate: no
Volume name(s):
Volume Session Id: 1
Volume Session Time: 1608735731
Last Volume Bytes: 0 (0 B)
Non-fatal FD errors: 2
SD Errors: 0
FD termination status: Fatal Error
SD termination status: Waiting on FD
Bareos binary info: bareos.org build: Get official binaries and vendor support on bareos.com
Termination: *** Backup Error ***


2020-12-23 16:28:25

storage1-fd JobId 9: Warning: lib/bsock_tcp.cc:125 Could not connect to Storage daemon on 127.0.0.1:9103. ERR=Verbindungsaufbau abgelehnt
Retrying ...

2020-12-23 16:23:15

storage1-fd JobId 9: Warning: lib/bsock_tcp.cc:125 Could not connect to Storage daemon on 127.0.0.1:9103. ERR=Verbindungsaufbau abgelehnt
Retrying ...

2020-12-23 16:18:05

storage1-fd JobId 9: Warning: lib/bsock_tcp.cc:125 Could not connect to Storage daemon on 127.0.0.1:9103. ERR=Verbindungsaufbau abgelehnt
Retrying ...

2020-12-23 16:12:55

storage1-fd JobId 9: Warning: lib/bsock_tcp.cc:125 Could not connect to Storage daemon on 127.0.0.1:9103. ERR=Verbindungsaufbau abgelehnt
Retrying ...

2020-12-23 16:07:45

storage1-fd JobId 9: Warning: lib/bsock_tcp.cc:125 Could not connect to Storage daemon on 127.0.0.1:9103. ERR=Verbindungsaufbau abgelehnt
Retrying ...

2020-12-23 16:02:35

storage1-fd JobId 9: Warning: lib/bsock_tcp.cc:125 Could not connect to Storage daemon on 127.0.0.1:9103. ERR=Verbindungsaufbau abgelehnt
Retrying ...

2020-12-23 16:02:24

bareos-dir JobId 9: Start Backup JobId 9, Job=BackupStorage1.2020-12-23_16.02.22_04

2020-12-23 16:02:24

bareos-dir JobId 9: Connected Storage daemon at 127.0.0.1:9103, encryption: TLS_CHACHA20_POLY1305_SHA256

2020-12-23 16:02:24

bareos-dir JobId 9: Using Device "LTO7-Tape-Drive" to write.

2020-12-23 16:02:24

bareos-dir JobId 9: Probing client protocol... (result will be saved until config reload)

2020-12-23 16:02:24

bareos-dir JobId 9: Connected Client: Storage1 at 192.168.130.170:9102, encryption: PSK-AES256-CBC-SHA

2020-12-23 16:02:24

bareos-dir JobId 9: Handshake: Immediate TLS

2020-12-23 16:02:24

bareos-dir JobId 9: Encryption: PSK-AES256-CBC-SHA

------------------------------------------------------------------------------

Explanation: Client is Storage1 => 192.168.130.170 only FD is running there:
# ps ax | grep bareos
32321 ?        Ssl    0:00 /usr/sbin/bareos-fd -g bareos

# netstat -tulpn | grep bareos
tcp        0      0 0.0.0.0:9102                0.0.0.0:*                   LISTEN      32321/bareos-fd

From this client, I can connect to the Backup-Server 192.168.130.211 with DIR and SD running on it:

# telnet 192.168.130.211 9101
Trying 192.168.130.211...
Connected to 192.168.130.211.
Escape character is '^]'.
x
▒Connection closed by foreign host.

]# telnet 192.168.130.211 9103
Trying 192.168.130.211...
Connected to 192.168.130.211.
Escape character is '^]'.
x
▒Connection closed by foreign host.

During the backup, I have read this status infos:

Client Details:
Connecting to Client Storage1 at 192.168.130.170:9102
 Handshake: Immediate TLS, Encryption: PSK-AES256-CBC-SHA

storage1-fd Version: 19.2.7 (16 April 2020)  Linux-3.10.0-1062.18.1.el7.x86_64 redhat CentOS release 6.10 (Final)
Daemon started 23-Dez-20 15:46. Jobs: run=0 running=0, bareos.org build binary
 Sizeof: boffset_t=8 size_t=8 debug=0 trace=0 bwlimit=0kB/s

Running Jobs:
JobId 9 Job BackupStorage1.2020-12-23_16.02.22_04 is running.
    Full System or Console Job started: 23-Dez-20 16:02
    Files=0 Bytes=0 Bytes/sec=0 Errors=0
    Bwlimit=0
    Files Examined=0
    SDSocket closed.
bareos-dir (director) connected at: 23-Dez-20 16:19

Director:
Running Jobs:
Console connected at 23-Dec-20 16:19
 JobId Level   Name                       Status
======================================================================
     9 Full    BackupStorage1.2020-12-23_16.02.22_04 is waiting for Client Storage1 to connect to Storage Tape
====

Storage:
Storage:

Running Jobs:
Backup Job BackupStorage1.2020-12-23_16.02.22_04 waiting for Client connection.
Writing: Full Backup job BackupStorage1 JobId=9 Volume="Full-0003"
    pool="Full" device="LTO7-Tape-Drive" (/dev/tape)
    spooling=0 despooling=0 despool_wait=0
    Files=0 Bytes=0 AveBytes/sec=0 LastBytes/sec=0
    FDSocket closed

My SD device conf:
Device {
Name = LTO7-Tape-Drive
Drive Index = 0
Media Type = LTO-7
Archive Device = /dev/tape
LabelMedia = yes;
AutomaticMount = yes;
RemovableMedia = yes;
AlwaysOpen = no;
RandomAccess = no;
Spool Directory = /backup_spool/spool
Maximum Concurrent Jobs = 1
MaximumFileSize = 20GB
}

I can access the tape without any problems from CLI:
# mt -f /dev/tape status
SCSI 2 tape drive:
File number=0, block number=0, partition=0.
Tape block size 0 bytes. Density code 0x5c (no translation).
Soft error count since last status=0
General status bits on (41010000):
 BOT ONLINE IM_REP_EN

--------------------------------------------------------
Why did the log says first:
Connected Storage daemon at 127.0.0.1:9103, encryption: TLS_CHACHA20_POLY1305_SHA256
and then later:
storage1-fd JobId 9: Warning: lib/bsock_tcp.cc:125 Could not connect to Storage daemon on 127.0.0.1:9103. ERR=Verbindungsaufbau abgelehnt
Retrying ..

As I do understand right, the fd on the client "storage1"is trying to connect to the storage daemon on 127.0.0.1 from it's point of view itself?
But there is no SD, this should be on 192.168.130.211


Jon SCHEWE

unread,
Dec 23, 2020, 12:20:02 PM12/23/20
to bareos-users, Frank Cherry
It sounds like you have the wrong hostname in the storage config for your director.

Make sure your address isn't localhost in /etc/bareos/bareos-dir.d/storage/*.conf
Storage {

    Address = 192.168.130.211

}

The director sends this address in a message to the client telling the client where to connect to the storage daemon.

Jon Schewe

Research Scientist


C: +1 612.263.2718

O: +1 952.545.5720

jon.s...@rtx.com


Raytheon Technologies

Raytheon Intelligence & Space

5775 Wayzata Blvd. Suite 630

St. Louis Park, MN 55416


From: 'Frank Cherry' via bareos-users <bareos...@googlegroups.com>
Sent: Wednesday, December 23, 2020 11:14
To: bareos-users <bareos...@googlegroups.com>
Subject: [External] Re: [bareos-users] some newbie questions
 
--
You received this message because you are subscribed to the Google Groups "bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.

DUCARROZ Birgit

unread,
Dec 23, 2020, 12:32:36 PM12/23/20
to Frank Cherry, bareos-users
You can test your config by using the command

bareos-dir -t -d100


here a script to run tests before starting the services:
---------8<---------------------
#!/bin/bash
# Purpose: Test config and restart bareos services.
# Interrupt in case of running services.
# Script created: 2019 /bdu
# Last update: 13.06.2020 /bdu
###################################################
echo
printf "%*s" $(tput cols) |tr " " "=\n"
echo "==> Testing bareos-sd..."
/usr/sbin/bareos-sd -t
echo Done.
echo
printf "%*s" $(tput cols) |tr " " "=\n"
echo "==> Testing bareos-fd..."
/usr/sbin/bareos-fd -t
echo Done.
echo
printf "%*s" $(tput cols) |tr " " "=\n"
echo "== > Testing bareos-dir..."
/usr/sbin/bareos-dir -t
echo Done.
printf "%*s" $(tput cols) |tr " " "=\n"
echo

echo
echo "Restart all services now? (sleeping some seconds, so you might
interrupt)"
for ((k = 0; k <= 10 ; k++))
do
echo -n "[ "
for ((i = 0 ; i <= k; i++)); do echo -n "###"; done
for ((j = i ; j <= 10 ; j++)); do echo -n " "; done
v=$((k * 10))
echo -n " ] "
echo -n "$v %" $'\r'
sleep 1
done
echo

# Check if there are actually jobs running. Exit the script if there is
at least one running job:
RJOBS=echo echo list jobs | bconsole | awk '{print $19}' |grep R |awk
'{print $1}' > run-bj
RJOBS=`cat run-bj`
if [[ $RJOBS != "" ]]; then
echo
echo "!!!! THERE ARE ACTUALLY SOME BACKUP JOBS RUNNING. Script
canceled. Please restart later !!!" && exit 1;
echo
else echo "No running jobs found, continuing..."
fi
rm run-bj
echo

echo Restarting bareos services...
/bin/systemctl restart bareos-dir bareos-sd bareos-fd
echo Done.

---------8<----------------

regards,
Birgit
> --
> You received this message because you are subscribed to the Google
> Groups "bareos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to bareos-users...@googlegroups.com
> <mailto:bareos-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/bareos-users/32319ed3-4414-49e7-8238-17a6cca56406n%40googlegroups.com
> <https://groups.google.com/d/msgid/bareos-users/32319ed3-4414-49e7-8238-17a6cca56406n%40googlegroups.com?utm_medium=email&utm_source=footer>.

Frank Kirschner | Celebrate Records GmbH

unread,
Dec 23, 2020, 12:43:15 PM12/23/20
to bareos-users

Yes, this was the trick to solve the problem, my first backup is running now.
Spooling over the disk, will tomorrow evaluate the time and traffic an the decide
between


a) spooling to disk and than to tape

or

b) backup to disk and when finished backup to tape


Thanks for this great support here.
Merry x-mas and stay healthy
Frank

Reply all
Reply to author
Forward
0 new messages