Bareos 20

[Robert@yahoomail]

Hi folks and a happy new year to all of you.
I have a running setup of Bareos 20 on Ubuntu 20.04 (Focal Fossa) with a LTO drive as
storage device attached. Once I have an "AlwaysOpen = Yes" in the SD device description
the backup stops after the first job and the status of the SD is like the following:

====

Device status:

Device "My-LTO" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD1_1391028290-nst) is not open.
    Device is being initialized.
==
====

At this point, Bareos is stuck and I need to cancel all remaining active jobs and do a "systemctl restart bareos-sd"
to continue with the backup. An "AlwaysOpen=No" works but then, the tape drive is repositioning the
tape to the current position again - not nice. Previous versions of Bareos handled the "AlwaysOpen=Yes" option correct
and process all the jobs w/o any new tape repositioning in between.
Did I miss something? Any help is appreciated.

Many thanks and best regard,
Robert

[DUCARROZ Birgit]

Hi,

Please try to follow the suggestions you will find in the following posts:
https://groups.google.com/g/bareos-users/c/O0HDRm1WTRc

Regards,
Birgit

[Robert@yahoomail]

Hi everyone,
to put some more light on this issue, I started the Storage-Daemon in command line mode
with a -d100 applied. I get the following after the first job finished:

...
bareos-sd (100): stored/append.cc:80-27 Start append data. res=1
bareos-sd (100): stored/acquire.cc:460-27 acquire_append device is tape
bareos-sd (100): stored/mount.cc:654-27 Swap unloading slot=65535 "My-LTO" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD1_1391028290-nst)
bareos-sd (100): stored/mount.cc:660-27 === set in_use vol=LTO_02
bareos-sd (100): stored/mount.cc:670-27 Set swap_dev=NULL for dev="My-LTO" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD1_1391028290-nst) swap_dev="My-LTO" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD1_1391028290-nst)
bareos-sd (100): stored/mount.cc:626-27 Must load "My-LTO" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD1_1391028290-nst)
bareos-sd (100): stored/autochanger.cc:137-27 Device "My-LTO" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD1_1391028290-nst) is not attached to an autochanger
bareos-sd (50): stored/askdir.cc:181-27 >dird CatReq Job=donald-backup.2021-01-08_14.26.37_05 GetVolInfo VolName=LTO_02 write=1
bareos-sd (50): stored/askdir.cc:108-27 <dird 1000 OK VolName=LTO_02 VolJobs=13 VolFiles=96 VolBlocks=1326427 VolBytes=85570523136 VolMounts=12 VolErrors=0 VolWrites=1326428 MaxVolBytes=0 VolCapacityBytes=0 VolStatus=Append Slot=0 MaxVolJobs=0 MaxVolFiles=0 InChanger=0 VolReadTime=0 VolWriteTime=5007510880 EndFile=95 EndBlock=657 LabelType=0 MediaId=2 EncryptionKey= MinBlocksize=0 MaxBlocksize=0
bareos-sd (50): stored/askdir.cc:140-27 DoGetVolumeInfo return true slot=0 Volume=LTO_02, VolminBlocksize=0 VolMaxBlocksize=0
bareos-sd (50): stored/askdir.cc:144-27 setting dcr->VolMinBlocksize(0) to vol.VolMinBlocksize(0)
bareos-sd (50): stored/askdir.cc:147-27 setting dcr->VolMaxBlocksize(0) to vol.VolMaxBlocksize(0)
bareos-sd (100): stored/autochanger.cc:137-27 Device "My-LTO" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD1_1391028290-nst) is not attached to an autochanger
bareos-sd (100): stored/dev.cc:618-27 open dev: type=2 dev_name="My-LTO" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD1_1391028290-nst) vol=LTO_02 mode=OPEN_READ_WRITE
bareos-sd (100): stored/dev.cc:638-27 call OpenDevice mode=OPEN_READ_WRITE
bareos-sd (100): backends/generic_tape_device.cc:58-27 Open dev: device is tape
bareos-sd (100): backends/generic_tape_device.cc:67-27 Try open "My-LTO" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD1_1391028290-nst) mode=OPEN_READ_WRITE
bareos-sd (100): backends/generic_tape_device.cc:87-27 Open error on "My-LTO" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD1_1391028290-nst) omode=2 oflags=2 errno=16: ERR=Device or resource busy
bareos-sd (100): backends/generic_tape_device.cc:87-27 Open error on "My-LTO" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD1_1391028290-nst) omode=2 oflags=2 errno=16: ERR=Device or resource busy
...

The last message is repeated many times and Bareos finally aborts after 10 minutes with an error message:

08-Jan 14:39 bareos-sd JobId 27: Warning: stored/mount.cc:275 Open device "My-LTO" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD1_1391028290-nst) Volume "LTO_02" failed: ERR=backends/generic_tape_device.cc:141 Unable to open device "My-LTO" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD1_1391028290-nst): ERR=Device or resource busy

Hope someone can use this to identify the root cause.

Best regards,
Robert

Am 07.01.21 um 17:26 schrieb 'Robert@yahoomail' via bareos-users:

[Dennis Benndorf]

Hi Robert,

this wont help you but our environment was also affected by that. So I can confirm that Tape-Setups (at least our two) stop working if upgrading to Bareos 20.

Our setup was Ubuntu 18.04 with Bareos 20. I had to downgrade to 19.2.7 (and restore the database before) to get it working again. 

Regards,

Dennis

-------- Weitergeleitete Nachricht --------

Von: 'Robert@yahoomail' via bareos-users <bareos...@googlegroups.com>

Antwort an: Robert@yahoomail <robert...@yahoo.de>

An: bareos...@googlegroups.com

Betreff: Re: [bareos-users] Bareos 20

Datum: Fri, 8 Jan 2021 14:47:14 +0100

[Robert@yahoomail]

Hi Dennis,
thanks for your quick reply. Don't say it won't help me. Now I know at least, that I'm not alone with this kind of issue. ;-)
Thanks for confirmation.

Best regards,
Robert

Am 08.01.21 um 15:28 schrieb Dennis Benndorf:

[Andreas Rogge]

Hi Folks,

could one of you check if the SD kept the device open when this error
occurs?

i.e. take a look at /proc/<pid of SD>/fd while you're seeing the "device
or resource busy" messages.
Maybe the SD tries to open a the device while it still has it open
(which would be a bug, of course).

Best Regards,
Andreas

--
Andreas Rogge andrea...@bareos.com
Bareos GmbH & Co. KG Phone: +49 221-630693-86
http://www.bareos.com

Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Komplementär: Bareos Verwaltungs-GmbH
Geschäftsführer: S. Dühr, M. Außendorf, J. Steffens, Philipp Storz

[Robert@yahoomail]

Hi Andreas,
thanks for looking into this.
Once the SD gets stuck, I see the following in the directory you mentioned:

root@server:/proc/36003/fd# ll
total 0
lr-x------ 1 bareos bareos 64 Jan 14 19:15 0 -> /dev/null
lr-x------ 1 bareos bareos 64 Jan 14 19:15 1 -> /dev/null
lr-x------ 1 bareos bareos 64 Jan 14 19:15 2 -> /dev/null
lrwx------ 1 bareos bareos 64 Jan 14 19:15 3 -> 'socket:[110568]'
lrwx------ 1 bareos bareos 64 Jan 14 19:15 4 -> 'socket:[117288]'
lrwx------ 1 bareos bareos 64 Jan 14 19:15 5 -> /dev/nst0
lrwx------ 1 bareos bareos 64 Jan 14 19:17 6 -> 'socket:[117291]'

The SD status out of bconsole is giving:

====
Device status:

Device "My-LTO" (/dev/tape/by-id/scsi-1IBM_ULTRIUM-TD1_1391028290-nst) is not open.
    Device is being initialized.
====

I hope this is useful for you.

Many thanks and best regards,
Robert


Am 14.01.21 um 10:59 schrieb Andreas Rogge:

[Andreas Rogge]

Hi Robert,

as you might have noticed, we're looking into the issue. However, we're
not (yet) able to reproduce it in our testing environment.
Would you mind reproducing it again setting the debuglevel to 200 so we
have more trace information? Right now we have a rough idea in what
state the SD is, but no clue how it ended up in that state.

[Robert@yahoomail]

Hi Andreas,

I created a logfile for you with a -d200 to the bares-sd call added.
Please see attached file for details.
Hope this helps.

Best regards,
Robert

Am 15.01.21 um 09:10 schrieb Andreas Rogge:

[Jürgen Echter]

Hi,

i just want to mention that our tape setup broke too since we upgraded to bareos 20, we can backup happily to the tape but if we want to verify or restore it just gets stuck with input/output error, then the drive is stuck.

So reading from the drive seems to fail completely. :(

Greetings

Juergen

--
You received this message because you are subscribed to the Google Groups "bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/59061b30-e883-7fad-7f8a-0840b5421117%40yahoo.de.

-- 
Mit freundlichen Grüssen

Jürgen Echter

ECHTER Küchen & Elektro GmbH Augsburger Str. 49 86529 Schrobenhausen Tel: 08252 / 8976 - 0 Fax: 08252 / 8976 - 10 e-mail: in...@echter-kuechen-elektro.de web: www.echter-kuechen-elektro.de

Reg.-Gericht: Ingolstadt Nr. HR B 101907 Ust.-Id. Nr.: DE234419866 Steuernummer: 124/125/51166 Geschäftsführer: Thomas Echter

[Frank Ueberschar]

Please try the latest Bareos-20 Release which has several bugfixes:

http://download.bareos.org/bareos/release/20/

Best, Frank

Am 03.03.21 um 07:49 schrieb Jürgen Echter:

To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/63b-603f3180-1d-7b577300%40150708432.

[Jürgen Echter]

Hi,

as far as i can tell now, it indeed seems to have fixed the issue we had.

I do some further testing, but it's looking good so far.

Thanks and have a nice day

Juergen

To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/a6fc18b4-8922-78a0-2bda-419bd952151a%40bareos.com.