Storage Daemon stops writing without any visible reason
Christian Garling
Good morning,
we are experiencing the following problem with our Bareos backup.
We are backing up round about 280 systems (most of them Debian 10 or 9 and some Windows servers). The greatest part of them are running version 20. The director and the storage daemon also is on version 20. The director and the clients are located in a datacenter together, the storage daemon is in antother datacenter, connected with a metro LAN connection. The Heartbeat Interval is set to 60 on all systems (director, storage, filedaemon). The tcp_keepalive_time is set to 900.
Sometimes the backup works good for several days, then suddenly it stops working. The jobs are marked as running, but no data is written on the storage daemon side. If I restart the storage daemon, sometimes the backup goes on until its finished, sometimes all unfinished jobs abort.
In the log file I found:
23-Feb 08:01 bareos-dir JobId 0: Fatal error: Director unable to authenticate with Storage daemon at "10.20.2.2:9103". Possible causes:
Passwords or names not the same or
TLS negotiation problem or
Maximum Concurrent Jobs exceeded on the SD or
SD networking messed up (restart daemon).
23-Feb 08:01 bareos-dir JobId 7897: Fatal error: Director's comm line to SD dropped.
23-Feb 08:01 bareos-dir JobId 7899: Fatal error: Director's comm line to SD dropped.
23-Feb 08:01 k4680-fd JobId 7902: Fatal error: Failed to authenticate Storage daemon.
23-Feb 08:01 bareos-dir JobId 7902: Fatal error: Bad response to Storage command: wanted 2000 OK storage
, got 2902 Bad storage
Mit freundlichen Grüßen / best regards
Christian Garling
Operating
Pix Software GmbH
An der Beek 255, 41372 Niederkrüchten, Germany
Tel.: +49 (0)2163 / 98999 71
Fax: +49 (0)2163 / 953031
E-Mail: christia...@pixsoftware.de
https://www.pixsoftware.de
Geschäftsführer: Arne Schirmacher, David-Andreas Bergens
Eingetragen beim Amtsgericht Mönchengladbach, HRB 16527
USt-IdNr. DE153180010
Frank Ueberschar
If you are using data-spooling on one or more jobs you may have hit a bug that we just fixed in the upcoming bareos-20.0.1.
Best, Frank
-- Mit freundlichen Grüßen Frank Ueberschar frank.ue...@bareos.com Bareos GmbH & Co. KG Phone: +49 221 63 06 93-88 http://www.bareos.com Fax: +49 221 63 06 93-10 Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646 Geschäftsführer: S. Dühr, M. Außendorf, J. Steffens, P. Storz
--
You received this message because you are subscribed to the Google Groups "bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/69b06f8cc95d4a56aa9a40bf918e2980%40pixsoftware.de.
Christian Garling
Hi Frank,
thank you for the quick response. I do not know, if we use data-spooling. I did not set any options fort hat explicit.
How can I check that?
Mit freundlichen Grüßen / best regards
Christian Garling
Operating
Pix Software GmbH
An der Beek 255, 41372 Niederkrüchten, Germany
Tel.: +49 (0)2163 / 98999 71
Fax: +49 (0)2163 / 953031
E-Mail: christia...@pixsoftware.de
https://www.pixsoftware.de
Geschäftsführer: Arne Schirmacher, David-Andreas Bergens
Eingetragen beim Amtsgericht Mönchengladbach, HRB 16527
USt-IdNr. DE153180010
Von: bareos...@googlegroups.com <bareos...@googlegroups.com>
Im Auftrag von Frank Ueberschar
Gesendet: Dienstag, 23. Februar 2021 14:48
An: bareos...@googlegroups.com
Betreff: Re: [bareos-users] Storage Daemon stops writing without any visible reason
If you are using data-spooling on one or more jobs you may have hit a bug that we just fixed in the upcoming bareos-20.0.1.
Best, Frank
--
Mit freundlichen Grüßen
Frank Ueberschar frank.ue...@bareos.com
Bareos GmbH & Co. KG Phone: +49 221 63 06 93-88
http://www.bareos.com Fax: +49 221 63 06 93-10
Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Geschäftsführer: S. Dühr, M. Außendorf, J. Steffens, P. Storz
Am 23.02.21 um 08:49 schrieb Christian Garling:
Good morning,
--
You received this message because you are subscribed to the Google Groups "bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/69b06f8cc95d4a56aa9a40bf918e2980%40pixsoftware.de.
--
You received this message because you are subscribed to the Google Groups "bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
bareos-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/7b1b00a2-bf69-b0eb-95b9-d77046e38c52%40bareos.com.
Christian Garling
Hi,
I updated Bareos to version 20.0.1 and made some changes to the VMs (specially director and storage daemon) right now and it looks more stable now.
But I still receive these messages nearly every minute while the backups are running:
17-Mar 05:35 bareos-dir JobId 0: Fatal error: TwoWayAuthenticate failed, because job was canceled.
17-Mar 05:35 bareos-dir JobId 0: Fatal error: Director unable to authenticate with Storage daemon at "10.20.2.2:9103". Possible causes:
Passwords or names not the same or
TLS negotiation problem or
Maximum Concurrent Jobs exceeded on the SD or
SD networking messed up (restart daemon).
I ran the storage daemon in the debug mode and found these messages:
bareos-sd (100): lib/bsock.cc:84-0 Construct BareosSocket
bareos-sd (200): lib/bsock.cc:745-0 Identified from Bareos handshake: bareos-dir-R_DIRECTOR version not recognized
bareos-sd (200): lib/try_tls_handshake_as_a_server.cc:61-0 TlsPolicy for bareos-dir is 0
It looks like the director first tries to connect with TLS but its disabled in all configs. Did I miss an option or how can I avoid these?
Mit freundlichen Grüßen / best regards
Christian Garling
Operating
Pix Software GmbH
An der Beek 255, 41372 Niederkrüchten, Germany
Tel.: +49 (0)2163 / 98999 71
Fax: +49 (0)2163 / 953031
E-Mail: christia...@pixsoftware.de
https://www.pixsoftware.de
Geschäftsführer: Arne Schirmacher, David-Andreas Bergens
Eingetragen beim Amtsgericht Mönchengladbach, HRB 16527
USt-IdNr. DE153180010
Von: bareos...@googlegroups.com <bareos...@googlegroups.com>
Im Auftrag von Christian Garling
Gesendet: Mittwoch, 24. Februar 2021 16:34
An: bareos...@googlegroups.com
Betreff: AW: [bareos-users] Storage Daemon stops writing without any visible reason
Hi Frank,
thank you for the quick response. I do not know, if we use data-spooling. I did not set any options fort hat explicit.
How can I check that?
Mit freundlichen Grüßen / best regards
Christian Garling
Operating
Pix Software GmbH
An der Beek 255, 41372 Niederkrüchten, Germany
Tel.: +49 (0)2163 / 98999 71
Fax: +49 (0)2163 / 953031
E-Mail: christia...@pixsoftware.de
https://www.pixsoftware.de
Geschäftsführer: Arne Schirmacher, David-Andreas Bergens
Eingetragen beim Amtsgericht Mönchengladbach, HRB 16527
USt-IdNr. DE153180010
Von: bareos...@googlegroups.com <bareos...@googlegroups.com> Im Auftrag von Frank Ueberschar
Gesendet: Dienstag, 23. Februar 2021 14:48
An: bareos...@googlegroups.com
Betreff: Re: [bareos-users] Storage Daemon stops writing without any visible reason
If you are using data-spooling on one or more jobs you may have hit a bug that we just fixed in the upcoming bareos-20.0.1.
Best, Frank
--
Mit freundlichen Grüßen
Frank Ueberschar frank.ue...@bareos.com
Bareos GmbH & Co. KG Phone: +49 221 63 06 93-88
http://www.bareos.com Fax: +49 221 63 06 93-10
Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Geschäftsführer: S. Dühr, M. Außendorf, J. Steffens, P. Storz
Am 23.02.21 um 08:49 schrieb Christian Garling:
Good morning,
--
You received this message because you are subscribed to the Google Groups "bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/69b06f8cc95d4a56aa9a40bf918e2980%40pixsoftware.de.
--
You received this message because you are subscribed to the Google Groups "bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
bareos-users...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/bareos-users/7b1b00a2-bf69-b0eb-95b9-d77046e38c52%40bareos.com.
--
You received this message because you are subscribed to the Google Groups "bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
bareos-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/431f7e295f1e4b75b06ab2a8300638f0%40pixsoftware.de.