Bconsole terminates without feedback when database takes too much time

Horst Horstmann

unread,

Oct 8, 2019, 11:24:58 AM10/8/19

to bareos-users

Hello,

I run 2 systems with Debian 8 and 9 and Bareos 18.2.5.

Both systems use Mariadb as a DB backend, but otherwise differ in both the data to be backed up and the hardware (XEON with traditional hard disks, Raid1, 32 Gbyte RAM, low load by the value of 1 ), Core i7 with SSD, Database on SSD.

If I use the command "restore" in bconsole and then select option 5, it happens on both systems that bconsole reports:

Building directory tree for JobId(s) x,y,z,

but then it doesn't go on, the following "++++++" don't appear and the prompt to select the files to recover doesn't appear.

I get dropped on the shell without any further message

I can see that on both systems the Maria DB systems create some load (without stressing the system), it makes the impression that the database just takes too long.

On one system I exported and re-imported the database (to prevent fragmentation), the dump file was about 2 Gbyte, in PhpMyAdmin the DB is almost 4 Gbyte, also the system was rebooted.

Didnt help.

I started bconsole with the option:

-u <nn> set command execution timeout to <nn> second

and a high movement in seconds, but that didn't do any good.

To counteract this problem, I'm now building new catalogs, and I've put them in Postgresql, since I noticed some performance improvements when using Postgresql in Mysql's time.

I hoped, that this topic was solved with MariaDB.

Now I'm afraid, that the problem will occur again with a correspondingly large catalog (storage time 3 months, ~ 17, or 13 million entries in the files table).

Does anyone know the problem and has an idea?

Many thanks and many greetings

Translated with www.DeepL.com/Translator

Andreas Rogge

unread,

Oct 9, 2019, 2:52:29 AM10/9/19

to bareos...@googlegroups.com

Am 08.10.19 um 17:24 schrieb 'Horst Horstmann' via bareos-users:

> Now I'm afraid, that the problem will occur again with a correspondingly
> large catalog (storage time 3 months, ~ 17, or 13 million entries in the
> files table).

I know people who run Bareos with PostgreSQL Catalog databases that are
larger than 2 TB. So what you're describing is a rather small catalog
and should work with the out-of-the-box configuration of PostgreSQL.

> Does anyone know the problem and has an idea?

The issue you're describing mostly happens on MySQL/MariaDB if you don't
have enough space in /tmp for the queries to run. There might be other
causes, but that one is rather common.

Best Regards,
Andreas
--
Andreas Rogge andrea...@bareos.com
Bareos GmbH & Co. KG Phone: +49 221-630693-86
http://www.bareos.com

Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Komplementär: Bareos Verwaltungs-GmbH
Geschäftsführer: S. Dühr, M. Außendorf, J. Steffens, Philipp Storz

signature.asc

Horst Horstmann

unread,

Oct 9, 2019, 6:08:50 AM10/9/19

to bareos-users

Andreas: Thank you.

I changed my config back to mariadb and tested and monitored /tmp

There are 86 Gbyte available for /tmp and it didnt grow so much, mariadb produced load all the time.

Than bconsole died.

Thats the debug output from bconsole:

Select the Client (1-27): bconsole (900): lib/btimers.cc:201-0 StopBsockTimer called with NULL btimer_id

bconsole (100): console/console.cc:345-0 Got poll BNET_SUB_PROMPT

7

bconsole (900): lib/btimers.cc:201-0 StopBsockTimer called with NULL btimer_id

Automatically selected FileSet: apollox7-fd-UserFiles

+--------+-------+----------+-----------------+---------------------+-----------------------------------------------------------+

+--------+-------+----------+-----------------+---------------------+-----------------------------------------------------------+

| 10,904 | F | 177,388 | 254,896,469,651 | 2019-10-01 01:50:11 | apollox7.fritz.box-2019-10-01-01:50:01-Full-Backup |

| 10,919 | I | 6,467 | 29,713,275,955 | 2019-10-02 04:50:12 | apollox7.fritz.box-2019-10-02-04:50:02-Incremental-Backup |

| 10,941 | I | 9,856 | 28,803,631,544 | 2019-10-05 04:50:15 | apollox7.fritz.box-2019-10-05-04:50:05-Incremental-Backup |

+--------+-------+----------+-----------------+---------------------+-----------------------------------------------------------+

You have selected the following JobIds: 10904,10919,10941

Building directory tree for JobId(s) 10904,10919,10941 ... bconsole (1900): lib/watchdog.cc:310-0 pthread_cond_timedwait 60