Bareos catalog growing too big

[Valentin Dzhorov]

So I am running an experimental Bareos setup backing up few real clients. So far I've fleshed out the issues I've had and only one remains. I am running the Bareos director on a PostgreSQL 12 and the catalog database is growing too big for the few clients I am backing up. For example, currently I have 5.2TB of backup space used and 70399391 of files listed in the catalog. The catalog however uses 21GB of disk space! That is about 320 bytes used per file. Isn't this quite large? Any tips on where this large disk usage for my catalog is coming from?

[Brock Palen]

I would expect this. I have used Bareos about 2 years and bacula 10 before that. The catalog size is heavily dependent on number of files.

My current psql directory is ~7GB for 19M files. Given you have 70M I think you are about right for ratio.



The most important thing you can do to control database size is file retention time, as that table is by far the largest.

Things to be careful for with reducing that to much is incremental backups including always incremental.

If you have the job entries still you can still restore from an older backup, you just cannot select the files, as bareos only knows where the job starts on the media and where it ends.

Personally I just use a cheap SSD pair to store my database on, it’s low cost enough I would not worry up to ~200GB or so.

Your backup media (Tape or disk) will cost much more than the space your DB takes up if that is a concern.


Brock Palen
1 (989) 277-6075
bro...@mlds-networks.com
www.mlds-networks.com
Websites, Linux, Hosting, Joomla, Consulting

> On Apr 21, 2020, at 10:32 AM, Valentin Dzhorov <vdzh...@gmail.com> wrote:
>
> So I am running an experimental Bareos setup backing up few real clients. So far I've fleshed out the issues I've had and only one remains. I am running the Bareos director on a PostgreSQL 12 and the catalog database is growing too big for the few clients I am backing up. For example, currently I have 5.2TB of backup space used and 70399391 of files listed in the catalog. The catalog however uses 21GB of disk space! That is about 320 bytes used per file. Isn't this quite large? Any tips on where this large disk usage for my catalog is coming from?
>

> --
> You received this message because you are subscribed to the Google Groups "bareos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/6edb197d-69d7-47e9-9209-9d9ea1899aa8%40googlegroups.com.

[Kai Zimmer]

Hi,

my bareos psql directory is 1.1TB in disk space. How do i find out the
number of total files in the catalog?

Best,
Kai

[Brock Palen]

This is imperfect but what I did

su - postgres
psql -d baroes

select count(*) from Files;


Will give you the number of rows in that table.

Brock Palen
1 (989) 277-6075
bro...@mlds-networks.com
www.mlds-networks.com
Websites, Linux, Hosting, Joomla, Consulting

> To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/97ea3340-3933-9d7a-a6ba-2bf9c4cf4aeb%40bbaw.de.

[Valentin Dzhorov]

Thank you for the response and suggestion. I have indeed set the file retention to 7 days and that is the intended time to keep the actual backups. I'll see what will happen when I move all backups to Bareos. So far all is looking good and running smoothly. It is also a big leap since I've been using a very old Bacula for the current backups (version 5) and it has been a struggle with keeping the integrity of the backups.