PostgreSQL Backup Configuration in Bacularis - Verification Needed

[Meth]

Hello everyone, I'm setting up PostgreSQL backup in Bacularis and would like to verify if my RunScript configuration is complete and correct. Here's my current setup: RunScript #1 (Before job): - Command: sudo mkdir -p /var/backups/bacula-pg_dump/ - FailJobOnError: No RunScript #2 (Before job): - Command: sudo chown postgres: /var/backups/bacula-pg_dump/ - FailJobOnError: No RunScript #3 (Before job): - Command: /bin/bash -c 'sudo -u postgres pg_dumpall --clean --inserts --verbose --file=/var/backups/bacula-pg_dump/`date +\"%Y-%m-%d_%H:%M\"`.sql 2>/var/log/bacula-pg_dump.log' - RunsOnFailure: Yes - FailJobOnError: No RunScript #4 (After job): - Command: rm -rf /var/backups/bacula-pg_dump - FailJobOnError: No My questions are: 1. Is this configuration sufficient for a complete PostgreSQL backup including the Bacula catalog? 2. Are there any potential issues or missing steps in this setup? 3. Should I add any additional error handling or verification steps? 4. Is the cleanup process (RunScript #4) appropriate, or should I handle it differently? Any suggestions for improving this backup configuration would be greatly appreciated. Thank you in advance!

[Marcin Haba]

On Fri, 20 Dec 2024 at 09:17, Meth <bys...@gmail.com> wrote:

Hi Meth,

Thanks for sharing with us your PostgreSQL backup job settings. I am
not a PostgreSQL expert, but I will try to answer.

> My questions are:
> 1. Is this configuration sufficient for a complete PostgreSQL backup including the Bacula catalog?

I think that it is OK however I don't know all pg_dumpall parameters
and I can't say for 100%.

> 2. Are there any potential issues or missing steps in this setup?

If your intention is to preserve log for all backups then you can change this:

2>/var/log/bacula-pg_dump.log

into this:

2>>/var/log/bacula-pg_dump.log

I would also propose to remove in Runscript #3 :

Runs On Failure: yes
FailJobOnError: No

because otherwise you will not know if the dump will fail.

> 3. Should I add any additional error handling or verification steps?

Bacula checks the error codes of the Runscript commands, so it can be
sufficient.

> 4. Is the cleanup process (RunScript #4) appropriate, or should I handle it differently?

Yes, it looks fine. If you want, you can create a bacula-pg_dump
directory only once, remove content of the bacula-pg_dump directory
after each backup and keep it empty:

rm -f /var/backups/bacula-pg_dump/*.sql

This way it does not need to be created for every backup and
Runscripts #1 and #2 in this case are not needed.

> Any suggestions for improving this backup configuration would be greatly appreciated.

Maybe an improvement could be to remove logging to
/var/log/bacula-pg_dump.log and let errors be displayed on stderr.
This way errors will be added to Bacula job log and if a job fails
then you can go to the job log to see errors.

If you want you can use some keywords from Bacula in the dump file
name (like: %s - job name %c - client name ...etc.). You can find them
here:

https://www.bacula.org/15.0.x-manuals/en/main/Configuring_Director.html#Director:Job:Runscript

At the end, generally in case runscripts I prefer to use scripts to
put all commands there. If you will have more commands to backup
PostgreSQL you can consider putting them in a script and use in one or
two Runscripts.

I hope it will be helpful.

Good luck!

Best regards,
Marcin Haba (gani)

[Meth]

Hi Marcin,  

Thank you so much for your detailed and helpful response! Your suggestions are incredibly valuable, especially regarding:  

- Changing the log redirection to append mode (>>)  
- Removing the "Runs On Failure" and "FailJobOnError" settings for better error visibility  
- The simplified cleanup process suggestion  
- Moving the commands to a dedicated script file  
- Using Bacula's built-in keywords for file naming  

I'll implement these improvements right away, particularly the logging changes to make error tracking more efficient through Bacula's job logs.  

Your expertise and time in providing such a comprehensive answer is greatly appreciated.  

Best regards,

20 Aralık 2024 Cuma tarihinde saat 13:46:22 UTC+3 itibarıyla Marcin Haba şunları yazdı:

[Marcin Haba]

On Fri, 20 Dec 2024 at 13:43, Meth <bys...@gmail.com> wrote:
>
> Hi Marcin,
>
> Thank you so much for your detailed and helpful response! Your suggestions are incredibly valuable, especially regarding:
>
> - Changing the log redirection to append mode (>>)
> - Removing the "Runs On Failure" and "FailJobOnError" settings for better error visibility
> - The simplified cleanup process suggestion
> - Moving the commands to a dedicated script file
> - Using Bacula's built-in keywords for file naming
>
> I'll implement these improvements right away, particularly the logging changes to make error tracking more efficient through Bacula's job logs.

Hi Meth,

I forgot to mention in my previous message that for backing up the
Bacula Catalog database Bacula provides two scripts:
make_catalog_backup and make_catalog_backup.pl The first one is a
shell script, the second one is a script written in Perl. Both scripts
do the same. The Perl script is a bit more modern.

> Your expertise and time in providing such a comprehensive answer is greatly appreciated.

Great. Thanks, Meth :-)