After Debian Server Update from 11 to 12, bareos-fd-postgresql plugin faulty

[Stefan Harbich]

Hello,
after Debian update from version 11 to 12 the bareos-fd-postgresql plugin no longer works. I get the following error message:
...
28-Okt 16:19 bareos-dir JobId 2317: Start Backup JobId 2317, Job=backup-postgresql.2024-10-28_16.19.33_10
28-Okt 16:19 bareos-dir JobId 2317: Connected Storage daemon at bareos.intern.example.com:9103, encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3
28-Okt 16:19 bareos-dir JobId 2317:  Encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3
28-Okt 16:19 bareos-dir JobId 2317: Probing client protocol... (result will be saved until config reload)
28-Okt 16:19 bareos-dir JobId 2317: Connected Client: bareos-fd at dsme01.intern.example.com:9102, encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3
28-Okt 16:19 bareos-dir JobId 2317:    Handshake: Immediate TLS
28-Okt 16:19 bareos-dir JobId 2317:  Encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3
28-Okt 16:19 bareos-dir JobId 2317: Using Device "FileStorage" to write.
28-Okt 16:19 dsme01-fd JobId 2317: Connected Storage daemon at bareos.intern.example.com:9103, encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3
28-Okt 16:19 dsme01-fd JobId 2317:  Encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3
28-Okt 16:19 dsme01-fd JobId 2317: Extended attribute support is enabled
28-Okt 16:19 dsme01-fd JobId 2317: ACL support is enabled
28-Okt 16:19 bareos-sd JobId 2317: Moving to end of data on volume "Full-0195"
28-Okt 16:19 bareos-sd JobId 2317: Ready to append to end of Volume "Full-0195" size=18653920751
28-Okt 16:19 dsme01-fd JobId 2317: Fatal error: filed/fd_plugins.cc:665 PluginSave: Command plugin "python:module_name=bareos-fd-postgresql:db_host=/run/postgresql/:wal_archive_dir=/media/usb/backups/dsme01.intern.example.com/postgresql/wal_archives/" requested, but is not loaded.
...
I have tried almost 30 different configurations:
- Changed permissions in the plugin directory
- Installed the pip package pg8000 under the user root, bareos and postgres
- Made adjustments to the configuration on the FileDaemon and DirectorDaemon (python or python3)
- Installed the operating system package python-pg8000, but has a low version (1.10.6-3)
Started the plugin over 30 times => an E R R O R every time!
Everything worked fine under Debian 11.
I need support.

Greetings from Stefan Harbich

[Silvio Schloeffel]

Hi Stefan,

I have no Debian system but have you controlled if you really have a
Python 2 on the Debian 12 server?

As far as I know you have to use "python3:module_path=..." for the new
modules with python3.

Best

Silvio




Am 28.10.24 um 16:34 schrieb Stefan Harbich:

> Hello,
> after Debian update from version 11 to 12 the bareos-fd-postgresql
> plugin no longer works. I get the following error message:
> ...
> 28-Okt 16:19 bareos-dir JobId 2317: Start Backup JobId 2317, Job=backup-
> postgresql.2024-10-28_16.19.33_10
> 28-Okt 16:19 bareos-dir JobId 2317: Connected Storage daemon at

> bareos.intern.harnet.de:9103, encryption: TLS_CHACHA20_POLY1305_SHA256

> TLSv1.3
> 28-Okt 16:19 bareos-dir JobId 2317:  Encryption:
> TLS_CHACHA20_POLY1305_SHA256 TLSv1.3
> 28-Okt 16:19 bareos-dir JobId 2317: Probing client protocol... (result
> will be saved until config reload)
> 28-Okt 16:19 bareos-dir JobId 2317: Connected Client: bareos-fd at

> dsme01.intern.harnet.de:9102, encryption: TLS_CHACHA20_POLY1305_SHA256

> --
> You received this message because you are subscribed to the Google
> Groups "bareos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to bareos-users...@googlegroups.com <mailto:bareos-
> users+un...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/bareos-
> users/8f7c4852-1a75-4552-9de5-ee178b9d346an%40googlegroups.com <https://
> groups.google.com/d/msgid/bareos-users/8f7c4852-1a75-4552-9de5-
> ee178b9d346an%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
Silvio Schloeffel | CIO
silvio.s...@musicdna.com
Tel. +49 3677 2094 840

the evolution of music continues

MusicDNA GmbH
Schwanitzstrasse 6,
98693 Ilmenau, Germany
Phone: +49 3677 2094 840 | Fax: +49 322 298 222 00

Steuer-NR.: DE 156/114/04317

Handelsregisterzeichen: HRB Jena 517208,
Geschäftsführer: Sebastian Schmidt
www.musicdna.com

[Stefan Harbich]

Hello Silvio,
i'm using Python version 3.11.2. I've adjusted it in the fileset
configuration. But I get the same error. As I said, everything worked
under Debian 11.
Fileset configuration:
...
> root@dsme01:/etc/bareos/bareos-dir.d/fileset# cat plugin-
> postgresql.conf
> FileSet {
>     Name = "plugin-postgresql"
>     Include  {
>         Options {
>             compression=LZ4
>             signature = XXH128
>         }
>         Plugin = "python3"
>                  ":module_name=bareos-fd-postgresql"
>                  ":db_host=127.0.0.1"
>                  ":db_user=postgres"
>                 
> ":wal_archive_dir=/media/usb/backups/dsme01.intern.example.com/postgr
> esql/wal_archives/"
>     }
> }
> root@dsme01:/etc/bareos/bareos-dir.d/fileset# 
> 
...
Error message (The same as before):
...
dsme01-fd JobId 2319: Fatal error: filed/fd_plugins.cc:665 PluginSave:
Command plugin "python3:module_name=bareos-fd-
postgresql:db_host=127.0.0.1:db_user=postgres:wal_archive_dir=/media/us

b/backups/dsme01.intern.example.com/postgresql/wal_archives/"
requested, but is not loaded.
...

Greetings Stefan Harbich

[Bruno Friedmann (bruno-at-bareos)]

HI Stephan

I assume you didn't forget to update bareos repository to point to the debian 12 instead debian 11

In doubt use the helper script in

https://download.bareos.org/current/Debian_12/add_bareos_repositories.sh

Would you mind to recheck which packages are installed on your debian 12.

dpk-query --list | grep "bareos"

dpkg-query --list | grep bareos
ii  bareos                                  23.0.5~pre146.7e91df1c0-130    amd64        Backup Archiving Recovery Open Sourced - metapackage
ii  bareos-bconsole                         23.0.5~pre146.7e91df1c0-130    amd64        Backup Archiving Recovery Open Sourced - text console
ii  bareos-client                           23.0.5~pre146.7e91df1c0-130    amd64        Backup Archiving Recovery Open Sourced - client metapackage
ii  bareos-common                           23.0.5~pre146.7e91df1c0-130    amd64        Backup Archiving Recovery Open Sourced - common files
ii  bareos-database-common                  23.0.5~pre146.7e91df1c0-130    amd64        Backup Archiving Recovery Open Sourced - common catalog files
ii  bareos-database-postgresql              23.0.5~pre146.7e91df1c0-130    amd64        Backup Archiving Recovery Open Sourced - PostgreSQL backend
ii  bareos-database-tools                   23.0.5~pre146.7e91df1c0-130    amd64        Backup Archiving Recovery Open Sourced - database tools
ii  bareos-director                         23.0.5~pre146.7e91df1c0-130    amd64        Backup Archiving Recovery Open Sourced - director daemon
ii  bareos-filedaemon                       23.0.5~pre146.7e91df1c0-130    amd64        Backup Archiving Recovery Open Sourced - file daemon
ii  bareos-filedaemon-python-plugins-common 23.0.5~pre146.7e91df1c0-130    amd64        Backup Archiving Recovery Open Sourced - file daemon Python plugin common files
ii  bareos-filedaemon-python3-plugin        23.0.5~pre146.7e91df1c0-130    amd64        Backup Archiving Recovery Open Sourced - file daemon Python plugin
ii  bareos-storage                          23.0.5~pre146.7e91df1c0-130    amd64        Backup Archiving Recovery Open Sourced - storage daemon
ii  bareos-tools                            23.0.5~pre146.7e91df1c0-130    amd64        Backup Archiving Recovery Open Sourced - common tools

Be sure also that the line activating plugin is not commented  in

/etc/bareos/bareos-fd.d/client/myself.conf
Client {
  Name = 4f9515981768-fd
  Maximum Concurrent Jobs = 20

  # remove comment from "Plugin Directory" to load plugins from specified directory.
  # if "Plugin Names" is defined, only the specified plugins will be loaded,
  # otherwise all filedaemon plugins (*-fd.so) from the "Plugin Directory".
  #
  Plugin Directory = "/usr/lib/bareos/plugins"
  # Plugin Names = ""
}

Under Debian, apt is not able to manage configuration files as rpm is able to do, and sometimes it replace them

instead creating a copy etc....

status client=bareos-fd should also show if plugins are activated

Status without plugin

*status client
Automatically selected Client: bareos-fd
Connecting to Client bareos-fd at localhost:9102

Probing client protocol... (result will be saved until config reload)

 Handshake: Immediate TLS, Encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3

4f9515981768-fd Version: 23.0.5~pre146.7e91df1c0 (11 October 2024)  Debian GNU/Linux 12 (bookworm)
Daemon started 29-Oct-24 08:39. Jobs: run=0 running=0, Bareos community binary
 Sizeof: boffset_t=8 size_t=8 debug=0 trace=0 bwlimit=0kB/s

Running Jobs:
bareos-dir (director) connected at: 29-Oct-24 08:39
No Jobs running.

Status with plugin activated

*status client
Automatically selected Client: bareos-fd
Connecting to Client bareos-fd at localhost:9102
 Handshake: Immediate TLS, Encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3

4f9515981768-fd Version: 23.0.5~pre146.7e91df1c0 (11 October 2024)  Debian GNU/Linux 12 (bookworm)
Daemon started 29-Oct-24 08:40. Jobs: run=0 running=0, Bareos community binary
 Sizeof: boffset_t=8 size_t=8 debug=0 trace=0 bwlimit=0kB/s
Plugin Info:
 Plugin     : bpipe-fd.so
 Description: Bareos Pipe File Daemon Plugin
 Version    : 2, Date: January 2014
 Author     : Kern Sibbald
 License    : Bareos AGPLv3
 Usage      : bpipe:file=<filepath>:reader=<readprogram>:writer=<writeprogram>
 readprogram runs on backup and its stdout is saved
 writeprogram runs on restore and gets restored data into stdin
 the data is internally stored as filepath (e.g. mybackup/backup1)

 Plugin     : python3-fd.so
 Description: Python File Daemon Plugin
 Version    : 4, Date: May 2020
 Author     : Bareos GmbH & Co. KG
 License    : Bareos AGPLv3
 Usage      : python3:module_name=<python-module-to-load>:module_path=<path-to-python-modules>:...

  module_name: The name of the Python module.
  module_path: Python search path for the module.
               The path '/usr/lib/bareos/plugins' is always checked for modules.
  Additional parameters are plugin specific.



Running Jobs:
bareos-dir (director) connected at: 29-Oct-24 08:40
No Jobs running.
====

[Andreas Rogge]

Am 28.10.24 um 17:05 schrieb Stefan Harbich:

> ...
> Error message (The same as before):
> ...
> dsme01-fd JobId 2319: Fatal error: filed/fd_plugins.cc:665 PluginSave:
> Command plugin "python3:module_name=bareos-fd-
> postgresql:db_host=127.0.0.1:db_user=postgres:wal_archive_dir=/media/us
> b/backups/dsme01.intern.example.com/postgresql/wal_archives/"
> requested, but is not loaded.
> ...

Does "status client" for that client show the python3 plugin? Otherwise
you're either missing the plugin package or your fd configuration
doesn't has plugins disabled.

Best Regards,
Andreas

--
Andreas Rogge andrea...@bareos.com
Bareos GmbH & Co. KG Phone: +49 221-630693-86
http://www.bareos.com

Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Komplementär: Bareos Verwaltungs-GmbH
Geschäftsführer: Stephan Dühr, Jörg Steffens, Philipp Storz

[Stefan Harbich]

Hello,

i was able to solve the problem. I had installed pip on the wrong Python version (3.10). Not on the version that ran in the system standard (3.11). This meant that the pg8000 package was bound to Python version 3.10. That is why Bareos could not find the package.
Solution:
Python version 3.10 deleted
pip installed on version 3.11
pg8000 package installed

Everything works now.
Greetings from Stefan Harbich

[Sebastian Sura]

Hi Stefan

the problem seems to be that the file daemon could not load the python plugin.  This normally happens on _startup_ of the file daemon.
Could you start the file daemon manually with the options `-f -d1000` and post that output ? It should contain some messages regarding the initial loading of the plugin.

Kind Regards
Sebastian Sura

Am 28.10.24 um 16:36 schrieb Stefan Harbich:

Greetings from Stefan Harbich --
You received this message because you are subscribed to the Google Groups "bareos-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/bareos-users/6a8cd098-8ef6-4bc1-9ed7-23ae317c03c4n%40googlegroups.com.

-- 
 Sebastian Sura                  sebasti...@bareos.com
 Bareos GmbH & Co. KG            Phone: +49 221 630693-0
 https://www.bareos.com