pgsql-plugin requested, but is not loaded errors
G Jo
I'm trying to get the pgsql-plugin configured to do hot postgres backup and test PITR but without much joy to date.
I've copied the example fileset configurations given at https://github.com/inteos/pgsql-plugin/wiki/FileSet-configuration and the entires from my bareos-dir.conf file are shown below.
Job {
Name = "Vormetric1-PostgresDb"
JobDefs = "DefaultJob"
Client = ip-172-31-72-8-fd
Level = Full
FileSet="Vormetric1DbFiles"
}
Job {
Name = "Vormetric1-ArchiveLogs"
JobDefs = "DefaultJob"
Client = c
Level = Full
FileSet="Vormetric1WALFiles"
}
#1. Backup of archived WAL files, you should use a following FileSet for that:
File Set {
Name = "Vormetric1WALFiles"
Include {
Options { Signature = MD5 }
Plugin = "pgsql:/opt/bacula/etc/pgsql.conf:wal"
}
}
#2. Database files backup (online) with a following FileSet:
File Set {
Name = "Vormetric1DbFiles"
Include {
Options { Signature = MD5 }
Plugin = "pgsql:/opt/bacula/etc/pgsql.conf:db"
}
}
However every time i try to run the any of the 2 jobs above from my director server via the remote file daemon client p-172-31-72-8-fd i get the following error
pgsql:/opt/bacula/etc/pgsql.conf:db" requested, but is not loaded.
I've checked everything i can think of. Permissions to the pgsql.conf file are ok etc.
Any pointers gratefully received and if you need me to expand on my configuration then let me know.
Regards,
George Johnston
Marco van Wieringen
>
> Hi,
>
> I'm trying to get the pgsql-plugin configured to do hot postgres backup
> and test PITR but without much joy to date.
>
> I've copied the example fileset configurations given at
> https://github.com/inteos/pgsql-plugin/wiki/FileSet-configuration
> and the entires from my
I guess you build your plugin using the code at:
https://github.com/bareos/contrib-pgsql-plugin
As that is the only one that might actually work on Bareos as
our plugin interface has been updated overtime to actually fully
work so the original code will not work as it won't get any events
as on Bareos you need to register your events while on the older
Baculas you always got all events even the ones you are not interested in.
Currently this plugin is not fully supported hence it being in a contrib
git repository. Last time I tried it worked though.
>
> pgsql:/opt/bacula/etc/pgsql.conf:db" requested, but is not loaded.
>
Ok what does a status client=ip-172-31-72-8-fd show in bconsole ?
It should show what plugins are loaded and should inlude the pgsql one
if it does not the the plugin is not loaded and the above error is correct.
Check your bareos-fd.conf on the client and make sure the PluginDir is
set. e.g.
Plugin Directory = ...
Plugin Names = "pgsql"
The Plugin Names will only try to load pgsql-fd.so and not any other plugin.
> I've checked everything i can think of. Permissions to the
> pgsql.conf file are ok etc.
>
> Any pointers gratefully received and if you need me to expand on
> my configuration then let me know.
>
--
Marco van Wieringen marco.van...@bareos.com
Bareos GmbH & Co. KG Phone: +49-221-63069389
http://www.bareos.com
Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Komplementär: Bareos Verwaltungs-GmbH
Geschäftsführer: Stephan Dühr, M. Außendorf, J. Steffens,
P. Storz, M. v. Wieringen
G Jo
Hi Marco,
Firstly thank you very much for taking the time to respond to my posting, it’s very much appreciated.
In turn to address the points and the queries from your reply:
[MVW] - I guess you build your plugin using the code at: https://github.com/bareos/contrib-pgsql-plugin]
[GJ] – Yes i believe that’s what we used.
[MVW] - Currently this plugin is not fully supported hence it being in a contrib git repository. Last time I tried it worked though.
[GJ] – Interesting to note both that the plugin is not fully supported but that you have gotten it to work previously.
[MVW] Ok what does a status client=ip-172-31-72-8-fd show in bconsole ?
Check your bareos-fd.conf on the client and make sure the PluginDir is
set. e.g.
Plugin Directory = ...
Plugin Names = "pgsql"
*status client=ip-172-31-72-8-fd
Connecting to Client ip-172-31-72-8-fd at ip-172-31-72-8:9102
I checked my initial bareos-fd.conf configuration and it was as follows with the Plugin Directory explicitly named as /opt/bacula/plugins but no entry for Plugin Names
FileDaemon { # this is me
Name = ip-172-31-72-8-fd
Plugin Directory = /opt/bacula/plugins
Permissions wise the /opt/bacula/plugins directory tree has ownership bareos:bareos and permissions 755. The pgsql-fd.so shared library is present in the directory e.g
[root@ip-172-31-72-8 plugins]# ls -l
-rwxr-xr-x 1 bareos bareos 100234 May 7 12:25 pgsql-fd.so
[MVW] The Plugin Names will only try to load pgsql-fd.so and not any other plugin.
[GJ] I modified my bareos-fd.conf file to explicitly name the pgsql Plugin e.g.
Plugin Directory = /opt/bacula/plugins
Plugin Names = "pgsql"
restarted the ip-172-31-72-8-fd fileDaemon but again the status client=ip-172-31-72-8-fd returned nothing in bconsole.
[GJ] Now the default bareos Plugin Directory which is initially commented in rgw bareos-fd.conf file out is
Plugin Directory = /usr/lib64/bareos/plugins
I modified the file to use this plugin directory instead of /opt/bacula/plugins and restarted the bareos-fd service.
This time the bconsole command status client=ip-172-31-72-8-fd returned details that the bpipe plugin had been loaded
*status client=ip-172-31-72-8-fd
Connecting to Client ip-172-31-72-8-fd at ip-172-31-72-8:9102
Plugin Info:
Plugin : bpipe-fd.so
Description: Bareos Pipe File Daemon Plugin
Version : 2, Date: January 2014
Author : Kern Sibbald
License : Bareos AGPLv3
Checking the contents of the /usr/lib64/bareos/plugins directory I found only the bpipe-fd.so library.
Having proved that the bpipe plugin library could be loaded from the /usr/lib64/bareos/plugins directory I copied the pgsql plugin library from /opt/bacula/plugins to leave the directory contents as
root@ip-172-31-72-8 plugins]# pwd
/usr/lib64/bareos/plugins
[root@ip-172-31-72-8 plugins]# ls -l
total 120
-rwxr-xr-x 1 root root 20056 Dec 31 18:00 bpipe-fd.so
-rwxr-xr-x 1 root root 100234 May 15 10:02 pgsql-fd.so
I restarted the bareos-fd daemon service but with the 2 plugin libraries in place console command status client=ip-172-31-72-8-fd only loaded the bpipe plugin.
I then tried changing the bareos-fd.conf config to name only the pgsql plugin using this configuraiton
Plugin Directory = /usr/lib64/bareos/plugins
Plugin Names = "pgsql"
And this time status client=ip-172-31-72-8-fd showed that no plugins where loaded.
As a penultimate attempt I tried with the pgsql-fd.so plugin being the only plugin library in /usr/lib64/bareos/plugins but again it failed to register as a plugin via status client=ip-172-31-72-8-fd with and without explicitly naming it in Plugin Names.
And finally I tried renaming the pgsql-fd.so to bpipe-fd.so to try and fool bareos into loading it but again no plugins where loaded.
So, in summary no matter what I tried, the pgsql plugin failed to load.
I’ve run out of things to try so I’d be grateful if could review my logic and see if you can suggest anything.
One thing I should mention in case it is relevant
On the director server our bareos catalog database is a mysql database containing 30 tables. However as the pgsql plugin demands a postgres database it is stored in a separate postgres instance with 4 tables.
The pgsql-archlog binary is successfully copying archive log metadata from the 17.31.72.8 server to the director server postgres instance
postgres@ip-172-31-21-156:~$ psql -Upgcat -dcatdb
psql (9.4.1)
Type "help" for help.
catdb=> select * from pgsql_archivelogs;
id | client | filename | create_date | mod_date | status
----+-------------+--------------------------+----------------------------+----------------------------+--------
20 | 172.31.72.8 | 000000010000000000000093 | 2015-05-20 05:32:15.292626 | 2015-05-20 05:32:15.914786 | 3
(20 rows)
Many thanks,
George
g.joh...@kainos.com
+44 28 90 571211
G Jo
Thanks,
George
Marco van Wieringen
> Hi Marco,
>
> Firstly thank you very much for taking the time to respond to my
> posting, it’s very much appreciated.
>
> In turn to address the points and the queries from your reply:
> [MVW] - I guess you build your plugin using the code at:
https://github.com/bareos/contrib-pgsql-plugin]
>
> [GJ] – Yes i believe that’s what we used.
>
Its kind of important that you know for sure. That is the only
plugin that will work.
>
> [GJ] – Interesting to note both that the plugin is not fully supported
> but that you have gotten it to work previously.
Its more that we don't check on regular basis if things still work
and we don't support it as standard part of the BAREOS subscription.
So any support is on best effort or as payed time and material basis.
>
> [GJ] On 1st execution the status client=ip-172-31-72-8-fd returned
> nothing so no plugins where loaded.
>
> *status client=ip-172-31-72-8-fd
> Connecting to Client ip-172-31-72-8-fd at ip-172-31-72-8:9102
>
> I checked my initial bareos-fd.conf configuration and it was
> as follows with the Plugin Directory explicitly named as
> /opt/bacula/plugins but no entry for Plugin Names
>
> FileDaemon { # this is me
> Name = ip-172-31-72-8-fd
> Plugin Directory = /opt/bacula/plugins
>
That is probably because Bacula Enterprise Edition installs
itself into /opt/bacula and Plugin Names is something only
Bareos supports as Bacula loads its plugins in fully random order
(e.g. the way they are found in the directory) and doesn't allow
you to only load some plugins from a dir and in a certain stacked order.
>
> [GJ] Now the default bareos Plugin Directory which is initially commented
> in rgw bareos-fd.conf file out is
>
> Plugin Directory = /usr/lib64/bareos/plugins
>
> I modified the file to use this plugin directory instead of
> /opt/bacula/plugins and restarted the bareos-fd service.
>
Yes the default for BAREOS are indeed different we use the normal
system paths and every supported plugin is installed there when
you install the different RPMs.
> This time the bconsole command status client=ip-172-31-72-8-fd
> returned details that the bpipe plugin had been loaded
>
> *status client=ip-172-31-72-8-fd
> Connecting to Client ip-172-31-72-8-fd at ip-172-31-72-8:9102
>
> Plugin Info:
> Plugin : bpipe-fd.so
> Description: Bareos Pipe File Daemon Plugin
> Version : 2, Date: January 2014
> Author : Kern Sibbald
> License : Bareos AGPLv3
>
> Checking the contents of the /usr/lib64/bareos/plugins directory I
> found only the bpipe-fd.so library.
>
> Having proved that the bpipe plugin library could be loaded from
> the /usr/lib64/bareos/plugins directory I copied the pgsql plugin
> library from /opt/bacula/plugins to leave the directory contents as
>
> root <at> ip-172-31-72-8 plugins]# pwd
> /usr/lib64/bareos/plugins
> [root <at> ip-172-31-72-8 plugins]# ls -l
> total 120
> -rwxr-xr-x 1 root root 20056 Dec 31 18:00 bpipe-fd.so
> -rwxr-xr-x 1 root root 100234 May 15 10:02 pgsql-fd.so
>
> I restarted the bareos-fd daemon service but with the 2 plugin
> libraries in place console command status
> client=ip-172-31-72-8-fd only loaded the bpipe plugin.
>
> I then tried changing the bareos-fd.conf config to name only the
> pgsql plugin using this configuraiton
>
> Plugin Directory = /usr/lib64/bareos/plugins
> Plugin Names = "pgsql"
>
> And this time status client=ip-172-31-72-8-fd showed that no plugins
> where loaded.
Seems the dlopen of the plugin fails probably because its missing some
symbols or such.
>
> As a penultimate attempt I tried with the pgsql-fd.so plugin being the
> only plugin library in /usr/lib64/bareos/plugins but again it failed
> to register as a plugin via status client=ip-172-31-72-8-fd with and
> without explicitly naming it in Plugin Names.
>
> And finally I tried renaming the pgsql-fd.so to bpipe-fd.so to try and
> fool bareos into loading it but again no plugins where loaded.
>
> So, in summary no matter what I tried, the pgsql plugin failed to load.
>
To be sure that the pgsql plugin is a proper plugin for bareos you
could run the bpluginfo binary which peeks at the plugin to see if
it can find the right entry points and some plugin specific MAGIC data.
For the bpipe-fd.so on Solaris that looks like:
15:14 [mvw:mercury][3967]
/projects/GITrepositories/BAREOS/contrib-pgsql-plugin >
/opt/bareos/sbin/i86/bpluginfo -iv /opt/bareos/lib/plugins/bpipe-fd.so
Plugin type: Bareos File Daemon plugin
Plugin magic: *FDPluginData*
Plugin version: 2
Plugin release date: January 2014
Plugin author: Kern Sibbald
Plugin licence: Bareos AGPLv3
Plugin description: Bareos Pipe File Daemon Plugin
Plugin API version: 9
Plugin usage:
bpipe:file=<filepath>:reader=<readprogram>:writer=<writeprogram>
readprogram runs on backup and its stdout is saved
writeprogram runs on restore and gets restored data into stdin
the data is internally stored as filepath (e.g. mybackup/backup1)
Plugin functions:
newPlugin()
freePlugin()
getPluginValue()
setPluginValue()
handlePluginEvent()
startBackupFile()
endBackupFile()
startRestoreFile()
endRestoreFile()
pluginIO()
createFile()
setFileAttributes()
checkFile()
> I’ve run out of things to try so I’d be grateful if could review my
> logic and see if you can suggest anything.
>
> One thing I should mention in case it is relevant
>
> On the director server our bareos catalog database is a mysql database
> containing 30 tables. However as the pgsql plugin demands a postgres
> database it is stored in a separate postgres instance with 4 tables.
>
The actual meta data for the backup program and for the plugin being
separate is fine I think its even better as I would not want to even
merge the two even when its possible. Its also not a problem yet as you
don't get the plugin loaded which is something not related to any config
data being used later on.
You can also run the bareos-fd in debug mode which will probably reveal
much more about the loading of the plugins etc.
e.g. something like bareos-fd -f -d 200
will run the bareos-fd in a pretty high debug level and show what its
doing regarding the loading of plugins.
e.g.
mercury-fd: plugins.c:222-0 load_plugins
mercury-fd: fd_plugins.c:1615-0 is_plugin_compatible called
mercury-fd: fd_plugins.c:1590-0 Loaded plugin: python-fd.so
Marco van Wieringen
I tried it myself and after hacking the Makefile.bareos a bit
(as that is mostly Linux based and I work on Solaris) it seems
things still work mostly.
16:58 [mvw:mercury][4121]
/projects/GITrepositories/BAREOS/contrib-pgsql-plugin >
/opt/bareos/sbin/i86/bpluginfo /opt/bareos/lib/plugins/pgsql-fd.so
pgsql plugin: version 2.2 April 2013 (c) 2011 by Inteos
pgsql plugin: connected to Bareos version 1
Plugin type: Bareos File Daemon plugin
Plugin version: 2.2
Plugin release date: April 2013
Plugin author: Inteos Sp. z o.o.
Plugin licence: AGPLv3
Plugin description: PostgreSQL online backup and recovery plugin (c) Inteos
Sp. z o.o.
Plugin API version: 9
root@mercury:~# /opt/bareos/sbin/i86/bareos-fd -f -d 200
bareos-fd: filed_conf.c:493-0 Inserting Director res: mercury-mon
mercury-fd: bsys.c:622-0 Could not open state file. sfd=-1 size=188: ERR=No
such file or directory
mercury-fd: plugins.c:222-0 load_plugins
mercury-fd: fd_plugins.c:1615-0 is_plugin_compatible called
pgsql plugin: version 2.2 April 2013 (c) 2011 by Inteos
pgsql plugin: connected to Bareos version 9
mercury-fd: fd_plugins.c:1615-0 is_plugin_compatible called
mercury-fd: fd_plugins.c:1590-0 Loaded plugin: python-fd.so
mercury-fd: fd_plugins.c:1590-0 Loaded plugin: pgsql-fd.so
mercury-fd: socket_server.c:96-0 filed: listening on port 8102
mercury-fd: bnet_server_tcp.c:166-0 Addresses host[ipv4;0.0.0.0;8102]
So my tip would be to clone
https://github.com/bareos/contrib-pgsql-plugin
cp Makefile.bareos Makefile
edit Makefile
make; make install
That plugin should a least load I had some problems with the location
of the PostgreSQL libs on Solaris so I added -R to the link stage.
Further more the Makefile.bareos uses the source dir as it uses the libtool
we distibute with BAREOS and not the system one now. So you also need to
checkout the current bareos version and build that or if that is to much
work change the Makefile to use the stuff that is available.
G Jo
> G Jo <g.johnston <at> kainos.com> writes:
>
> > Hi Marco,
> >
> > Firstly thank you very much for taking the time to respond to my
> > posting, it’s very much appreciated.
> >
> > In turn to address the points and the queries from your reply:
> > [MVW] - I guess you build your plugin using the code at:
> https://github.com/bareos/contrib-pgsql-plugin]
> >
> > [GJ] – Yes i believe that’s what we used.
> >
> Its kind of important that you know for sure. That is the only
> plugin that will work.
>
> >
> > [GJ] – Interesting to note both that the plugin is not fully supported
> > but that you have gotten it to work previously.
> Its more that we don't check on regular basis if things still work
> and we don't support it as standard part of the BAREOS subscription.
> So any support is on best effort or as payed time and material basis.
when i try to use the bpluginfo on the fileDaemon client ip-172-31-72-8 the output is as below
[root@ip-172-31-72-8 plugins]# pwd
/usr/lib64/bareos/plugins
[root@ip-172-31-72-8 plugins]# ls -l
total 120
-rwxr-xr-x 1 root root 20056 May 20 11:36 bpipe-fd.so
-rwxr-xr-x 1 root root 100234 May 20 11:38 pgsql-fd.so
[root@ip-172-31-72-8 plugins]# /opt/bacula/bin/bpluginfo -iv /usr/lib64/bareos/plugins/bpipe-fd.so
Cannot load a plugin: -iv: cannot open shared object file: No such file or directory
[root@ip-172-31-72-8 plugins]# /opt/bacula/bin/bpluginfo -iv /usr/lib64/bareos/plugins/pgsql-fd.so
Cannot load a plugin: -iv: cannot open shared object file: No such file or directory
Obviously the files exist and there is no problem with permissions or paths. I used the file command to check shared object files and output was as follows:
[root@ip-172-31-72-8 plugins]# file bpipe-fd.so
bpipe-fd.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=0x1b8398db6fcb03dfa13ff5fe3ecdfb7b8d15bcdf, stripped
[root@ip-172-31-72-8 plugins]# file pgsql-fd.so
pgsql-fd.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=0xd65c834f3093ad70601f1fc074e7dad60ad2bbf8, not stripped
And likewise on the director server ip-172-31-21-156-dir where the local filedaemon has successfully loaded the bpipe plugin
*status client=ip-172-31-21-156-fd
Connecting to Client ip-172-31-21-156-fd at ip-172-31-21-156:9102
Plugin Info:
Plugin : bpipe-fd.so
but where the bpluginfo can read neither .so
[root@ip-172-31-21-156 bareos]# /opt/bacula/bin/bpluginfo -iv /usr/lib64/bareos/plugins/bpipe-fd.so
Cannot load a plugin: -iv: cannot open shared object file: No such file or directory
[root@ip-172-31-21-156 bareos]# /opt/bacula/bin/bpluginfo -iv /usr/lib64/bareos/plugins/pgsql-fd.so
Cannot load a plugin: -iv: cannot open shared object file: No such file or directory
[root@ip-172-31-21-156 bareos]# file /usr/lib64/bareos/plugins/pgsql-fd.so
/usr/lib64/bareos/plugins/pgsql-fd.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=0xd65c834f3093ad70601f1fc074e7dad60ad2bbf8, not stripped
[root@ip-172-31-21-156 bareos]# file /usr/lib64/bareos/plugins/bpipe-fd.so
/usr/lib64/bareos/plugins/bpipe-fd.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=0x1b8398db6fcb03dfa13ff5fe3ecdfb7b8d15bcdf, stripped
> > I’ve run out of things to try so I’d be grateful if could review my
> > logic and see if you can suggest anything.
> >
> > One thing I should mention in case it is relevant
> >
> > On the director server our bareos catalog database is a mysql database
> > containing 30 tables. However as the pgsql plugin demands a postgres
> > database it is stored in a separate postgres instance with 4 tables.
> >
> The actual meta data for the backup program and for the plugin being
> separate is fine I think its even better as I would not want to even
> merge the two even when its possible. Its also not a problem yet as you
> don't get the plugin loaded which is something not related to any config
> data being used later on.
>
> You can also run the bareos-fd in debug mode which will probably reveal
> much more about the loading of the plugins etc.
>
> e.g. something like bareos-fd -f -d 200
> will run the bareos-fd in a pretty high debug level and show what its
> doing regarding the loading of plugins.
>
> e.g.
>
> mercury-fd: plugins.c:222-0 load_plugins
> mercury-fd: fd_plugins.c:1615-0 is_plugin_compatible called
> mercury-fd: fd_plugins.c:1590-0 Loaded plugin: python-fd.so
Didn't know about the debug capability so thank you very much as just as i was beginning to tear the rest of my hair out i think we've uncovered the root cause of the problem which is plugin version incompatibility.
Full Output from /sbin/bareos-fd -f -d 200 is shown below but the key line is
ip-172-31-72-8-fd: fd_plugins.c:1632-0 Plugin version incorrect. Plugin=pgsql-fd.so wanted=9 got=7
Does that mean bareos/bacula wants version 9 of the pgsql-plugin but we only have version 7. Question is then how we match up the incompatibility? Can we configure our bareos install to ask for the earlier version?
[root@ip-172-31-72-8 bareos]# /sbin/bareos-fd -f -d 200
bareos-fd: filed_conf.c:484-0 Inserting director res: ip-172-31-72-8-mon
ip-172-31-72-8-fd: jcr.c:141-0 read_last_jobs seek to 192
ip-172-31-72-8-fd: jcr.c:148-0 Read num_items=10
ip-172-31-72-8-fd: plugins.c:222-0 load_plugins
ip-172-31-72-8-fd: plugins.c:302-0 Found plugin: name=bpipe-fd.so len=11
ip-172-31-72-8-fd: fd_plugins.c:1616-0 is_plugin_compatible called
ip-172-31-72-8-fd: plugins.c:302-0 Found plugin: name=pgsql-fd.so len=11
ip-172-31-72-8-fd: fd_plugins.c:1616-0 is_plugin_compatible called
ip-172-31-72-8-fd: fd_plugins.c:1632-0 Plugin version incorrect. Plugin=pgsql-fd.so wanted=9 got=7
ip-172-31-72-8-fd: plugins.c:106-0 Got plugin=pgsql-fd.so but not accepted.
ip-172-31-72-8-fd: fd_plugins.c:1591-0 Loaded plugin: bpipe-fd.so
ip-172-31-72-8-fd: filed.c:267-0 filed: listening on port 9102
ip-172-31-72-8-fd: bnet_server_tcp.c:166-0 Addresses host[ipv4;0.0.0.0;9102]
Once again,
Many Many Thanks for your invaluable help here.
Regards,
George
Bruno Friedmann
It seems you try to debug a bareos plugin with a bacula binary
Sound to me like a call to Hell :-)
/opt/bacula/bin/bpluginfo -iv /usr/lib64/bareos/plugins/bpipe-fd.so
At your place I would recheck both of the installation (like being sure each rpm is safely installed
in its own place, check any conflicts that could occur, check the path and ldd.conf loading etc.
I place a bet there's some trouble to try to have both bacula enterprise and bareos installed at the same time.
Some binaries have the same names etc, who's called first, etc....
(It could be possible with a personal build and configuration to separate both of them, but it's not seems the case in your case)
Be sure also if you build from source the plugin, to have a make clean before building for another product ;-)
--
Bruno Friedmann
Ioda-Net Sàrl www.ioda-net.ch
openSUSE Member & Board, fsfe fellowship
GPG KEY : D5C9B751C4653227
irc: tigerfoot
Marco van Wieringen
> [GJ] yet more unexpected errors unfortunately. :(
> when i try to use the bpluginfo on the fileDaemon client ip-172-31-72-8 the output is as below
>
> [root@ip-172-31-72-8 plugins]# pwd
> /usr/lib64/bareos/plugins
> [root@ip-172-31-72-8 plugins]# ls -l
> total 120
> -rwxr-xr-x 1 root root 20056 May 20 11:36 bpipe-fd.so
> -rwxr-xr-x 1 root root 100234 May 20 11:38 pgsql-fd.so
> [root@ip-172-31-72-8 plugins]# /opt/bacula/bin/bpluginfo -iv /usr/lib64/bareos/plugins/bpipe-fd.so
>
> Cannot load a plugin: -iv: cannot open shared object file: No such file or directory
>
> [root@ip-172-31-72-8 plugins]# /opt/bacula/bin/bpluginfo -iv /usr/lib64/bareos/plugins/pgsql-fd.so
>
> Cannot load a plugin: -iv: cannot open shared object file: No such file or directory
>
> Obviously the files exist and there is no problem with permissions or paths. I used the file command to check shared object files and output was as follows:
going to work.
The bpluginfo in bacula is kind of stupid in option parsing.
I would first uninstall all the old Bacula stuff given you have all kind
of stuff in
/opt/bacula it gives the impression the Enterprise Edition is installed
(maybe a test version)
but that is serious incompatible with BAREOS.
> [**** GJ ****]
> Didn't know about the debug capability so thank you very much as just as i was beginning to tear the rest of my hair out i think we've uncovered the root cause of the problem which is plugin version incompatibility.
>
> Full Output from /sbin/bareos-fd -f -d 200 is shown below but the key line is
>
> ip-172-31-72-8-fd: fd_plugins.c:1632-0 Plugin version incorrect. Plugin=pgsql-fd.so wanted=9 got=7
>
> Does that mean bareos/bacula wants version 9 of the pgsql-plugin but we only have version 7. Question is then how we match up the incompatibility? Can we configure our bareos install to ask for the earlier version?
>
> [root@ip-172-31-72-8 bareos]# /sbin/bareos-fd -f -d 200
> bareos-fd: filed_conf.c:484-0 Inserting director res: ip-172-31-72-8-mon
> ip-172-31-72-8-fd: jcr.c:141-0 read_last_jobs seek to 192
> ip-172-31-72-8-fd: jcr.c:148-0 Read num_items=10
> ip-172-31-72-8-fd: plugins.c:222-0 load_plugins
> ip-172-31-72-8-fd: plugins.c:302-0 Found plugin: name=bpipe-fd.so len=11
> ip-172-31-72-8-fd: fd_plugins.c:1616-0 is_plugin_compatible called
> ip-172-31-72-8-fd: plugins.c:302-0 Found plugin: name=pgsql-fd.so len=11
> pgsql plugin: version 2.2 April 2013 (c) 2011 by Inteos
> pgsql plugin: connected to Bacula version 9
> ip-172-31-72-8-fd: fd_plugins.c:1616-0 is_plugin_compatible called
> ip-172-31-72-8-fd: fd_plugins.c:1632-0 Plugin version incorrect. Plugin=pgsql-fd.so wanted=9 got=7
> ip-172-31-72-8-fd: plugins.c:106-0 Got plugin=pgsql-fd.so but not accepted.
> ip-172-31-72-8-fd: fd_plugins.c:1591-0 Loaded plugin: bpipe-fd.so
> ip-172-31-72-8-fd: filed.c:267-0 filed: listening on port 9102
> ip-172-31-72-8-fd: bnet_server_tcp.c:166-0 Addresses host[ipv4;0.0.0.0;9102]
pgsql plugin: connected to Bacula version 9 our plugin says pgsql
need you to register your events you want to get and not unlike Bacula
send you
each and every event even when you don't want them.
So see my previous mail and ditch the old stuff and use that.
G Jo
Thanks for the reply.
Along with the invaluable help provided my Marco you've helped steer us in the right direction.
Regards,
George
G Jo
Thanks for your responses over the last couple of days. The expertise you have very kindly taken the time to provide has been invaluable in moving us forward.
Just for background relevance unfortunately as you suspected we where not building either bareos or the pgsql-plugin from the sources we needed to.
From the presence of a directory /home/centos/bacula-7.0.5 it seems we built bacula-7.0.5 from source and then built the bacula version of the PGSQL Plugin. The README.md in the pgsql-plugin directory has the text “PGSQL Plugin is a Bacula File Daemon plugin”.
Anyway now that you put us on the right path I’ve being trying to follow the tip approach you outlined on a totally clean server with the correct bareos versions of the software.
Of course I’ve encountered some issues so I’m hoping that again you’ll be able to spot what’s going wrong and point us in the right direction.
To outline what I’ve done to date
1) downloaded the pgsql-plugin source using
wget https://github.com/bareos/contrib-pgsql-plugin/archive/bareos-master.zip
README.md for it has text “contrib-pgsql-plugin - PGSQL Plugin is a Bareos File Daemon plugin” so correct version confirmed.
2) Downloaded the current bareos version source code using wget https://github.com/bareos/bareos/archive/master.zip
3) We have attempted to build bareos from source using our best guess at the command options. I couldn’t locate any installation build instructions at https://github.com/bareos/bareos or in the extracted source code directory so if there is documentation available please point us towards it.
What we attempted from the extracted bareos source code directory was to configure the build to use a postgres backend db with the command
./configure --with-postgresql=/usr/pgsql-9.4
That offered the version info “configuring for Bareos 15.2.0 (25 April 2015)” along with reams of other output and completed successfully with a couple of warnings.
4) However and attempt to issue make then returns warnings and the fatal error below
Compiling passphrase.c
passphrase.c: In function ‘char* generate_crypto_passphrase(int)’:
passphrase.c:145:38: warning: array subscript has type ‘char’ [-Wchar-subscripts]
passphrase[cnt] = valid_chars[c];
^
Compiling tls_none.c
tls_none.c: In function ‘bool get_tls_verify_peer(TLS_CONTEXT*)’:
tls_none.c:71:22: error: invalid use of incomplete type ‘TLS_CONTEXT {aka struct TLS_Context}’
return (ctx) ? ctx->verify_peer : false;
^
In file included from ../lib/lib.h:45:0,
from ../include/bareos.h:152,
from tls_none.c:27:
../lib/tls.h:34:16: error: forward declaration of ‘TLS_CONTEXT {aka struct TLS_Context}’
typedef struct TLS_Context TLS_CONTEXT;
^
tls_none.c:72:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
make[1]: *** [tls_none.lo] Error 1
make[1]: Leaving directory `/root/bareos-master/src/lib'
make: *** [src/lib] Error 2
5) Can you advise on this error please? Should we be using additional switches to the configure command?
As an alternative approach you mentioned in your tip advice “
if that is to much work change the Makefile to use the stuff that is available”
I’m not sure what you mean by the “stuff that is available” so if you could expand on that please I’d be grateful.
Thank you once again.
Regards,
George
Marco van Wieringen
> Hi Marco,
>
> Thanks for your responses over the last couple of days. The expertise
> you have very kindly taken the time to provide has been invaluable in
> moving us forward.
>
> Just for background relevance unfortunately as you suspected we where
> not building either bareos or the pgsql-plugin from the sources we
> needed to.
>
> From the presence of a directory /home/centos/bacula-7.0.5 it seems we
> built bacula-7.0.5 from source and then built the bacula version of
> the PGSQL Plugin. The README.md in the pgsql-plugin directory has the
> text “PGSQL Plugin is a Bacula File Daemon plugin”.
>
Ok that indeed is never going to work. 7.0.5 is incompatible in many ways.
> Anyway now that you put us on the right path I’ve being trying to follow
> the tip approach you outlined on a totally clean server with the correct
> bareos versions of the software.
>
> Of course I’ve encountered some issues so I’m hoping that again you’ll
> be able to spot what’s going wrong and point us in the right direction.
>
> To outline what I’ve done to date
>
> 1) downloaded the pgsql-plugin source using
> wget https://github.com/bareos/contrib-pgsql-plugin/archive/bareos-master.zip
>
> README.md for it has text “contrib-pgsql-plugin - PGSQL Plugin is
> a Bareos File Daemon plugin” so correct version confirmed.
>
Ok better.
> 2) Downloaded the current bareos version source code using wget
> https://github.com/bareos/bareos/archive/master.zip
>
> 3) We have attempted to build bareos from source using our best guess at
> the command options. I couldn’t locate any installation build
> instructions at https://github.com/bareos/bareos or in the extracted
> source code directory so if there is documentation available please
> point us towards it.
There is a working specfile in platforms/packaging/bareos.spec which
is used to build the software using OpenSuse Build Service.
As you have no encryption enabled it tries building the dummy routines
which indeed have some problems. If you compile with openssl it probably
works fine. But I will push a commit fixing the above problem.
The patch is kind of obvious
commit 7f645303b655eaf56b65c041723166f647e9cf65
Author: Marco van Wieringen <marco.van...@bareos.com>
Date: Thu May 21 16:55:06 2015 +0200
Fix typo.
diff --git a/src/lib/tls_none.c b/src/lib/tls_none.c
index 2a7af12..8a666f6 100644
--- a/src/lib/tls_none.c
+++ b/src/lib/tls_none.c
@@ -68,7 +68,7 @@ void set_tls_enable(TLS_CONTEXT *ctx, bool value)
bool get_tls_verify_peer(TLS_CONTEXT *ctx)
{
- return (ctx) ? ctx->verify_peer : false;
+ return false;
}
TLS_CONNECTION *new_tls_connection(TLS_CONTEXT *ctx, int fd, bool server)
> 5) Can you advise on this error please? Should we be using additional
> switches to the configure command?
>
> As an alternative approach you mentioned in your tip advice “
> if that is to much work change the Makefile to use the stuff that is
> available”
>
> I’m not sure what you mean by the “stuff that is available” so if you
> could expand on that please I’d be grateful.
I was trying to say instead of compiling the sources you can also use
the packaged include files and libraries from the packaging but that is
probably a bit of surgery to the Makefile
G Jo
I'm not ashamed to admit that, as i'm sure you'll have guessed, a lot of the open source build world is new to me. After installing the OpenSuse buiild Service my command attempt to build the package is
osc build openSUSE_12.4 x86_64 /root/bareos-master/platforms/packaging/bareos.spec
However this returns the following error.
Error: '/root/bareos-master/platforms/packaging' is not an osc project dir or working copy
Am i along the right lines here and if not could you please tell me what command i use to build the bareos.
Thanks,
George
Marco van Wieringen
>
> Hi,
>
> I'm not ashamed to admit that, as i'm sure you'll have guessed, a lot
> of the open source build world is new to me. After installing the
> OpenSuse buiild Service my command attempt to build the package is
>
> osc build openSUSE_12.4 x86_64
/root/bareos-master/platforms/packaging/bareos.spec
>
> However this returns the following error.
>
> Error: '/root/bareos-master/platforms/packaging' is not an osc project
> dir or working copy
>
> Am i along the right lines here and if not could you please tell me what
> command i use to build the bareos.
>
You probably need to build a proper OSC project for it first which may
be somewhat overkill for what you are trying to accomplish.
Given I have scripts for setting up a build environment myself I forgot
that we have good documentation for developers for doing the setup and
compiling including a mostly accurate configure cmdline. See:
http://www.bareos.org/en/howto-contribute.html
G Jo
Thanks for pointing me towards the command line compilation documentation.
A couple of iterations of attempting to use the configure command in the Howto Compile section led to getting a version that completed successfully with the following changes:
1. Added path to postgresql base dir
--with-postgresql=/usr/pgsql-9.4 \
2. Removed the 3 switches based on errors they caused and intention to use postgres as backend db for bareos.
--enable-acl \
--with-mysql \
--with-sqlite3 \
3. This left the following configure command which completes successfully.
./configure --prefix=/usr \
--sbindir=/usr/sbin \
--with-sbin-perm=755 \
--sysconfdir=/etc/bareos \
--with-archivedir=/var/lib/bareos/storage \
--with-scriptdir=/usr/lib/bareos/scripts \
--with-plugindir=/usr/lib/bareos/plugins \
--with-working-dir=/var/lib/bareos \
--with-pid-dir=/var/lib/bareos \
--with-bsrdir=/var/lib/bareos \
--with-logdir=/var/log/bareos \
--with-subsys-dir=/var/lock \
--enable-smartalloc \
--disable-conio \
--enable-readline \
--enable-batch-insert \
--enable-dynamic-cats-backends \
--enable-bat \
--enable-traymonitor \
--enable-xattr \
--enable-scsi-crypto \
--enable-ndmp \
--enable-ipv6 \
--with-postgresql=/usr/pgsql-9.4 \
--with-tcp-wrappers \
--with-openssl \
--with-dir-user=${DIRECTOR_DAEMON_USER} \
--with-dir-group=${DAEMON_GROUP} \
--with-sd-user=${STORAGE_DAEMON_USER} \
--with-sd-group=${STORAGE_DAEMON_GROUP} \
--with-fd-user=${FILE_DAEMON_USER} \
--with-fd-group=${DAEMON_GROUP} \
--with-dir-password="Evolve" \
--with-fd-password="Evolve" \
--with-sd-password="Evolve" \
--with-mon-dir-password="Evolve" \
--with-mon-fd-password="Evolve" \
--with-mon-sd-password="Evolve" \
--with-basename="BackupServer" \
--with-hostname="BackupServer" \
--enable-includes
4. Once I added the openssl-devel yum I got past the TLS_CONTEXT error in a subsequent make.
5. Unfortunately the make step then fails with the following error
make -C src/qt-console
make[1]: Entering directory `/root/bareos-master/src/qt-console'
make[1]: *** No targets specified and no makefile found. Stop.
make[1]: Leaving directory `/root/bareos-master/src/qt-console'
make: *** [src/qt-console] Error 2
Would you have any idea what configuration I am missing here?
I had to install the PyQt4-devel.x86_64 yum to fix a previous configure step error i.e “configure: error: Unable to find suitable Qt4 installation needed by bat”
Thanks again for all the help.
Regards,
George
Marco van Wieringen
>
> Hi Marco,
>
> Thanks for pointing me towards the command line compilation documentation.
>
> A couple of iterations of attempting to use the configure command in the Howto Compile section led to getting a version that completed successfully with the following changes:
>
> 1. Added path to postgresql base dir
> --with-postgresql=/usr/pgsql-9.4 \
> 2. Removed the 3 switches based on errors they caused and intention to use postgres as backend db for bareos.
>
> --enable-acl \
> --with-mysql \
> --with-sqlite3 \
you don't need them, for gettingthing to work you can probably even do a
--enable-client-only but given you already have the restworking it might
not be wise to change to much. With these setting you are close to how
things
are normally build for most platforms.
> 5. Unfortunately the make step then fails with the following error
>
> make -C src/qt-console
> make[1]: Entering directory `/root/bareos-master/src/qt-console'
> make[1]: *** No targets specified and no makefile found. Stop.
> make[1]: Leaving directory `/root/bareos-master/src/qt-console'
> make: *** [src/qt-console] Error 2
>
> Would you have any idea what configuration I am missing here?
>
> I had to install the PyQt4-devel.x86_64 yum to fix a previous configure step error i.e “configure: error: Unable to find suitable Qt4 installation needed by bat”
G Jo
Apologies for the delay in getting back to you regarding this but i've been out of the office for most of the week.
I just wanted to let you know that i finally got bareos (and the pgsql-plugin) to build from source guided of course by all your invaluable help.
I would never have gotten this to build without your assistance so i would just like to take the opportunity to say once again thank very much for taking the time and effort to help me out.
Regards,
George