Why Gluster API only on RHEL7?

[John Naggets]

Hello,

We are running a Debian-only Linux-parc and would like to use Bareos to replace our Bacula server but with GluserFS as storage.

Now I just read in the Bareos 14.2 release that Gluster API is only supported on RHEL 7. Why only RHEL? and is there a change to see Gluster API support on other Linux distributions such as Debian?

Thanks!

Regards
John.

[John Naggets]

Hello houston, any one there?

[Stephan Dühr]

You are probably referring to http://www.bareos.org/en/whats_new.html
listing Gluster API (on RHEL 7) under Cloud Storage Backends.

That's mainly for packaging reasons, RHEL 7/CentOS 7 provide the packages
required to build and run with GlusterFS API, while Debian and Ubuntu don't.
It does not make much sense for the Bareos Community project to work on
packaging based on gluster.org packages, as there are too many different
versions and it is still rarely used.

However, this does not necesarily mean it would not work with Debian, it is
possible to download the Bareos source code from github and compile it yourself
on Debian with gluster storage backend enabled.
Configure will even find out what backends it can build given the packages you
have installed so there is no need to enable something specific. If people use
the same compile options as the Debian build system uses and they have either
CEPH or Gluster installed things will be build.

And in essence you only need the gluster storage plugin which should fit in the
pre-compiled packages (as long as you use the same version of Bareos of course).

About supported in the sense of professional/commercial support for GlusterFS:
As far as I know only Red Hat provides such support with is Red Hat
Storage product.

If interested in Bareos professional support please see http://www.bareos.com/

Regards
--
Stephan Dühr stepha...@bareos.com
Bareos GmbH & Co. KG Phone: +49 221-630693-90
http://www.bareos.com

Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Komplementär: Bareos Verwaltungs-GmbH
Geschäftsführer: S. Dühr, M. Außendorf,
J. Steffens, Philipp Storz, M. v. Wieringen

[Stephan Dühr]

On 02/11/2015 07:42 PM, John Naggets wrote:
> Hello houston, any one there?
>

Why so impatient?

Purpose of this mailinglist is to discuss the usage of Bareos with the Bareos community.

Anybody else here using GlusterFS? Or Ceph? Other types of Cloud Storage?

[John Naggets]

On Wednesday, February 11, 2015 at 9:57:19 PM UTC+1, Stephan Duehr wrote:

> That's mainly for packaging reasons, RHEL 7/CentOS 7 provide the packages
> required to build and run with GlusterFS API, while Debian and Ubuntu don't.
> It does not make much sense for the Bareos Community project to work on
> packaging based on gluster.org packages, as there are too many different
> versions and it is still rarely used.

I understand that and it totally makes sense to me. I hope that Debian jessie will include GlusterFS packages then I suppose it will be possible for Bareos to include support for GlusterFS in their Debian packages.

> However, this does not necesarily mean it would not work with Debian, it is
> possible to download the Bareos source code from github and compile it yourself
> on Debian with gluster storage backend enabled.
> Configure will even find out what backends it can build given the packages you
> have installed so there is no need to enable something specific. If people use
> the same compile options as the Debian build system uses and they have either
> CEPH or Gluster installed things will be build.
>
> And in essence you only need the gluster storage plugin which should fit in the
> pre-compiled packages (as long as you use the same version of Bareos of course).

That's interesting, so you mean I could use the Bareos Debian packages and then simply compile the Gluster storage plugin from Bareos and I am ready to go? If yes how do I compile and install only the Gluster storage plugin from Bareos? On github (https://github.com/bareos) I did not see a separate repository for this plugin.

About being so impatient, in my opinion including GlusterFS support to a backup software such as Bareos is a killer and anyone should see that as exciting ;) I was happy with Bacula but for me having Gluster support is THE reason to change to Bareos.

Regards
J.

[Stephan Dühr]

On 02/12/2015 09:03 AM, John Naggets wrote:
> On Wednesday, February 11, 2015 at 9:57:19 PM UTC+1, Stephan Duehr wrote:
>
>> That's mainly for packaging reasons, RHEL 7/CentOS 7 provide the packages
>> required to build and run with GlusterFS API, while Debian and Ubuntu don't.
>> It does not make much sense for the Bareos Community project to work on
>> packaging based on gluster.org packages, as there are too many different
>> versions and it is still rarely used.
>
> I understand that and it totally makes sense to me. I hope that Debian jessie will include GlusterFS packages then I suppose it will be possible for Bareos to include support for GlusterFS in their Debian packages.
>
>
>> However, this does not necesarily mean it would not work with Debian, it is
>> possible to download the Bareos source code from github and compile it yourself
>> on Debian with gluster storage backend enabled.
>> Configure will even find out what backends it can build given the packages you
>> have installed so there is no need to enable something specific. If people use
>> the same compile options as the Debian build system uses and they have either
>> CEPH or Gluster installed things will be build.
>>
>> And in essence you only need the gluster storage plugin which should fit in the
>> pre-compiled packages (as long as you use the same version of Bareos of course).
>
> That's interesting, so you mean I could use the Bareos Debian packages and then simply compile the Gluster storage plugin from Bareos and I am ready to go? If yes how do I compile and install only the Gluster storage plugin from Bareos? On github (https://github.com/bareos) I did not see a separate repository for this plugin.

There's no separate repository, the gfapi storage backend is in the direcotory
src/stored/backends of the repository https://github.com/bareos/bareos

You'll need to compile everything, look for "Howto compile" in
http://www.bareos.org/en/howto-contribute.html
but as you'll probably build for 14.2, look for configure options
at https://github.com/bareos/bareos/blob/bareos-14.2/debian/rules

If using Bareos 14.2 packages, you must also do
git checkout origin/bareos-14.2
after git clone https://github.com/bareos/bareos.git

If all worked well, you'll find
libbareossd-gfapi-14.2.2.so
that must be copied to /usr/lib/bareos/backends/
and there must be the symlink created:
ln -s libbareossd-gfapi libbareossd-gfapi-14.2.2.so

>
> About being so impatient, in my opinion including GlusterFS support to a backup software such as Bareos is a killer and anyone should see that as exciting ;) I was happy with Bacula but for me having Gluster support is THE reason to change to Bareos.

Although any should work by using gluster fuse mount, better performance is expected by
native gfapi access due to less overhead than fuse.

Regards,

[John Naggets]

On Thursday, February 12, 2015 at 10:40:52 AM UTC+1, Stephan Duehr wrote:

> There's no separate repository, the gfapi storage backend is in the direcotory
> src/stored/backends of the repository https://github.com/bareos/bareos
>
> You'll need to compile everything, look for "Howto compile" in
> http://www.bareos.org/en/howto-contribute.html
> but as you'll probably build for 14.2, look for configure options
> at https://github.com/bareos/bareos/blob/bareos-14.2/debian/rules
>
> If using Bareos 14.2 packages, you must also do
> git checkout origin/bareos-14.2
> after git clone https://github.com/bareos/bareos.git
>
> If all worked well, you'll find
> libbareossd-gfapi-14.2.2.so
> that must be copied to /usr/lib/bareos/backends/
> and there must be the symlink created:
> ln -s libbareossd-gfapi libbareossd-gfapi-14.2.2.so

Thanks for the pointer to the documentation. I followed these instructions and unfortunately it looks like the configure does not find Gluster's header files, here is an extract of the config.log file:

| #include <api/glfs.h>
configure:31521: result: no
configure:31521: checking api/glfs.h presence
configure:31521: gcc -E -g -O2 conftest.c
conftest.c:207:22: fatal error: api/glfs.h: No such file or directory

I used the following configure parameters based on the 14.2 Debian rules file:

CONFIGURE_COMMON="--with-sbin-perm=755 --libdir=/usr/lib/bareos --sysconfdir=/etc/bareos --with-archivedir=/var/lib/bareos/storage --with-backenddir=/usr/lib/bareos/backends --
with-configtemplatedir=/usr/lib/bareos/defaultconfigs --with-scriptdir=/usr/lib/bareos/scripts --with-plugindir=/usr/lib/bareos/plugins --with-working-dir=/var/lib/bareos --wit
h-pid-dir=/var/lib/bareos --with-bsrdir=/var/lib/bareos --with-logdir=/var/log/bareos --with-subsys-dir=/var/lock --enable-smartalloc --disable-conio --enable-readline --enable
-batch-insert --enable-dynamic-debian-package-list --enable-dynamic-cats-backends --enable-dynamic-storage-backends --enable-acl --enable-xattr --enable-scsi-crypto --enable-lm
db --enable-ndmp --enable-ipv6 --with-postgresql --with-mysql --with-sqlite3 --with-tcp-wrappers --with-openssl --with-glusterfs=/usr/include/glustserfs --with-dir-user=$DIRECT
OR_DAEMON_USER --with-dir-group=$DAEMON_GROUP --with-sd-user=$STORAGE_DAEMON_USER --with-sd-group=$STORAGE_DAEMON_GROUP --with-fd-user=$FILE_DAEMON_USER --with-fd-group=$DAEMON
_GROUP --with-dir-password="XXX_REPLACE_WITH_DIRECTOR_PASSWORD_XXX" --with-fd-password="XXX_REPLACE_WITH_CLIENT_PASSWORD_XXX" --with-sd-password="XXX_REPLACE_WITH_STORAGE_PASSW
ORD_XXX" --with-mon-dir-password="XXX_REPLACE_WITH_DIRECTOR_MONITOR_PASSWORD_XXX" --with-mon-fd-password="XXX_REPLACE_WITH_CLIENT_MONITOR_PASSWORD_XXX" --with-mon-sd-password=
"XXX_REPLACE_WITH_STORAGE_MONITOR_PASSWORD_XXX" --with-basename="XXX_REPLACE_WITH_LOCAL_HOSTNAME_XXX" --with-hostname="XXX_REPLACE_WITH_LOCAL_HOSTNAME_XXX" --enable-includes --
disable-rpath"

Note here that I added --with-glusterfs=/usr/include/glustserfs for Glustser. I am using the official Gluster 3.5 Debian repository for wheezy and installed the glusterfs-client and glusterfs-server packages.

You have any ideas what I am doing wrong here?

> Although any should work by using gluster fuse mount, better performance is expected by
> native gfapi access due to less overhead than fuse.

Mounting a GlusterFS volume using FUSE is simply horrible, you do not want to do that to yourself or anyone... Performance is a disaster. You can watch the context switches on your machine going nuts during a simple file copy, never saw that before. You definitely want to use API access and avoid FUSE by all means. NFS is not so much better neither except that it handles a large amount of small files slightly better.

[Stephan Dühr]

looks like api/glfs.h are in the glusterfs-common deb packages
on http://download.gluster.org/pub/gluster/glusterfs/
libgfapi.so is also in glusterfs-common, so you'll also need it at runtime.

>
>> Although any should work by using gluster fuse mount, better performance is expected by
>> native gfapi access due to less overhead than fuse.
>
> Mounting a GlusterFS volume using FUSE is simply horrible, you do not want to do that to yourself or anyone... Performance is a disaster. You can watch the context switches on your machine going nuts during a simple file copy, never saw that before. You definitely want to use API access and avoid FUSE by all means. NFS is not so much better neither except that it handles a large amount of small files slightly better.
>

[John Naggets]

On Thursday, February 12, 2015 at 4:08:38 PM UTC+1, Stephan Duehr wrote:
> looks like api/glfs.h are in the glusterfs-common deb packages
> on http://download.gluster.org/pub/gluster/glusterfs/
> libgfapi.so is also in glusterfs-common, so you'll also need it at runtime.

The glusterfs-common package is a dependency of glusterfs-client so I got that one installed. The problem here was the "--enable-includes" configure parameter, I removed it and then the configure script found my api/glfs.h header file. Now I was able to run the make but the libbareossd-gfapi-14.2.3.so library did not get compiled on it's own so I had to go into the src/stored/backends and type "make" manually.

So far so I good I still have one issue left: why do I get version 14.2.3 of Bareos, although I did a "git checkout origin/bareos-14.2" as you recommended. Any ideas?

[Stephan Dühr]

That should not matter, the glusterfs storage backend code has not
been changed since 14.2.2 release.

Four your build, you could also use the source archive from
https://github.com/bareos/bareos/releases/tag/Release%2F14.2.2

For changes since then see commits for the 14.2 branch at
https://github.com/bareos/bareos/commits/bareos-14.2

One of the values of a bareos.com subscription are minor version
update binary packages beeing provided to subscription customers,
while bareos.org community packages are major version only
plus experimental nightly build packages based on github master branch.
Source code is always updated on github, that's why the
minor version number was increased.

Regards

[John Naggets]

I have a small issue here: I have installed Bareos 14.2.2 for Debian 7 from the Bareos official repository and compiled the source release of 14.2.2 for the stored GFAPI library but unfortunately once this library copied to the /usr/lib/bareos/backends directory as recommended it does not find the main lib bareos library as you can see here:

ldd /usr/lib/bareos/backends/libbareossd-gfapi-14.2.2.so
linux-vdso.so.1 => (0x00007fffd9fa4000)
libgfapi.so.0 => /usr/lib/x86_64-linux-gnu/libgfapi.so.0 (0x00007f0e29de5000)
libbareos-14.2.2.so => not found
libwrap.so.0 => /lib/x86_64-linux-gnu/libwrap.so.0 (0x00007f0e29bd9000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f0e299c2000)
libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f0e29762000)
libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f0e29369000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0e2914d000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f0e28f49000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f0e28c41000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f0e289bf000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0e28633000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f0e2841c000)
libglusterfs.so.0 => /usr/lib/x86_64-linux-gnu/libglusterfs.so.0 (0x00007f0e2817a000)
libgfrpc.so.0 => /usr/lib/x86_64-linux-gnu/libgfrpc.so.0 (0x00007f0e27f5f000)
libgfxdr.so.0 => /usr/lib/x86_64-linux-gnu/libgfxdr.so.0 (0x00007f0e27d45000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f0e27b3d000)
libnsl.so.1 => /lib/x86_64-linux-gnu/libnsl.so.1 (0x00007f0e27925000)
/lib64/ld-linux-x86-64.so.2 (0x00007f0e2a20c000)

libbareos-14.2.2.so is actually located in /usr/lib/bareos. Am I missing something here?

[John Naggets]

I managed resolve the ldd libary issue by adding the path /usr/lib/bareos to the file /etc/ld.so.conf.d/bareos.conf and re-running ldconfig.

Now to test I am using the default bareos dir/sd/fd config files but simply added the following Device to the sd config:

Device {
Name = GlusterStorage
Archive Device = gluster://192.168.10.123/backup/bareos
Device Type = gfapi
Media Type = GlusterFile
Label Media = yes
Random Access = yes
Automatic Mount = yes
Removable Media = no
Always Open = no
}

Then in the director config file I changed the File Storage to use the GlusterStorage Device instead of the FileStorage.

That's all I did but after changing this I can not restart the bacula storage daemon. I get the following mail warning:

Subject: Bareos NONE traceback of bareos-sd on myserver
No debugger available on this platform, please install one to get proper stack trace

Any ideas what's wrong here?

[Stephan Dühr]

On 02/16/2015 07:18 PM, John Naggets wrote:
> I managed resolve the ldd libary issue by adding the path /usr/lib/bareos to the file /etc/ld.so.conf.d/bareos.conf and re-running ldconfig.

normally that's not necesary, should be fixable at compile time

> Now to test I am using the default bareos dir/sd/fd config files but simply added the following Device to the sd config:
>
> Device {
> Name = GlusterStorage
> Archive Device = gluster://192.168.10.123/backup/bareos
> Device Type = gfapi
> Media Type = GlusterFile
> Label Media = yes
> Random Access = yes
> Automatic Mount = yes
> Removable Media = no
> Always Open = no
> }
>
> Then in the director config file I changed the File Storage to use the GlusterStorage Device instead of the FileStorage.
>
> That's all I did but after changing this I can not restart the bacula storage daemon. I get the following mail warning:
>
> Subject: Bareos NONE traceback of bareos-sd on myserver
> No debugger available on this platform, please install one to get proper stack trace
>
> Any ideas what's wrong here?
>

no, but to see what's happening, it's always a good idea to first
try to run bareos-sd in foreground with a debug-level >= 100
for example:

chsh -s /bin/bash bareos
service bareos-sd stop
su - bareos
/usr/sbin/bareos-sd -f -d 100

[John Naggets]

On Monday, February 16, 2015 at 9:28:56 PM UTC+1, Stephan Duehr wrote:

> no, but to see what's happening, it's always a good idea to first
> try to run bareos-sd in foreground with a debug-level >= 100
> for example:
>
> chsh -s /bin/bash bareos
> service bareos-sd stop
> su - bareos
> /usr/sbin/bareos-sd -f -d 100

Thanks for the tip. I now ran bareos-sd in debugging mode as recommended and here would be the output:

bareos-srv-sd: jcr.c:141-0 read_last_jobs seek to 192
bareos-srv-sd: jcr.c:148-0 Read num_items=1
bareos-srv-sd: crypto_cache.c:55-0 Could not open crypto cache file. /var/lib/bareos/bareos-sd.9103.cryptoc ERR=No such file or directory
bareos-srv-sd: bnet_server_tcp.c:166-0 Addresses host[ipv4;0.0.0.0;9103]
bareos-srv-sd: stored.c:609-0 calling init_dev /var/lib/bareos/storage
bareos-srv-sd: dev.c:377-0 init_dev: tape=0 dev_name=/var/lib/bareos/storage
bareos-srv-sd: dev.c:379-0 dev=/var/lib/bareos/storage dev_max_bs=0 max_bs=0
bareos-srv-sd: stored.c:611-0 SD init done /var/lib/bareos/storage
bareos-srv-sd: block.c:127-0 created new block of blocksize 64512 (dev->device->label_block_size) as dev->max_block_size is zero
bareos-srv-sd: stored.c:609-0 calling init_dev gluster://192.168.10.198/backup/bareos
bareos-srv-sd: sd_backends.c:111-0 init_backend_dev: testing backend /usr/lib/bareos/backends/libbareossd-gfapi.so
17-Feb 10:50 bareos-srv-sd ABORTING due to ERROR
Unable to load any shared library for libbareossd-gfapi.so
BAREOS forced SEG FAULT to obtain traceback.
BAREOS interrupted by signal 11: Segmentation violation
Kaboom! bareos-sd, bareos-srv-sd got signal 11 - Segmentation violation. Attempting traceback.
Kaboom! exepath=/usr/sbin/
Calling: /usr/sbin/btraceback /usr/sbin/bareos-sd 8335 /var/lib/bareos
It looks like the traceback worked...

Looks like a shared library issue... just in case here is the content of the bareos backends lib directory:

$ ls -la /usr/lib/bareos/backends/

total 96
drwxr-xr-x 2 root root 4096 Feb 16 17:45 .
drwxr-xr-x 6 root root 4096 Feb 16 17:39 ..
-rw-r--r-- 1 root root 32848 Dec 31 19:17 libbareoscats-postgresql-14.2.2.so
lrwxrwxrwx 1 root root 34 Dec 31 19:16 libbareoscats-postgresql.so -> libbareoscats-postgresql-14.2.2.so
-rwxr-xr-x 1 root root 51377 Feb 16 17:44 libbareossd-gfapi-14.2.2.so
lrwxrwxrwx 1 root root 27 Feb 16 17:45 libbareossd-gfapi.so -> libbareossd-gfapi-14.2.2.so

[Stephan Dühr]

Don't know what went wrong there, with segfaults it's always difficult
to find out.

Trying a different approach by doing a rebuild as described at
https://www.debian-administration.org/article/20/Rebuilding_Debian_packages

Like this, it worked for me, using gluster.org 3.6.2 package as my test-Server
is also on that version. No need for any ld.so.conf changes:

root@deb7tst01:~# wget -O - http://download.gluster.org/pub/gluster/glusterfs/3.6/3.6.2/Debian/wheezy/pubkey.gpg | apt-key add -

root@deb7tst01:~# echo "deb http://download.gluster.org/pub/gluster/glusterfs/3.6/3.6.2/Debian/wheezy/apt wheezy main" > /etc/apt/sources.list.d/gluster.list

root@deb7tst01:~# apt-get update

root@deb7tst01:~# apt-get install glusterfs-client

root@deb7tst01:~# echo "deb-src http://download.bareos.org/bareos/release/14.2/Debian_7.0/ /" >> /etc/apt/sources.list.d/bareos.list

root@deb7tst01:~# cat /etc/apt/sources.list.d/bareos.list
deb http://download.bareos.org/bareos/release/14.2/Debian_7.0/ /
deb-src http://download.bareos.org/bareos/release/14.2/Debian_7.0/ /

root@deb7tst01:~# apt-get build-dep bareos

root@deb7tst01:~# apt-get install devscripts

stephand@deb7tst01:~/work/src$ apt-get source bareos
stephand@deb7tst01:~/work/src$ cd bareos-14.2.2
stephand@deb7tst01:~/work/src/bareos-14.2.2$ debuild -us -uc

root@deb7tst01:~# cp /home/stephand/work/src/bareos-14.2.2/debian/tmp/usr/lib/bareos/backends/libbareossd-gfapi-14.2.2.so /usr/lib/bareos/backends/
root@deb7tst01:~# cd /usr/lib/bareos/backends/
root@deb7tst01:/usr/lib/bareos/backends# ln -s libbareossd-gfapi-14.2.2.so libbareossd-gfapi.so

[John Naggets]

On Tuesday, February 17, 2015 at 11:04:26 PM UTC+1, Stephan Duehr wrote:

> Trying a different approach by doing a rebuild as described at
> https://www.debian-administration.org/article/20/Rebuilding_Debian_packages

Thanks, building the package from your Bareos Debian's source package worked nicely and bacula-sd is now running.

Now to the next step: running a test backup with a default installation. Somehow Bareos gets stuck at "waiting for a mount request" when running a test backup using GlusterFS storage. As you can see here an extract from the status dir:

Running Jobs:
Console connected at 18-Feb-15 11:56
JobId Level Name Status
======================================================================
10 Increme BackupClient1.2015-02-18_11.57.01_12 is waiting for a mount request
====


and an extract from "status storage":

Device "FileStorage" (/var/lib/bareos/storage) is not open.
==

Device "GlusterStorage" (gluster://192.168.10.123/backup) is not open.
Device is BLOCKED waiting for mount of volume "Incremental-0002",
Pool: Incremental
Media type: GlusterFile

Any ideas why it can't open?

btw I had to first allow insecure connections to the GlusterFS with the "server.allow-insecure: on" parameter on the backup volume and "option rpc-auth-allow-insecure on" option of glusterd.

[John Naggets]

I found my issue: I had to change the permissions of my GlusterFS volume for user bareos and group bareos. Now it works. I will do some tests and keep you posted how it goes.

[John Naggets]

For your information, I now had Bareos perform a full backup of a real production server using the GlusterFS storage backend. This is really a great feature and I am looking forward to put this into production.

I even used the bacula-fd client on the server which was already running as I already have a Bacula server. Even this worked without having to install the Bareos FD client. Really pleased about that too. At the end I backed up 121 GB at a rate of 17.51 MB/s which is not bad considered this server has SATA disks. The Gluster server during the backup performed very silently, low CPU usage and especially 1/4 of context switches in comparison to the crappy FUSE client.

Thumbs up to Bareos! Now I will start scaling out by adding more clients and hopefully the Gluster storage backend also allows for concurrent backups.

[Stephan Dühr]

Good to know that it works for you now.

Yes, you are right, that is necesary because bareos-sd runs as user bareos:
gluster volume set VOLNAME server.allow-insecure on
and adding the line
option rpc-auth-allow-insecure on
to /etc/glusterfs/glusterd.vol
and restart glusterd:
service glusterd restart

Without these settings, there will be error messages like
E [rpcsvc.c:617:rpcsvc_handle_rpc_call] 0-rpc-service: Request received from non-privileged port. Failing request
in /var/log/glusterfs/etc-glusterfs-glusterd.vol.log

Not sure, but I don't think that this makes it much more insecure, as it
could be hacked from privileged ports as well. There may be other options
for increasing security or restricting access.

There are also other reasons that require these setting, for example see
https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3/html/Administration_Guide/chap-Managing_Snapshots.html#Prerequisites37

Some more hints here, just in case somebody else wants to test it:

To create the directory for bareos-sd in the glusterfs Volume,
a FUSE mount can be used, eg.

mount -t glusterfs c6glfs01.stephand.kvm:/repvol1 /mnt
mkdir -p /mnt/deb7tst01/bareos-storage
chown bareos.bareos /mnt/deb7tst01/bareos-storage
chmod 770 /mnt/deb7tst01/bareos-storage

It's easiest to do this from the system that runs bareos-sd, because
the numerical uid/gid must match:

root@deb7tst01:~# ls -la /mnt/deb7tst01/
total 0
drwxr-xr-x 3 root root 27 Feb 17 22:12 .
drwxr-xr-x 8 root root 123 Feb 17 22:11 ..
drwxrwx--- 2 bareos bareos 22 Feb 17 22:25 bareos-storage
root@deb7tst01:~# ls -lan /mnt/deb7tst01/
total 0
drwxr-xr-x 3 0 0 27 Feb 17 22:12 .
drwxr-xr-x 8 0 0 123 Feb 17 22:11 ..
drwxrwx--- 2 104 106 22 Feb 17 22:25 bareos-storage
root@deb7tst01:~# id bareos
uid=104(bareos) gid=106(bareos) groups=106(bareos),6(disk),26(tape)

umount /mnt

bareos-sd config example:

#
# Preparations:
#
# include this configuration file in bareos-sd.conf by
# @/etc/bareos/bareos-sd.d/device-gluster.conf
#

Device {
Name = GlusterStorage
Archive Device = gluster://192.168.122.47/repvol1/deb7tst01/bareos-storage

Device Type = gfapi
Media Type = GlusterFile
Label Media = yes
Random Access = yes
Automatic Mount = yes
Removable Media = no
Always Open = no
}

and add configuration in bareos-dir.conf:

Storage {
Name = GlusterFile
# Do not use "localhost" here
Address = deb7tst01.stephand.kvm # N.B. Use a fully qualified name here
Password = "xJe56FBrbUnKJFR0/Z1X/HCm3NiaJ0XwK84XFv+WtQ3Y"
Device = GlusterStorage
Media Type = GlusterFile
}

The Name is arbitrary, but Media Type must be GlusterFile.

To run a test, in bconsole do:

run job=BackupClient1 level=Full storage=GlusterFile

Regards,

[John Naggets]

Thanks for your precisions Stephan, this is exactly what I did. Allowing insecure connections in GlusterFS is not as bad as it sounds and is also required for example when you have virtualisation servers such as QEMU/KVM in order to access the images stored on a GlusterFS volume through the libgfapi.

I will now play around with Bareos to scale it up for concurrent connections with one volume/pool per server and see how that goes.

Regards,
J.