GlusterFS Backup with Wrapper Script
Michael Roth
Die First Full Backup is working fine, but incremental backup is not working, I got the following error message.
20-Apr 15:38 bareos-dir JobId 33: Start Backup JobId 33, Job=backup-bareos-fd.2017-04-20_15.38.14_44
20-Apr 15:38 bareos-dir JobId 33: Using Device "FileStorage" to write.
20-Apr 15:38 gluster-admin-fd JobId 33: shell command: run ClientBeforeJob "/usr/lib/bareos/scripts/bareos-glusterfind-wrapper -l Incremental -s 1492695129 prebackup test-volume /tmp/bareos_test-volume"
20-Apr 15:38 gluster-admin-fd JobId 33: ClientBeforeJob: Generated output file /tmp/bareos_test-volume
20-Apr 15:38 bareos-sd JobId 33: Volume "Incremental-0005" previously written, moving to end of data.
20-Apr 15:38 bareos-sd JobId 33: Ready to append to end of Volume "Incremental-0005" size=666
20-Apr 15:38 gluster-admin-fd JobId 33: Fatal error: Failed to enable all entries in Seen bitmap, not an accurate backup ?
20-Apr 15:38 bareos-sd JobId 33: Elapsed time=00:00:01, Transfer rate=0 Bytes/second
20-Apr 15:38 gluster-admin-fd JobId 33: shell command: run ClientAfterJob "/usr/lib/bareos/scripts/bareos-glusterfind-wrapper postbackup test-volume /tmp/bareos_test-volume"
20-Apr 15:38 bareos-dir JobId 33: Error: Bareos bareos-dir 16.2.4 (01Jul16):
I would like to backup GlusterFS this ways in combination with incremental forever , that would solve a big problem for me. We have 7TB of Space with more than 15 million files to backup, there is no way to backup this with common file backup in time.
Mike Hulsman
Have you tried the " Accurate = no" in the Job configuratioen
Michael Roth
Perhaps the plugin is not compatible with GlusterFS Version 3.10
Michael Roth
Make Full Backup, Backup is ok
Make Incremental Backup, no file has added or changed, Backup failed
Make Incremental Backup, I added a file before, Backup ok
Make Incremental Backup, no file has added or changed, Backup failed
It seems that the Backup is failing if the Filelist is empty, that's strange.
Stephan Duehr
looking at the code:
https://github.com/bareos/bareos/blob/master/src/plugins/filed/gfapi-fd.c#L1535
https://github.com/bareos/bareos/blob/master/src/filed/fd_plugins.c#L2585
it calls accurate_mark_all_files_as_seen(jcr), even before reading anything
from the file list generated by glusterfind. Should work with an empty list.
So when using the gfapi-fd plugin
with glusterfind, make sure to always use
Accurate = yes
Could you verify if it works then?
Regards,
Stephan
Stephan Dühr stepha...@bareos.com
Bareos GmbH & Co. KG Phone: +49 221-630693-90
http://www.bareos.com
Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Komplementär: Bareos Verwaltungs-GmbH
Geschäftsführer: S. Dühr, M. Außendorf, J. Steffens, Philipp Storz
Mike Hulsman
JobId 31: Sending Accurate information.
JobId 31: Fatal error: fd_plugins.c:720 Command plugin"gfapi:volume=gluster\://v4s2r01.mst.proxy.nl/vol01:gffilelist=/var/lib/bareos/bareos_vol01": no type in startBackupFile packet.
This is the error when there are no files found whith an incremental glusterfind. The output file from glusterfind is empty.
When I touch 1 file, and run an Incremental again, the backup is succesful.
Stephan Duehr
I'll try to reproduce it and see how we can fix it.
Regards,
Stephan Dühr
Dakota Pilot
> Ok, thanks Mike, that's a different error message.
> I'll try to reproduce it and see how we can fix it.
I have encountered the same error, gluster 3.10, Bareos 16 latest release. Like the others a run jobname created a full backup fine because it did not see any previous backups. The scheduled incremental failed with the "... entries seen in bitmap.." error. I am using glusterfind also. My Bareos server runs the glusterfind with the bareos wrapper on the client to be backed up.
Here's the log from bareos list log.
17-May 04:00 bareos-dir JobId 162: Start Backup JobId 162, Job=ProjectsGFS.2017-05-17_04.00.00_14
17-May 04:00 bareos-dir JobId 162: Created new Volume "Inc-Projects-0045" in catalog.
17-May 04:00 bareos-dir JobId 162: Using Device "Projects" to write.
17-May 04:01 srv-1-gfs1-fd JobId 162: shell command: run ClientBeforeJob "/usr/lib/bareos/scripts/bareos-glusterfind-wrapper -l Incremental -s 1494969419 prebackup dev-projects dev-projects.out"
17-May 04:03 srv-1-gfs1-fd JobId 162: ClientBeforeJob: Generated output file /var/lib/bareos/dev-projects.out
17-May 04:03 bareos-sd JobId 162: Labeled new Volume "Inc-Projects-0045" on device "Projects" (Projects).
17-May 04:03 bareos-sd JobId 162: Wrote label to prelabeled Volume "Inc-Projects-0045" on device "Projects" (Projects)
17-May 04:05 srv-1-gfs1-fd JobId 162: Fatal error: Failed to enable all entries in Seen bitmap, not an accurate backup ?
17-May 04:05 bareos-sd JobId 162: Elapsed time=00:02:32, Transfer rate=0 Bytes/second
17-May 04:05 srv-1-gfs1-fd JobId 162: shell command: run ClientAfterJob "/usr/lib/bareos/scripts/bareos-glusterfind-wrapper postbackup dev-projects dev-projects.out"
17-May 04:05 bareos-dir JobId 162: Error: Bareos bareos-dir 16.2.4 (01Jul16):
Build OS: x86_64-redhat-linux-gnu redhat CentOS Linux release 7.0.1406 (Core)
JobId: 162
Job: ProjectsGFS.2017-05-17_04.00.00_14
Backup Level: Incremental, since=2017-05-16 17:16:59
Client: "gfssrv1" 16.2.4 (01Jul16) x86_64-redhat-linux-gnu,redhat,CentOS Linux release 7.0.1406 (Core) ,CentOS_7,x86_64
FileSet: "ProjectsGFS" 2017-05-16 00:37:59
Pool: "ProjectsIncremental" (From Job IncPool override)
Catalog: "MyCatalog" (From Client resource)
Storage: "Projects" (From Job resource)
Scheduled time: 17-May-2017 04:00:00
Start time: 17-May-2017 04:03:14
End time: 17-May-2017 04:05:52
Elapsed time: 2 mins 38 secs
Priority: 10
FD Files Written: 0
SD Files Written: 0
FD Bytes Written: 0 (0 B)
SD Bytes Written: 0 (0 B)
Rate: 0.0 KB/s
Software Compression: None
VSS: no
Encryption: no
Accurate: no
Volume name(s):
Volume Session Id: 27
Volume Session Time: 1494885040
Last Volume Bytes: 0 (0 B)
Non-fatal FD errors: 1
SD Errors: 0
FD termination status: Fatal Error
SD termination status: Running
Termination: *** Backup Error ***
Dakota Pilot
Dakota Pilot
> More information. I also had zero files so I added a file to the volume. It still failed.
Okay. I added Accurate = yes and it now does a backup. However, when I list the fileset (list files jobid=xx) I do not see any new files and I did add some.
Dakota Pilot
Still not working properly. The incs and diffs backup only backed up a directory that had not changed and missed the changed.
I then deleted the volumes from bconsole and tried a full backup again. After 12 hours I canceled it. The directory and it's children are 1.5 gig according to du -hsc and after 12 hours there was a 2 gig full backup that was growing.
Bruno Friedmann
> You received this message because you are subscribed to the Google Groups "bareos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.
> To post to this group, send email to bareos...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
Dakota Pilot
> Did you check the following estimate listing to check what dirs and files will be picked ?
snipped
I finally got incremental working with Accurate = yes despite it not working for some. Now the issue is with Full backups. After 12 hours Bareos has taken 1.5 gig of files (as shown by per du -sch) and created a 2.3 gig backup and is still running.
I'm backing up a volume on a Gluster pool to another volume on the pool.
Dakota Pilot
>
>
>
I've found that if I run estimate job=MyJobName listing from the server it fails with this message yet it runs fine if the job is run with run job=MyJobName
sing Catalog "MyCatalog"
Connecting to Client gfssrv1 at gfssrv1:9102
fd_plugins.c:1011 Command plugin "gfapi:volume=gluster\://gfssrv1/dev-projects:gffilelist=dev-projects.out" not found.
2000 OK estimate files=0 bytes=0
For some reason bconsole can not handle running the job under estimate. Nor can the WebUI display any files in the volume if data is backed up from the Gluster File Server.