Simulating a job run

[David Pearce]

I'm trying to run a job and see all of the effective config after all of the parsing of the configurations without actually running the job. How can I do this?

[Stonegate]

Am Samstag, 2. Januar 2016 19:26:26 UTC+1 schrieb David Pearce:
> I'm trying to run a job and see all of the effective config after all of the parsing of the configurations without actually running the job. How can I do this?

Hi David,

what i do is i start bconsole and then i enter into it:

1) @output /tmp/list
2) estimate job=jobname listing level=full


Then i open a 2nd ssh session to the server and do a:

tail -f /tmp/list

I can watch in realtime what it would do in that job.

However the output is pretty strange to me.
I have 1,1 TB of data to save. An estimate tells me it would save 868 GB of it (roughly 3 Million of Files less than on the HDD) and if the real full job is done it only saved 597 GB and 6 Million Files (of around 16 Millions stored on the HDD).

I dont want to rant but i spent the last 2 weeks on Bareos and i dont have a feeling that it would be a reliable backup solution at all. When it runs it backups things yes - but each time it´s different things - more or less files. And even i have some reasonable excludes, it is absolutely not logical that it saves less than half of what it should.

I created some test directories and files and it did not even save them in the Full Run. Currently it obviously tries to backup the missing rest in an incremental. Why is that? I dont know.

One big disadvantage of this software is, that it is not even telling you what it did not backup.

Every commercial (windows backup) product i know at least sent you a list of files that were not backed up after the job.

Instead of this useful information i get emails with 50 pages of what files the director had to create to populate it´s internal file storage database. I could not care less about this information. I dont want to know whats going on internally. I want to know things an admin has to know.

Well sorry - i started to rant now though ;)

Just like i am pretty tired on spending all this time on a solution i cannot trust in the end.

KR
Stoney

[Bruno Friedmann]

On Sunday 03 January 2016 03.30:16 Stonegate wrote:
> I dont want to rant but i spent the last 2 weeks on Bareos and i dont have a feeling that it would be a reliable backup solution at all. When it runs it backups things yes - but each time it´s different things - more or less files. And even i have some reasonable excludes, it is absolutely not logical that it saves less than half of what it should.
>

I'm sorry, but I can let you tell those false assumption (until they are proved by a real bug report).

It is a powerful and reliable solution.

Not less than one week ago, I migrated a DDS tape made in 2005 which has been transcoded to FileStorage media in 2010
(was with bacula) to its new location on Bareos and I can still read and restore what was stored on it.

Yes it is a very highly configurable high end software, that can easily confuse new comers.
You need to invest on reading, understanding, configuring, adapting the documentation to your environment.
It can take up to 2-3 months with the try & error methodology.
You can also create a budget to pay support from Bareos or its partners to help you making it right the first time....
Especially if you're thinking of using it professionally.

;-)


--

Bruno Friedmann
Ioda-Net Sàrl www.ioda-net.ch

openSUSE Member & Board, fsfe fellowship
GPG KEY : D5C9B751C4653227
irc: tigerfoot

[Stonegate]

Am Samstag, 2. Januar 2016 19:26:26 UTC+1 schrieb David Pearce:

> I'm trying to run a job and see all of the effective config after all of the parsing of the configurations without actually running the job. How can I do this?

Hi Bruno,

please forgive me being unhappy with the current outcome. I am just de-motivated because i spend so much time with Bareos and the results are more or less not useable.

I get totally different results from the "estimate" run compared to the real backup job. I dont know what files are being left out of a backup. The webui seems buggy (shows "loading" for almost 1 hour before it prints the files inside the backup), i am missing more than half of the files i want to backup for just no obvious reason, it takes 2 days to backup 630 GB and in fact it should backup around 1,1 TB.. it shows the last Full Backup under "Terminated Jobs" with Status OK (how can it be terminated (i didnt do that btw!) and have the status "OK" at the same time?

As you say - it is too confusing for newcomers. In the end i want to master this challenge.. but i have a feeling this will bring me some grey hairs ^^

KR
Stoney

[Bruno Friedmann]

Perhaps start simple. your published fileset few days ago, let me in the state "What he want to achieve with this?
This stuff will pretty much not work" :-)


I don't know what you have now but here an estimate and its full job are the same


2000 OK estimate files=29,006 bytes=718,481,646,405

03-Jan 15:44 oceania-dir JobId 6129: Bareos oceania-dir 14.2.6 (16Nov15):
Build OS: x86_64-suse-linux-gnu suse openSUSE 13.1 (Bottle) (x86_64)
JobId: 6129
Job: rasters.2016-01-03_12.00.00_50
Backup Level: Full
Client: "europe-fd" 14.2.6 (16Nov15) Microsoft Windows Server 2012 Standard Edition (build 9200), 64-bit,Cross-compile,Win64
FileSet: "rasters" 2013-08-16 15:02:00
Pool: "rasters" (From Job resource)
Catalog: "MyCatalog" (From Client resource)
Storage: "rasters" (From Pool resource)
Scheduled time: 03-Jan-2016 12:00:00
Start time: 03-Jan-2016 12:00:00
End time: 03-Jan-2016 15:44:27
Elapsed time: 3 hours 44 mins 27 secs
Priority: 10
FD Files Written: 29,006
SD Files Written: 29,006
FD Bytes Written: 526,141,296,389 (526.1 GB)
SD Bytes Written: 526,146,962,216 (526.1 GB)
Rate: 39068.9 KB/s
Software Compression: 26.8 % (lzo)
VSS: no
Encryption: no
Accurate: yes
Volume name(s): rasters_0225
Volume Session Id: 45
Volume Session Time: 1451213082
Last Volume Bytes: 526,573,276,442 (526.5 GB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK

Any detail has its explanation, don't give up ...
Once you will master it a bit more, you won't be able to afford any other solutions.

[Harry Martin]

I know this thread is old, but maybe this will be useful to new bareos users.

Throughout this thread, I note not one mention is made re compression. This can have significant impact in the results. An estimate of the backup will not provide a perfectly accurate size estimate if you are using compression. I suppose it could be implemented by bareos devs, but I know how long some compression algorithms can take. At least for my own purposes, a general idea of the size of the backup has been more than sufficient.