Data spooling / Shoe-shining metrics
36 views
Skip to first unread message
Christian Svensson
Jul 7, 2021, 7:37:03 AM7/7/21
to bareos-users
Good day,
I would like to add some monitoring to tape backup performance in
order to make sure I am running an optimized environment that is
well-tuned.
Is there a way, preferably programmatically using some sort of
metrics, that I can figure out if my backups are in need of faster
data spooling?
Regards,
I would like to add some monitoring to tape backup performance in
order to make sure I am running an optimized environment that is
well-tuned.
Is there a way, preferably programmatically using some sort of
metrics, that I can figure out if my backups are in need of faster
data spooling?
Regards,
Brock Palen
Jul 7, 2021, 8:12:08 AM7/7/21
to Christian Svensson, bareos-users
Assuming your using the default sg driver that most LTO drives use as opposed to something like ibm lintape,
look at the tapestat command, it should show your tape drive throughput, look for pauses or very large fluxtuations in performance (drive compression may make this more difficult). If your spool storage is fast enough you should see performance close to the drive streaming performance assuming no compression.
Brock Palen
bro...@mlds-networks.com
www.mlds-networks.com
Websites, Linux, Hosting, Joomla, Consulting
> --
> You received this message because you are subscribed to the Google Groups "bareos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/CADiuDASb9RWa0%3DH6e3HC7m%2BFf1NmiKuJejJijRZYoyvxWGmWbQ%40mail.gmail.com.
look at the tapestat command, it should show your tape drive throughput, look for pauses or very large fluxtuations in performance (drive compression may make this more difficult). If your spool storage is fast enough you should see performance close to the drive streaming performance assuming no compression.
Brock Palen
bro...@mlds-networks.com
www.mlds-networks.com
Websites, Linux, Hosting, Joomla, Consulting
> You received this message because you are subscribed to the Google Groups "bareos-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/CADiuDASb9RWa0%3DH6e3HC7m%2BFf1NmiKuJejJijRZYoyvxWGmWbQ%40mail.gmail.com.
Christian Svensson
Jul 7, 2021, 8:20:45 AM7/7/21
to Brock Palen, bareos-users
Hi Brock, thanks for your answers!
On Wed, Jul 7, 2021 at 2:12 PM Brock Palen <bro...@mlds-networks.com> wrote:
>
> Assuming your using the default sg driver that most LTO drives use as opposed to something like ibm lintape,
It does, yes.
> look at the tapestat command, it should show your tape drive throughput, look for pauses or very large fluxtuations in performance (drive compression may make this more difficult). If your spool storage is fast enough you should see performance close to the drive streaming performance assuming no compression.
Very useful, thanks! It seems tape metric support for the Prometheus
node exporter is in the works,
https://github.com/prometheus/node_exporter/pull/2044, so I will use
that for now - but the idea is exactly what you are proposing, just
through another tool.
Thanks!
On Wed, Jul 7, 2021 at 2:12 PM Brock Palen <bro...@mlds-networks.com> wrote:
>
> Assuming your using the default sg driver that most LTO drives use as opposed to something like ibm lintape,
> look at the tapestat command, it should show your tape drive throughput, look for pauses or very large fluxtuations in performance (drive compression may make this more difficult). If your spool storage is fast enough you should see performance close to the drive streaming performance assuming no compression.
node exporter is in the works,
https://github.com/prometheus/node_exporter/pull/2044, so I will use
that for now - but the idea is exactly what you are proposing, just
through another tool.
Thanks!
Christian Svensson
Jul 7, 2021, 11:48:29 AM7/7/21
to Brock Palen, bareos-users
A quick follow-up on this.
Looking at these metrics it was pretty clear that I was shoe-shining.
I have LTO-4 drives so I should be targeting 120 MB/s.
I was getting on average 55 MB/s. After adding data spooling I reached
80 MB/s. Reading the tape tuning guide from Bareos I added new pools
with a block size 1M and maximum file size of 50G which caused the
average to end up at 120 MB/s when despooling. Very nice!
Now my biggest annoyance is that spooling and despooling is not
simultaneous - but I guess if I run multiple jobs they should
interleave nicely.
I will have to test that on the next full run.
Regards,
Looking at these metrics it was pretty clear that I was shoe-shining.
I have LTO-4 drives so I should be targeting 120 MB/s.
I was getting on average 55 MB/s. After adding data spooling I reached
80 MB/s. Reading the tape tuning guide from Bareos I added new pools
with a block size 1M and maximum file size of 50G which caused the
average to end up at 120 MB/s when despooling. Very nice!
Now my biggest annoyance is that spooling and despooling is not
simultaneous - but I guess if I run multiple jobs they should
interleave nicely.
I will have to test that on the next full run.
Regards,
Reply all
Reply to author
Forward
0 new messages