bareos-fd for windows performance problems (slow throughput)

[Spiros Papageorgiou]

Hi all,

I'm havng a bareos setup, where I have a bareos-dr/sd on a centos7 VM and I'm trying to backup a folder on a windows 7 PC.

The problem is that the bareos-fd on windows doesn't seem to be able to reach over than 230Mbit/s when spooling data, which looks good, but it isn't if you consider the parameters of the problem. So the facts are:

1. bareos-sd/VM --GElink--switch---10G---Router--...10G network...----Router--10G--switch---GE---WinPC
2. Latency end-2-end is 3-4ms.
3. MTU is 9K and bareos-sd talks with winbareos-fd with a jumbo packets.
4. iperf between the bareosVM and winPC delivers 430Mbps (it is slow). A freeBSD machine reaches 880Mbps with the bareosVM.
5. bareosVM has managed to backup at a rate of 800mbps files from an another CentOS7 host, on the same infrastructure with the bareosVM (0ms latency).
6. A Windows server 2012 with winbareos-fd, on the same infra with bareos (0ms latency), also delivered 230Mbps.

So, from 5) I presume that the bareosVM can reach 800Mbps backup speed.
from 4) I presume that my network end-2-end can reach at least 800Mbps and I also have jumbo frames.

So, why the windows7 PC and the windows 2012 server PC only achieve 230Mbps when spooling data?

Has anybody managed to reach more backup speeds of more than 230Mbps when using winbareos-fd as client?

Thank you,
Spiros

PS: I have played with many windows TCP settings, in order to enhance ipv4 TCP performance, with no results.

[Marco Weiß]

Hi Spiros,

are you sure that your windows 7 PC can read more than 230Mbit/s?
Is that windows 7 and 2012 server are virtual or physical?

Regards Marco

[Spiros Papageorgiou]

Hi Marco,

The win7, is my desktop PC and it is physical. An i5-quad core machine with 8GB RAM. The hard disk can surely deliver more than 30MB/s. It is not the system disk or anything. It is a dedicated HD that only stores large files (average size 500MB) and on disk-to-disk copies it delivers about 140MB/s sustained. That's why I'm not thinking about the disk throughput.

The win2012, is a VM, that resides in the same VMWare cluster with the bareos-server (also a VM). The win2012 and bareos are on the same lan, so no router exists in between them. The switch that is between them is GE and the ethernet cards of the ESXis are also GE.

I would like to try with a more powerful machine but I don't have any handy.

Sp

[Marco Weiß]

Hi Spiros,

i think that your problem consists of more factors. One is your PC's harddisk. A single dedicated SATAII (for example seagate baracuda ST3000DM001) has an avarage read performance of 156 MB/s.
So you are limited that way. The Bareos FD have to read the files, process them and transmitting it with its own protocol. A GE Ethernet means you can not transmit more than 125 MB/s in theory.
Your iperf if i understand it tells you that you have transferred 430 mega bit per second? mbps ..
Thats only 53,75 Mega Byte per second and iperf tests are mostly a bit faster than file transfer in real life ...
So the functional chain is long, Bareos FD, HDD, Switch, Protocols, Bareos SD, Serverdisks, Drivers on windows and linux till down the Nagle algorithm which is not good implemented in every operating system and many more.

Yes 30 MB/s is slow but to optimize your setup you have to do deep performance analysis on every part that is involved..

Did you ever have done a simple file copy test from the PC to an fileserver?
Is the performance much betterß

Regards,
Marco

[Spiros Papageorgiou]

Hi Marco,

It is true what you are saying.

I have done an scp from my win7 PC (the bareos-fd) to the bareos-sd linux VM and it reached the iperf perfomance of about 400+Mbit/s. So, the whole chain of elements can reach that number. So I'm thinking that when the scp client (filezilla) is replaced with bareos-fd on my PC, and the sshd on the bareosVM is replaced with bareos-sd, I can't get more than 230Mbit/s, so it might have to do with the bareos software. That's why I asked in the forum.

In general, I'm really disappointed with the windows performance (not the bareos software) and I'm trying to find ways to improve it, but that's another topic for other forums.

So, the question remains if anybody has reached better throughputs with the bareos-fd on windows. It might give me a hint on where to look for, in order to enhance the bareos performance.

Regards,
Sp

[Marco Weiß]

Hi Spiros,

you can play with the Maximum Network Buffer Size. If you are on a fast local Ethernet you can increase that value to 65,536 bytes like it is written in the documentation it could increase the throughput five to ten times.
You can play with compression on and of. Because a windows 7 pc with a weak cpu can slow down the FD by compressing the data. If you need compression the bareos-sd can do that with autoxflate-sd.
If you have enabled TLS and the data flows through a VLAN and isn't sensitive you can try to turn it of and have a look if it changes something.
Maybe it could be that with spooling the data before writing to "tape/volume/disk" will allow bareos-sd and bareos-fd to transmit data more performant.

I don't know much about your network infrastructure but if you have more than 10 Switches and over 200 devices it could be better to put your backup infrastructure in an separated serverice VLAN. Makes only sense if your Routing device is fast enough for example a L3 switch.
Otherwise you can split your network in VLAN segments for decreasing broadcast domains. In that case your bareos have to have a IP in each VLAN. If it is possible try to connect your physical server with bareos-sd to your core switch and a direct connection from your windows PC if possible.

Just so be sure, windows 7 jumboframes are activated ;) ?


Have a nice day,
Marco