Kvm bridged mode TCP/IP network performance fix: disable ethernet tso, sg, gro, gso offload

[Lari Hotari]

Hi,

I had problems with KVM network performance and I found a solution which
I'd like to share to others.

These instructions are for Ubuntu/Debian:

I had to add these 2 lines to /etc/network/interfaces under the br0
definition:
pre-up /sbin/ethtool --offload br0 gso off tso off sg off gro off
pre-up /sbin/ethtool --offload eth0 gso off tso off sg off gro off

(in this example br0 is my bridge interface, eth0 is the network interface)
make sure you have ethtool installed, "apt-get install ethtool"

references:
https://access.redhat.com/knowledge/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Host_Configuration_and_Guest_Installation_Guide/ch10s04.html


I'm using KVM, but I believe the same applies to XEN.
http://serverfault.com/questions/272483/why-is-tcp-accept-performance-so-bad-under-xen/272496#272496

Without these settings the network traffic started crawling from
time-to-time . Now I get full performance with 1GB network (transfer
speed around 112MB/s).

For some reason ethernet offload settings cause problems at least on the
ethernet network adapter hardware I'm using (Intel Corporation 82576
Gigabit Network Connection, igb driver)

I hope this helps others that have network performance problems.


Regards,

Lari

[Guido Trotter]

Lari, thanks a lot for your suggestion.

Team: should I copy this under

http://code.google.com/p/ganeti/wiki/PerformanceTuning

for posterity?

Thanks,
Guido

[Iustin Pop]

On Mon, Oct 29, 2012 at 02:37:36PM +0100, Guido Trotter wrote:
> Lari, thanks a lot for your suggestion.
>
> Team: should I copy this under
>
> http://code.google.com/p/ganeti/wiki/PerformanceTuning
>
> for posterity?

I was thinking about the same, but I'm not sure whether there, or in the
actual docs (doc/install.rst) - as in, we have two places for
documentation, and no clear rules about what goes where…

iustin

[Guido Trotter]

I believe it goes to the wiki as it's a performance improvement trick,
but not something that is ganeti specific or that you need because of
ganeti or a specific version of it: as such I wouldn't put it in the
install guide. At most we can have a link to our wiki from the install
guide.

Thanks,
Guido

--
Guido Trotter
SRE - Corp Computing Services (aka Horsepower)
Google Germany

[Iustin Pop]

On Mon, Oct 29, 2012 at 02:41:35PM +0100, Guido Trotter wrote:
> I believe it goes to the wiki as it's a performance improvement trick,
> but not something that is ganeti specific or that you need because of
> ganeti or a specific version of it: as such I wouldn't put it in the
> install guide. At most we can have a link to our wiki from the install
> guide.

Sounds reasonable, indeed. Thanks!