problem KVM + bond + bridge performance - ganeti

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion problem KVM + bond + bridge performance
Received-SPF: pass (google.com: domain of ultrot...@google.com designates 209.85.216.51 as permitted sender) client-ip=209.85.216.51;
MIME-Version: 1.0
In-Reply-To: <d3303d09-d017-4f08-bed1-26df5b93b...@l18g2000vbv.googlegroups.com>
References: <092c7222-6b58-4b52-8f77-0ad094d70...@p22g2000vby.googlegroups.com>
	<d3303d09-d017-4f08-bed1-26df5b93b...@l18g2000vbv.googlegroups.com>
Date: Wed, 14 Nov 2012 11:41:07 +0100
Message-ID: <CADUU3JznKUES3uTpt88LZz1VJcn9S7hoYpF5uHn+ZnKU=3s...@mail.gmail.com>
Subject: Re: problem KVM + bond + bridge performance
From: Guido Trotter <ultrot...@google.com>
To: ganeti@googlegroups.com
Content-Type: text/plain; charset=UTF-8

Great news! :)

Sorry we didn't help, but it was very hardware specific for us to know!

Guido


On Wed, Nov 14, 2012 at 11:33 AM, tschend <thomas.sch...@gmail.com> wrote:
> Hi,
>
> just for the records, i fixed the problem.
>
> The intel x520-T2 10GBe dual port card was the problem.
>
> The ixgbe driver version of kernel 3.2.0 (3.6.7-k) caused the error.
>
> I installed 3.11.33 and turned off large receive offload (ethtool -K
> ethX lro off) and it works like a chrarm now.
>
> CHeers
> Thomas
>
> On 12 Nov., 15:55, tschend <thomas.sch...@gmail.com> wrote:
>> Hi,
>>
>> i have a problem with the performance of guest network speed when
>> using bond + bridge.
>>
>> we are testing debian squeeze + kernel 3.2 form backports + kvm 1.2 +
>> ganeti 2.6.1. Works great so far.
>>
>> The setup of the bridges looks like usual
>>
>> eth4/eth5 <--> bond0 <---> gnt-br0
>>
>> Bond settings is active/passiv. Switch is not configured in any way.
>>
>> The tap of the test vm has the same mac as the bridge
>>
>> gnt-br0   Link encap:Ethernet  HWaddr 26:4b:49:53:a7:c8
>> tap0      Link encap:Ethernet  HWaddr 26:4b:49:53:a7:c8
>>
>> the active nic is marked as local in the bridge
>>
>> Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
>>
>> Bonding Mode: fault-tolerance (active-backup)
>> Primary Slave: None
>> Currently Active Slave: eth5
>> MII Status: up
>> MII Polling Interval (ms): 100
>> Up Delay (ms): 0
>> Down Delay (ms): 0
>>
>> Slave Interface: eth4
>> MII Status: up
>> Speed: 10000 Mbps
>> Duplex: full
>> Link Failure Count: 1
>> Permanent HW addr: 68:05:ca:04:21:c6
>> Slave queue ID: 0
>>
>> Slave Interface: eth5
>> MII Status: up
>> Speed: 10000 Mbps
>> Duplex: full
>> Link Failure Count: 1
>> Permanent HW addr: 68:05:ca:04:21:c7
>> Slave queue ID: 0
>>
>> brctl showmacs gnt-br0
>>
>> port no mac addr                is local?       ageing timer
>>   1     00:50:56:9c:0c:10       no                 0.00
>>   2     26:4b:49:53:a7:c8       yes                0.00
>>   1     3e:b6:00:6a:d3:da       no                64.96
>>   1     68:05:ca:04:21:c6       yes                0.00
>>   1     68:05:ca:04:22:38       no                 0.19
>>   1     68:05:ca:04:22:86       no                 0.50
>>   1     aa:00:00:1f:5c:e0       no                58.93
>>   1     aa:00:00:3d:8f:be       no                65.28
>>   2     aa:00:00:4b:f4:6c       no               119.13
>>   1     aa:00:00:4e:ff:92       no                65.09
>>   1     aa:00:00:75:c8:fa       no               288.39
>>   1     aa:00:00:8d:6b:4b       no                64.86
>>   1     aa:00:00:a3:81:14       no                64.97
>>   1     aa:00:00:b6:e1:8e       no                64.54
>>   1     aa:00:00:f5:da:bc       no                57.38
>>
>> When i run netio in the vm on the local node it looks pretty good
>>
>> TCP connection established.
>> Packet size  1k bytes:  703.86 MByte/s Tx,  1.024 GByte/s Rx.
>> Packet size  2k bytes:  1.396 GByte/s Tx,  1.362 GByte/s Rx.
>> Packet size  4k bytes:  1.503 GByte/s Tx,  1.362 GByte/s Rx.
>> Packet size  8k bytes:  1.639 GByte/s Tx,  1.318 GByte/s Rx.
>> Packet size 16k bytes:  1.734 GByte/s Tx,  1.263 GByte/s Rx.
>> Packet size 32k bytes:  1.805 GByte/s Tx,  1.277 GByte/s Rx.
>> Done.
>>
>> When i do netio to another ganeti node in the testcluster or vm on a
>> different node (the other node has no bonding!)
>>
>> TCP connection established.
>> Packet size  1k bytes:  277.63 MByte/s Tx,  6656.17 KByte/s Rx.
>> Packet size  2k bytes:  309.53 MByte/s Tx,  1159.69 KByte/s Rx.
>> Packet size  4k bytes:  323.91 MByte/s Tx,  2002.59 KByte/s Rx.
>> Packet size  8k bytes:  321.66 MByte/s Tx,  2265.05 KByte/s Rx.
>> Packet size 16k bytes:  336.52 MByte/s Tx,  4963.96 KByte/s Rx.
>> Packet size 32k bytes:  305.09 MByte/s Tx,  2134.77 KByte/s Rx.
>> Done.
>>
>> Looks like the traffic going in is being dropped or something.
>> I found something on the kvm mailinglist but this is pretty old and
>> already fixed.
>> Any ideas on this?
>>
>> Regards
>> Thomas



-- 
Guido Trotter
SRE - Corp Computing Services (aka Horsepower)
Google Germany
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
