Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Disconnects in double natted vlan enabled network
5 views
Skip to first unread message
Christian Schröder
Mar 25, 2013, 12:39:41 PM3/25/13
to
Hi!
After a recent upgrade to our network architecture, we experience strange network problems. It sometimes happens that when we send an email, the email client reports a connection reset after a while. Although this error does not always occur, if it occurs, it can be reproduced several times. The problem did never occur before we changed the network, so I guess that it has something to do with it.
Our network architecture is as follows: We have two dsl lines to connect to the internet. The dsl connections are pppoe connections managed by two hosts ("dsl1" and "dsl2"). Each host uses a dedicated network connection to the dsl modem and has a second network interface which is connected to our (external) network. The hosts have no firewall, but masquerade (nat) connections from our network to the dsl line. Besides the two dsl hosts, a third host ("gateway") is part of the external network. The gateway host also has two network interfaces, one to the external network and one to our internal network. A firewall is running on the gateway host which filters the traffic between the internal and the external network. This host also masquerades the connections, so we actually have double nat.
Things become complicated because most parts of the architecture described above are virtualized: We have two (physical) xen hosts ("xen1" and "xen2"). Each of them has two network cards, one connected to our switch, the other to one of the dsl modems. The dsl hosts (dsl1 and dsl2) are virtual machines, each hosted on one of the xen hosts. The external network card of each xen host is mapped on pci level into the virtual machine, so it is completely invisible to the xen host. The virtual machine opens the pppoe connection using this interface. On top of the internal network card of each xen host, two vlan interfaces are created in the xen host (vlan1 and vlan2). vlan1 is used for the internal network, vlan2 is used for the external network. Two bridge devices are created, and each gets assigned one of the vlan interfaces (br1 -> vlan1, br2 -> vlan2). The bridge device br2 is assigned to the dsl virtual machines. The gateway virtual machine gets both bridge devices to be able to perform the routing. Both vlans are configured in the switch, with vlan1 being the default, so tagged packets for vlan1 are forwarded by the switch to the remaining untagged ports to which the client machines in our internal network are connected.
Everything seems to work fine, except for the aforementioned problems. I tried to analyze the network traffic on the gateway using a packet sniffer. One thing I noticed was that the clients seem to send more packets than the server can handle. I see that the ACK packets from the server lag behind the packets sent from the client, and the "bytes in flight" value displayed by wireshark increases. At some point in time the server initiates a fast retransmit and sometimes even a normal retransmit.
Can the problems be related to the double natting? Or is it MTU related? I have read about changing the MTU in the xen guests because of the vlan tag and also disabling some xen related optimizations, but as far as I have understood, incorrect settings would lead to a complete failure of connections.
Any help would be appreciated!
Regards,
Christian
After a recent upgrade to our network architecture, we experience strange network problems. It sometimes happens that when we send an email, the email client reports a connection reset after a while. Although this error does not always occur, if it occurs, it can be reproduced several times. The problem did never occur before we changed the network, so I guess that it has something to do with it.
Our network architecture is as follows: We have two dsl lines to connect to the internet. The dsl connections are pppoe connections managed by two hosts ("dsl1" and "dsl2"). Each host uses a dedicated network connection to the dsl modem and has a second network interface which is connected to our (external) network. The hosts have no firewall, but masquerade (nat) connections from our network to the dsl line. Besides the two dsl hosts, a third host ("gateway") is part of the external network. The gateway host also has two network interfaces, one to the external network and one to our internal network. A firewall is running on the gateway host which filters the traffic between the internal and the external network. This host also masquerades the connections, so we actually have double nat.
Things become complicated because most parts of the architecture described above are virtualized: We have two (physical) xen hosts ("xen1" and "xen2"). Each of them has two network cards, one connected to our switch, the other to one of the dsl modems. The dsl hosts (dsl1 and dsl2) are virtual machines, each hosted on one of the xen hosts. The external network card of each xen host is mapped on pci level into the virtual machine, so it is completely invisible to the xen host. The virtual machine opens the pppoe connection using this interface. On top of the internal network card of each xen host, two vlan interfaces are created in the xen host (vlan1 and vlan2). vlan1 is used for the internal network, vlan2 is used for the external network. Two bridge devices are created, and each gets assigned one of the vlan interfaces (br1 -> vlan1, br2 -> vlan2). The bridge device br2 is assigned to the dsl virtual machines. The gateway virtual machine gets both bridge devices to be able to perform the routing. Both vlans are configured in the switch, with vlan1 being the default, so tagged packets for vlan1 are forwarded by the switch to the remaining untagged ports to which the client machines in our internal network are connected.
Everything seems to work fine, except for the aforementioned problems. I tried to analyze the network traffic on the gateway using a packet sniffer. One thing I noticed was that the clients seem to send more packets than the server can handle. I see that the ACK packets from the server lag behind the packets sent from the client, and the "bytes in flight" value displayed by wireshark increases. At some point in time the server initiates a fast retransmit and sometimes even a normal retransmit.
Can the problems be related to the double natting? Or is it MTU related? I have read about changing the MTU in the xen guests because of the vlan tag and also disabling some xen related optimizations, but as far as I have understood, incorrect settings would lead to a complete failure of connections.
Any help would be appreciated!
Regards,
Christian
0 new messages