Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
ganeti
Home
+ new post
About this group
Join this group
Strange networking issue...
Options
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
   6 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Timothy Scoppetta  
View profile  
 More options Aug 24 2012, 5:12 pm
From: Timothy Scoppetta <t...@eecs.berkeley.edu>
Date: Fri, 24 Aug 2012 14:12:39 -0700
Local: Fri, Aug 24 2012 5:12 pm
Subject: Strange networking issue...
Print | Individual message | Show original | Report this message | Find messages by this author

Hey Folks,

I;m have a strange networking issue that I was hoping you guys could help
with. Or maybe point me in the right direction. I have a very simple bridge
set up on my vmhosts:

~# brctl show br0
bridge name bridge id STP enabled interfaces
br0 8000.bc305bedf538 yes eth0
 eth1
eth2

Nothing fancy going on there, no NAT configured, etc. However I'm noticing
that on some hosts (specifically windows hosts) incoming traffic is having
it's source IP address rewritten to be that of the vmhost as opposed to the
original sender of the packet. I can't imagine WHY it would be doing this.
I have no NAT configured or anything like that, just a very basic bridge.
However, I came across it while configuring zabbix and got the following
error:

260:20120823:142808.938 Listener error: Connection from [128.32.224.132]
rejected. Allowed server is [imon-0.EECS.Berkeley.EDU,
imon-1.EECS.Berkeley.EDU,imon-2.EECS.Berkeley.EDU].

This (the error message, not the issue) makes perfect sense because that's
the IP address of the vmhost itself. I went ahead and captured some traffic
and it appears that's exactly whats happening. I'm having trouble tacking
down WHY that ip address is getting changed at all. I know this is a bit
vague but... I'm not quite sure where else to look at this point.

Any ideas?
Tim

--

Timothy Scoppetta
UC Berkeley

E: t...@eecs.berkeley.edu
P: 845-459-3002


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Guido Trotter  
View profile   Translate to Translated (View Original)
 More options Aug 24 2012, 5:29 pm
From: Guido Trotter <ultrot...@gmail.com>
Date: Fri, 24 Aug 2012 22:29:04 +0100
Local: Fri, Aug 24 2012 5:29 pm
Subject: Re: Strange networking issue...
Print | Individual message | Show original | Report this message | Find messages by this author

Hi Timothy,

Could you please show us a brctl show on the node on which the Windows
System is up, and perhaps an ip link list, and an iptables -t nat -L just
to be sure?

Also can you show a gnt-instance info, and the setting for proxy arp in
/proc/sys/net/... ?

Thanks,

Guido
On 24 Aug 2012 22:13, "Timothy Scoppetta" <t...@eecs.berkeley.edu> wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Timothy Scoppetta  
View profile  
 More options Aug 24 2012, 5:42 pm
From: Timothy Scoppetta <t...@eecs.berkeley.edu>
Date: Fri, 24 Aug 2012 14:42:46 -0700
Local: Fri, Aug 24 2012 5:42 pm
Subject: Re: Strange networking issue...
Print | Individual message | Show original | Report this message | Find messages by this author

So, while gathering up this information I realized something. Apparently
the windows host using tap0? I'm not entirely sure why (as the device is
listed as br0 in gnt-instance info) but if I take the tap interface out of
the bridge things come tumbling down. is this the expected behavior for a
windows host? I've never seen my linux instances behave like this.

All info requested below, let me know if I missed anything or if anything
looks out of place...

vmhost-0:~# brctl show
bridge name bridge id STP enabled interfaces
br0 8000.b221a3821727 yes eth0
 eth1
eth2
 eth3
tap0
 tap1

vmhost-0:~# ip link list
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN mode
DEFAULT
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0
state UP mode DEFAULT qlen 1000
    link/ether bc:30:5b:ee:06:30 brd ff:ff:ff:ff:ff:ff
3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master br0
state DOWN mode DEFAULT qlen 1000
    link/ether bc:30:5b:ee:06:31 brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master br0
state DOWN mode DEFAULT qlen 1000
    link/ether bc:30:5b:ee:06:32 brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0
state UP mode DEFAULT qlen 1000
    link/ether bc:30:5b:ee:06:33 brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
qlen 1000
    link/ether a0:36:9f:07:12:c4 brd ff:ff:ff:ff:ff:ff
7: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
qlen 1000
    link/ether a0:36:9f:07:12:c6 brd ff:ff:ff:ff:ff:ff
8: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
mode DEFAULT
    link/ether bc:30:5b:ee:06:30 brd ff:ff:ff:ff:ff:ff
10: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state
UNKNOWN mode DEFAULT qlen 500
    link/ether b2:21:a3:82:17:27 brd ff:ff:ff:ff:ff:ff
11: tap1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state
UNKNOWN mode DEFAULT qlen 500
    link/ether ca:ef:89:13:00:da brd ff:ff:ff:ff:ff:ff

vmhost-0:~# iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  anywhere             anywhere

vmhost-1:~# gnt-instance info dc-dev
Instance name: dc-dev.eecs.berkeley.edu
UUID: 0033c0ca-5eb9-40f2-b9f9-e828e678bdd9
Serial number: 10
Creation time: 2012-08-15 16:19:20
Modification time: 2012-08-20 15:14:27
State: configured to be up, actual state is up
  Nodes:
    - primary: vmhost-0.eecs.berkeley.edu
      group: default (UUID 6e1b2abd-69c2-4b31-b2df-f96aa5122d76)
    - secondaries: vmhost-1.eecs.berkeley.edu (group default, group UUID
6e1b2abd-69c2-4b31-b2df-f96aa5122d76)
  Operating system: image+default
  Allocated network port: 11028
  Hypervisor: kvm
    - console connection: vnc to 127.0.0.1:11028 (node
vmhost-0.eecs.berkeley.edu) (display 5128)
    - acpi: default (True)
    - boot_order: default (disk)
    - cdrom2_image_path: default ()
    - cdrom_disk_type: default ()
    - cdrom_image_path: default ()
    - cpu_mask: default (all)
    - disk_cache: default (default)
    - disk_type: default (paravirtual)
    - floppy_image_path: default ()
    - initrd_path: default ()
    - kernel_args: default (ro)
    - kernel_path: default ()
    - keymap: default ()
    - kvm_flag: default ()
    - mem_path: default ()
    - migration_downtime: default (30)
    - nic_type: default (paravirtual)
    - reboot_behavior: default (reboot)
    - root_path: default (/dev/vda1)
    - security_domain: default ()
    - security_model: default (none)
    - serial_console: default (True)
    - spice_bind: default ()
    - spice_image_compression: default ()
    - spice_ip_version: default (0)
    - spice_jpeg_wan_compression: default ()
    - spice_password_file: default ()
    - spice_playback_compression: default (True)
    - spice_streaming_video: default ()
    - spice_tls_ciphers: default (HIGH:-DES:-3DES:-EXPORT:-ADH)
    - spice_use_tls: default (False)
    - spice_use_vdagent: default (True)
    - spice_zlib_glz_wan_compression: default ()
    - usb_mouse: default ()
    - use_chroot: default (False)
    - use_localtime: default (False)
    - vhost_net: default (False)
    - vnc_bind_address: 127.0.0.1
    - vnc_password_file: default ()
    - vnc_tls: default (False)
    - vnc_x509_path: default ()
    - vnc_x509_verify: default (False)
  Hardware:
    - always_failover: default (False)
    - auto_balance: default (True)
    - maxmem: 1024
    - memory: default (1024)
    - minmem: 1024
    - spindle_use: default (1)
    - vcpus: default (1)
    - NICs:
      - nic/0: MAC: AA:00:00:71:FF:D2, IP: None, mode: bridged, link: br0
  Disk template: drbd
  Disks:
    - disk/0: drbd8, size 20.0G
      access mode:  rw
      nodeA:        vmhost-0.eecs.berkeley.edu, minor=2
      nodeB:        vmhost-1.eecs.berkeley.edu, minor=2
      port:         11030
      auth key:     947f50988193e7217ef1f4f630c6454ddc1ab392
      on primary:   /dev/drbd2 (147:2) in sync, status ok
      on secondary: /dev/drbd2 (147:2) in sync, status ok
      child devices:
        - child 0: lvm, size 20.0G
          logical_id:
ganeti/b6cdae80-3134-4ecd-a426-a9519816edf4.disk0_data
          on primary:
/dev/ganeti/b6cdae80-3134-4ecd-a426-a9519816edf4.disk0_data (254:4)
          on secondary:
/dev/ganeti/b6cdae80-3134-4ecd-a426-a9519816edf4.disk0_data (254:4)
        - child 1: lvm, size 128M
          logical_id:
ganeti/b6cdae80-3134-4ecd-a426-a9519816edf4.disk0_meta
          on primary:
/dev/ganeti/b6cdae80-3134-4ecd-a426-a9519816edf4.disk0_meta (254:5)
          on secondary:
/dev/ganeti/b6cdae80-3134-4ecd-a426-a9519816edf4.disk0_meta (254:5)
    - disk/1: drbd8, size 20.0G
      access mode:  rw
      nodeA:        vmhost-0.eecs.berkeley.edu, minor=3
      nodeB:        vmhost-1.eecs.berkeley.edu, minor=3
      port:         11031
      auth key:     843408ec249db76dd6b5cae72ca13dd258c06c61
      on primary:   /dev/drbd3 (147:3) in sync, status ok
      on secondary: /dev/drbd3 (147:3) in sync, status ok
      child devices:
        - child 0: lvm, size 20.0G
          logical_id:
ganeti/a3b0afde-c9bd-4dda-bdf0-b6ea818df5fb.disk1_data
          on primary:
/dev/ganeti/a3b0afde-c9bd-4dda-bdf0-b6ea818df5fb.disk1_data (254:6)
          on secondary:
/dev/ganeti/a3b0afde-c9bd-4dda-bdf0-b6ea818df5fb.disk1_data (254:6)
        - child 1: lvm, size 128M
          logical_id:
ganeti/a3b0afde-c9bd-4dda-bdf0-b6ea818df5fb.disk1_meta
          on primary:
/dev/ganeti/a3b0afde-c9bd-4dda-bdf0-b6ea818df5fb.disk1_meta (254:7)
          on secondary:
/dev/ganeti/a3b0afde-c9bd-4dda-bdf0-b6ea818df5fb.disk1_meta (254:7)

Thanks!
Tim

--

Timothy Scoppetta
UC Berkeley

E: t...@eecs.berkeley.edu
P: 845-459-3002


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Timothy Scoppetta  
View profile  
 More options Aug 24 2012, 5:49 pm
From: Timothy Scoppetta <t...@eecs.berkeley.edu>
Date: Fri, 24 Aug 2012 14:49:54 -0700
Local: Fri, Aug 24 2012 5:49 pm
Subject: Re: Strange networking issue...
Print | Individual message | Show original | Report this message | Find messages by this author

Oh, ip link list was slightly out of date as I removed tap0 and tap1 and
then put them back in when I realized my windows host had stopped
networking:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN mode
DEFAULT
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0
state UP mode DEFAULT qlen 1000
    link/ether bc:30:5b:ee:06:30 brd ff:ff:ff:ff:ff:ff
3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master br0
state DOWN mode DEFAULT qlen 1000
    link/ether bc:30:5b:ee:06:31 brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master br0
state DOWN mode DEFAULT qlen 1000
    link/ether bc:30:5b:ee:06:32 brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0
state UP mode DEFAULT qlen 1000
    link/ether bc:30:5b:ee:06:33 brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
qlen 1000
    link/ether a0:36:9f:07:12:c4 brd ff:ff:ff:ff:ff:ff
7: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
qlen 1000
    link/ether a0:36:9f:07:12:c6 brd ff:ff:ff:ff:ff:ff
8: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
mode DEFAULT
    link/ether b2:21:a3:82:17:27 brd ff:ff:ff:ff:ff:ff
10: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
master br0 state UNKNOWN mode DEFAULT qlen 500
    link/ether b2:21:a3:82:17:27 brd ff:ff:ff:ff:ff:ff
11: tap1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
master br0 state UNKNOWN mode DEFAULT qlen 500
    link/ether ca:ef:89:13:00:da brd ff:ff:ff:ff:ff:ff

On Fri, Aug 24, 2012 at 2:42 PM, Timothy Scoppetta <t...@eecs.berkeley.edu>wrote:

...

read more »


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Guido Trotter  
View profile   Translate to Translated (View Original)
 More options Aug 24 2012, 6:29 pm
From: Guido Trotter <ultrot...@gmail.com>
Date: Fri, 24 Aug 2012 23:29:48 +0100
Local: Fri, Aug 24 2012 6:29 pm
Subject: Re: Strange networking issue...
Print | Individual message | Show original | Report this message | Find messages by this author

Hi Timothy,

It's normal for a kvm guests to use a tap interface : br0 is just the
bridge to connect it to: so yes, tap0 must be enslaved to br0, usually by
the kvm ifup script provided by ganeti : did you unenslave it or override
the script?

Also as you see your postrouting table is masquerading all routed (l3)
traffic by default, so you are natting.

So the behaviour you see is completely expected, for how your system is
configured.

Thanks,

Guido
On 24 Aug 2012 22:43, "Timothy Scoppetta" <t...@eecs.berkeley.edu> wrote:

...

read more »


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Timothy Scoppetta  
View profile  
 More options Aug 24 2012, 6:34 pm
From: Timothy Scoppetta <t...@eecs.berkeley.edu>
Date: Fri, 24 Aug 2012 15:34:13 -0700
Local: Fri, Aug 24 2012 6:34 pm
Subject: Re: Strange networking issue...
Print | Individual message | Show original | Report this message | Find messages by this author

I noticed the postrouting rule and removed it and everything seems to be
better now. Oddly it was only that one vmhost (out of seven) that was set
up like that. Someone's been monkeying with my cluster again.

Thanks for the help :)

-Tim

...

read more »


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »

Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
©2013 Google