OVS (rel 1.9.0) and routeflow.

ian collett

unread,

May 8, 2013, 5:47:41 AM5/8/13

to routeflo...@googlegroups.com

I have been working on a project to build a “network” within a virtual environment (the host and guests are built on Fedora 18 and KVM, apart from the RF guest that is using the RF VM from the web site), mainly for training and testing. As such I have built the network in the diagram attached, all three nodes (ldn, ny and roma) are the same with different mac and IP address, the ldn node is expanded in the diagram, and the configs are below.

Each OVS switch is connected to the controller, and route flow is running, all VM – DM –VS mapping are present and correct (all 6 columns filled in, also see logs below), on all the links between each node I can see OSPF hello messages (with the correct address), and in the LXC VM on the controller OSPF is active and I can ping between the interfaces (on the LXC VM).

NOTE – I AM USING THE OLDER ROUTEFLOW VM, NOT THE NEWEST GIT UPDATE FROM A FEW WEEKS AGO.

From a routing point of view I have one OVS bridge per node, and two eth cards connected to that bridge. The OVS bridge is connected to the controller and the controller see both links (see RF log). Using tcpdump (on the host) I can see that all the correct ospf hello packets are on the correct links. One point I have found is that is does NOT seem possible to model the OVS bridge in the LXC containers as a LXC does not allow you to load either ovs or bridge code as both need access to the Kernel.

I have also applied Mininet to this config and this seems to work (an issue found with Mininet seems to be that you cannot define the eth port to be used for a connection, by default is used eth0, then eth1, eth2 etc. – this means in this config you need to fiddle around get mininet to work, but it did).

Problem – I cannot ping between the host machines and I think (but maybe wrong) this is because the mac address allocated the OVS bridge is not being used by the RF controller as the “switch” address. I read in the FAQ that a common MAC address is used, and that seems to be the first eth address, but then OVS switch has a different mac address, and all frames are set to that mac address and the rules do not hit.

Any help on what I am doing wrong (is this the correct way to network with OVS)?

Regards

IAN

Looking at ldn (all other hosts are the same);

The link configs are (Note: eth1 and eth2 are connected to the OVS bridge called ldn-ovs);

[root@ldn ~]# ifconfig

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 192.168.100.101 netmask 255.255.255.0 broadcast 192.168.100.255

inet6 fe80::ff:fe00:100 prefixlen 64 scopeid 0x20<link>

ether 02:00:00:00:01:00 txqueuelen 1000 (Ethernet)

RX packets 2622 bytes 313074 (305.7 KiB)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 2701 bytes 464647 (453.7 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet6 fe80::ff:fe00:101 prefixlen 64 scopeid 0x20<link>

ether 02:00:00:00:01:01 txqueuelen 1000 (Ethernet)

RX packets 608 bytes 45793 (44.7 KiB)

RX errors 0 dropped 2 overruns 0 frame 0

TX packets 535 bytes 37674 (36.7 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet6 fe80::ff:fe00:102 prefixlen 64 scopeid 0x20<link>

ether 02:00:00:00:01:02 txqueuelen 1000 (Ethernet)

RX packets 614 bytes 46458 (45.3 KiB)

RX errors 0 dropped 3 overruns 0 frame 0

TX packets 547 bytes 38603 (37.6 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eth3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 10.101.1.10 netmask 255.255.255.0 broadcast 10.101.1.255

inet6 fe80::ff:fe00:103 prefixlen 64 scopeid 0x20<link>

ether 02:00:00:00:01:03 txqueuelen 1000 (Ethernet)

RX packets 6 bytes 468 (468.0 B)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 559 bytes 41655 (40.6 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ldn-ovs: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 10.101.1.100 netmask 255.255.255.0 broadcast 10.101.1.255

inet6 fe80::ff:fe00:199 prefixlen 64 scopeid 0x20<link>

ether 02:00:00:00:01:99 txqueuelen 0 (Ethernet)

RX packets 3 bytes 126 (126.0 B)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 129 bytes 7938 (7.7 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 16436

inet 127.0.0.1 netmask 255.0.0.0

inet6 ::1 prefixlen 128 scopeid 0x10<host>

loop txqueuelen 0 (Local Loopback)

RX packets 78 bytes 7210 (7.0 KiB)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 78 bytes 7210 (7.0 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

(Note: eth1 and eth2 are connected to the OVS bridge called ldn-ovs)

The OVS config is (note the mac address on the OVS bridge to avoid a random address being used, and this being used as the datapath_id towards RF, I have the SAME problem if I allow a random mac address to be used.);

[root@ldn ~]# ovs-vsctl show

eec8da21-e833-401f-901a-4a7c4dc72d7a

Bridge ldn-ovs

Controller "tcp:192.168.100.150:6633"

is_connected: true

Port ldn-ovs

Interface ldn-ovs

type: internal

Port "eth1"

Interface "eth1"

Port "eth3"

Interface "eth3"

Port "eth2"

Interface "eth2"

ovs_version: "1.9.0"

[root@ldn ~]# ovs-vsctl list bridge

_uuid : e9d75ee0-54bc-4141-bceb-755115903eef

controller : [d054366b-1c0f-47d2-a2d0-23da7484c1a2]

datapath_id : "0000020000000199"

datapath_type : ""

external_ids : {}

fail_mode : []

flood_vlans : []

flow_tables : {}

mirrors : []

name : ldn-ovs

netflow : []

other_config : {hwaddr="02:00:00:00:01:99"}

ports : [2ab2c4c6-8f67-4ead-b975-5e7564b6d914, 446dc19d-8780-413f-a663-f3d8c8df8376, 7c58a3fb-7cf0-4975-b2be-5d0d281ac50e, dc223c98-3cb0-475e-98df-cdc326d05950]

sflow : []

status : {}

stp_enable : false

[root@ldn ~]#

The flows form the ldn-ovs switch

[root@ldn ~]# ovs-ofctl dump-flows ldn-ovs

NXST_FLOW reply (xid=0x4):

cookie=0x0, duration=1078.906s, table=0, n_packets=4, n_bytes=168, idle_age=1044, arp actions=CONTROLLER:65535

cookie=0x0, duration=1078.906s, table=0, n_packets=0, n_bytes=0, idle_age=1078, tcp,tp_dst=179 actions=CONTROLLER:65535

cookie=0x0, duration=1078.906s, table=0, n_packets=0, n_bytes=0, idle_age=1078, udp,nw_dst=224.0.0.9 actions=CONTROLLER:65535

cookie=0x0, duration=1044.45s, table=0, n_packets=0, n_bytes=0, idle_age=1044, priority=32792,ip,dl_dst=02:00:00:00:01:01,nw_dst=10.10.102.0/24 actions=mod_dl_src:02:00:00:00:01:01,mod_dl_dst:02:00:00:00:03:01,output:1

cookie=0x0, duration=1050.456s, table=0, n_packets=0, n_bytes=0, idle_age=1050, priority=32792,ip,dl_dst=02:00:00:00:01:01,nw_dst=10.102.1.0/24 actions=mod_dl_src:02:00:00:00:01:01,mod_dl_dst:02:00:00:00:02:01,output:2

cookie=0x0, duration=1044.45s, table=0, n_packets=0, n_bytes=0, idle_age=1044, priority=32792,ip,dl_dst=02:00:00:00:01:01,nw_dst=10.103.1.0/24 actions=mod_dl_src:02:00:00:00:01:01,mod_dl_dst:02:00:00:00:03:01,output:1

cookie=0x0, duration=1078.906s, table=0, n_packets=26, n_bytes=2548, idle_age=133, icmp actions=CONTROLLER:65535

cookie=0x0, duration=1078.906s, table=0, n_packets=2125, n_bytes=174470, idle_age=0, ip,nw_proto=89 actions=CONTROLLER:65535

[root@ldn ~]#

Log from RF starting;

routeflow@routeflow:~/RouteFlow/sdnDemo$ sudo ./rfSdnDemo --pox

[sudo] password for routeflow:

-> Starting OVS daemons...

ovsdb-server: /usr/local/var/run/openvswitch/ovsdb-server.pid: already running as pid 1229, aborting

ovs-vswitchd: /usr/local/var/run/openvswitch/ovs-vswitchd.pid: already running as pid 1231, aborting

-> Stopping and resetting LXC VMs...

-> Deleting data from previous runs...

-> Starting MongoDB...

.all output going to: /dev/null

-> Starting the virtual machines...

-> Starting the management network (br0)...

-> Starting the controller (POX) and RFPRoxy...

..INFO:rfproxy:RFProxy running.

INFO:core:POX 0.0.0 is up.

-> Starting RFServer...

-> Starting the control plane network (dp0 VS)...

INFO:openflow.of_01:[Con 1/None] closing connection

INFO:openflow.of_01:[Con 2/8243406406160905843] Connected to 76-73-72-66-76-73|29286

---

Control ready, type ctrl-c to stop.

---

INFO:openflow.topology:Switch 76-73-72-66-76-73|29286 connected

INFO:rfproxy:Registering datapath port (dp_id=0x7266767372667673, dp_port=1)

INFO:rfproxy:Registering datapath port (dp_id=0x7266767372667673, dp_port=2)

INFO:rfproxy:Registering datapath port (dp_id=0x7266767372667673, dp_port=3)

INFO:rfproxy:Registering datapath port (dp_id=0x7266767372667673, dp_port=4)

INFO:rfproxy:Registering datapath port (dp_id=0x7266767372667673, dp_port=5)

INFO:rfproxy:Registering datapath port (dp_id=0x7266767372667673, dp_port=6)

INFO:rfproxy:Registering datapath port (dp_id=0x7266767372667673, dp_port=7)

INFO:rfproxy:Registering datapath port (dp_id=0x7266767372667673, dp_port=8)

INFO:rfproxy:Registering datapath port (dp_id=0x7266767372667673, dp_port=9)

INFO:rfserver:Configuring RFVS (dp_id=0x7266767372667673)

INFO:rfproxy:ofp_flow_mod(config) was sent to datapath (dp_id=0x7266767372667673)

INFO:openflow.of_01:[Con 3/2199023255961] Connected to 02-00-00-00-01-99

INFO:openflow.topology:Switch 02-00-00-00-01-99 connected

INFO:rfproxy:Registering datapath port (dp_id=0x20000000199, dp_port=1)

INFO:rfserver:Registering datapath port as idle (dp_id=0x20000000199, dp_port=1)

INFO:rfproxy:Registering datapath port (dp_id=0x20000000199, dp_port=2)

INFO:rfproxy:Registering datapath port (dp_id=0x20000000199, dp_port=3)

INFO:openflow.of_01:[Con 4/2199023256473] Connected to 02-00-00-00-03-99

INFO:openflow.topology:Switch 02-00-00-00-03-99 connected

INFO:rfproxy:Registering datapath port (dp_id=0x20000000399, dp_port=1)

INFO:rfproxy:Registering datapath port (dp_id=0x20000000399, dp_port=2)

INFO:rfproxy:Registering datapath port (dp_id=0x20000000399, dp_port=3)

INFO:rfserver:Configuring datapath (dp_id=0x20000000199)

INFO:rfserver:Registering datapath port as idle (dp_id=0x20000000199, dp_port=2)

INFO:rfserver:Configuring datapath (dp_id=0x20000000199)

INFO:rfserver:Registering datapath port as idle (dp_id=0x20000000199, dp_port=3)

INFO:rfproxy:ofp_flow_mod(config) was sent to datapath (dp_id=0x20000000199)

INFO:rfserver:Registering datapath port as idle (dp_id=0x20000000399, dp_port=1)

INFO:rfproxy:ofp_flow_mod(config) was sent to datapath (dp_id=0x20000000199)

INFO:rfserver:Configuring datapath (dp_id=0x20000000399)

INFO:rfserver:Registering datapath port as idle (dp_id=0x20000000399, dp_port=2)

INFO:rfserver:Configuring datapath (dp_id=0x20000000399)

INFO:rfserver:Registering datapath port as idle (dp_id=0x20000000399, dp_port=3)

INFO:rfproxy:ofp_flow_mod(config) was sent to datapath (dp_id=0x20000000399)

INFO:openflow.of_01:[Con 6/259] Connected to 00-00-00-00-01-03

INFO:openflow.topology:Switch 00-00-00-00-01-03 connected

INFO:openflow.of_01:[Con 5/258] Connected to 00-00-00-00-01-02

INFO:openflow.topology:Switch 00-00-00-00-01-02 connected

INFO:rfproxy:ofp_flow_mod(config) was sent to datapath (dp_id=0x20000000399)

INFO:openflow.of_01:[Con 7/257] Connected to 00-00-00-00-01-01

INFO:openflow.topology:Switch 00-00-00-00-01-01 connected

INFO:openflow.of_01:[Con 8/2199023256217] Connected to 02-00-00-00-02-99

INFO:openflow.topology:Switch 02-00-00-00-02-99 connected

INFO:rfproxy:Registering datapath port (dp_id=0x20000000299, dp_port=1)

INFO:rfproxy:Registering datapath port (dp_id=0x20000000299, dp_port=2)

INFO:rfproxy:Registering datapath port (dp_id=0x20000000299, dp_port=3)

INFO:rfserver:Registering datapath port as idle (dp_id=0x20000000299, dp_port=1)

INFO:rfserver:Configuring datapath (dp_id=0x20000000299)

INFO:rfserver:Registering datapath port as idle (dp_id=0x20000000299, dp_port=2)

INFO:rfserver:Configuring datapath (dp_id=0x20000000299)

INFO:rfproxy:ofp_flow_mod(config) was sent to datapath (dp_id=0x20000000299)

INFO:rfserver:Registering datapath port as idle (dp_id=0x20000000299, dp_port=3)

INFO:rfproxy:ofp_flow_mod(config) was sent to datapath (dp_id=0x20000000299)

INFO:openflow.discovery:link detected: 02-00-00-00-01-99.2 -> 02-00-00-00-02-99.1

INFO:openflow.discovery:link detected: 02-00-00-00-01-99.1 -> 02-00-00-00-03-99.2

INFO:openflow.discovery:link detected: 02-00-00-00-03-99.2 -> 02-00-00-00-01-99.1

INFO:openflow.discovery:link detected: 02-00-00-00-03-99.1 -> 02-00-00-00-02-99.2

INFO:openflow.discovery:link detected: 02-00-00-00-02-99.2 -> 02-00-00-00-03-99.1

INFO:openflow.discovery:link detected: 02-00-00-00-02-99.1 -> 02-00-00-00-01-99.2

INFO:rfserver:Asking client for mapping message for port (vm_id=0x20000000100, vm_port=1)

INFO:rfserver:Registering client port and associating to datapath port (vm_id=0x20000000100, vm_port=1, dp_id=2199023255961, dp_port=1)

INFO:rfserver:Asking client for mapping message for port (vm_id=0x20000000100, vm_port=2)

INFO:rfserver:Registering client port and associating to datapath port (vm_id=0x20000000100, vm_port=2, dp_id=2199023255961, dp_port=2)

INFO:rfserver:Asking client for mapping message for port (vm_id=0x20000000100, vm_port=3)

INFO:rfserver:Registering client port and associating to datapath port (vm_id=0x20000000100, vm_port=3, dp_id=2199023255961, dp_port=3)

INFO:rfproxy:Received mapping packet (vm_id=0x20000000100, vm_port=2, vs_id=8243406406160905843, vs_port=2)

INFO:rfproxy:Received mapping packet (vm_id=0x20000000100, vm_port=3, vs_id=8243406406160905843, vs_port=7)

INFO:rfproxy:Received mapping packet (vm_id=0x20000000100, vm_port=1, vs_id=8243406406160905843, vs_port=1)

INFO:rfserver:Mapping client-datapath association (vm_id=0x20000000100, vm_port=2, dp_id=0x20000000199, dp_port=2, vs_id=0x7266767372667673, vs_port=2)

INFO:rfserver:Mapping client-datapath association (vm_id=0x20000000100, vm_port=3, dp_id=0x20000000199, dp_port=3, vs_id=0x7266767372667673, vs_port=7)

INFO:rfserver:Mapping client-datapath association (vm_id=0x20000000100, vm_port=1, dp_id=0x20000000199, dp_port=1, vs_id=0x7266767372667673, vs_port=1)

INFO:rfserver:Asking client for mapping message for port (vm_id=0x20000000200, vm_port=1)

INFO:rfserver:Registering client port and associating to datapath port (vm_id=0x20000000200, vm_port=1, dp_id=2199023256217, dp_port=1)

INFO:rfserver:Asking client for mapping message for port (vm_id=0x20000000200, vm_port=2)

INFO:rfserver:Registering client port and associating to datapath port (vm_id=0x20000000200, vm_port=2, dp_id=2199023256217, dp_port=2)

INFO:rfserver:Asking client for mapping message for port (vm_id=0x20000000200, vm_port=3)

INFO:rfserver:Registering client port and associating to datapath port (vm_id=0x20000000200, vm_port=3, dp_id=2199023256217, dp_port=3)

INFO:rfproxy:Received mapping packet (vm_id=0x20000000200, vm_port=3, vs_id=8243406406160905843, vs_port=8)

INFO:rfproxy:Received mapping packet (vm_id=0x20000000200, vm_port=1, vs_id=8243406406160905843, vs_port=3)

INFO:rfproxy:Received mapping packet (vm_id=0x20000000200, vm_port=2, vs_id=8243406406160905843, vs_port=4)

INFO:rfserver:Mapping client-datapath association (vm_id=0x20000000200, vm_port=3, dp_id=0x20000000299, dp_port=3, vs_id=0x7266767372667673, vs_port=8)

INFO:rfserver:Mapping client-datapath association (vm_id=0x20000000200, vm_port=1, dp_id=0x20000000299, dp_port=1, vs_id=0x7266767372667673, vs_port=3)

INFO:rfserver:Mapping client-datapath association (vm_id=0x20000000200, vm_port=2, dp_id=0x20000000299, dp_port=2, vs_id=0x7266767372667673, vs_port=4)

INFO:rfproxy:ofp_flow_mod(add) was sent to datapath (dp_id=0x20000000199)

INFO:rfproxy:ofp_flow_mod(add) was sent to datapath (dp_id=0x20000000299)

INFO:rfserver:Asking client for mapping message for port (vm_id=0x20000000300, vm_port=1)

INFO:rfserver:Registering client port and associating to datapath port (vm_id=0x20000000300, vm_port=1, dp_id=2199023256473, dp_port=1)

INFO:rfserver:Asking client for mapping message for port (vm_id=0x20000000300, vm_port=2)

INFO:rfserver:Registering client port and associating to datapath port (vm_id=0x20000000300, vm_port=2, dp_id=2199023256473, dp_port=2)

INFO:rfserver:Asking client for mapping message for port (vm_id=0x20000000300, vm_port=3)

INFO:rfserver:Registering client port and associating to datapath port (vm_id=0x20000000300, vm_port=3, dp_id=2199023256473, dp_port=3)

INFO:rfproxy:Received mapping packet (vm_id=0x20000000300, vm_port=1, vs_id=8243406406160905843, vs_port=5)

INFO:rfproxy:Received mapping packet (vm_id=0x20000000300, vm_port=2, vs_id=8243406406160905843, vs_port=6)

INFO:rfproxy:Received mapping packet (vm_id=0x20000000300, vm_port=3, vs_id=8243406406160905843, vs_port=9)

INFO:rfserver:Mapping client-datapath association (vm_id=0x20000000300, vm_port=1, dp_id=0x20000000399, dp_port=1, vs_id=0x7266767372667673, vs_port=5)

INFO:rfserver:Mapping client-datapath association (vm_id=0x20000000300, vm_port=2, dp_id=0x20000000399, dp_port=2, vs_id=0x7266767372667673, vs_port=6)

INFO:rfserver:Mapping client-datapath association (vm_id=0x20000000300, vm_port=3, dp_id=0x20000000399, dp_port=3, vs_id=0x7266767372667673, vs_port=9)

INFO:rfproxy:ofp_flow_mod(add) was sent to datapath (dp_id=0x20000000399)

INFO:rfproxy:ofp_flow_mod(add) was sent to datapath (dp_id=0x20000000199)

ian collett

unread,

May 13, 2013, 5:21:09 PM5/13/13

to routeflo...@googlegroups.com

Any suggests welcome :-), has any one got OVS (openvswitch) working as part of a openflow network? - IAN

<font color="#000000" size="3" face="Time...

Allan Vidal

unread,

May 14, 2013, 8:23:10 AM5/14/13

to routeflo...@googlegroups.com

Hi Ian,

Sorry for the late answer, I've been busy.

I'm having trouble understanding your setup. Please, show us what you want RouteFlow to control, separating the routers, bridges and network hosts.

As for OVS 1.9.0, I think newer versions of OVS implement a mix of OpenFlow 1.0, 1.2 and 1.3. You could limit it to 1.0:

sudo ovs-vsctl set bridge br0 protocols=OpenFlow10

But I'm not sure that will help... It's just an attempt :)

As for running networks using OVS, I managed to do it with OVS 1.4, even linking bridges acting as routers controlled by RouteFlow to normal bridges. Mininet runs OVS too.

Allan

--
You received this message because you are subscribed to the Google Groups "RouteFlow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to routeflow-disc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Message has been deleted

ian collett

unread,

May 14, 2013, 3:36:43 PM5/14/13

to routeflo...@googlegroups.com

Tks - Allan,

I will not have time today to see if setting the OF version of 1.0 fixes things.

What I am trying to build is a routed network between the guest machines. I am looking for routeflow to provide the routing between the machines. Each guest has two links and both links are conected the an OVS bridge.

Below you said you had "linking bridges (OVS rel 1.4) acting as routers controlled by RouteFlow" could you give me some details on how you did that? How did the networking look? I have the feeling I have some basic logic wrong.

Tks in advance,

IAN

ian collett

unread,

May 16, 2013, 7:41:11 AM5/16/13

to routeflo...@googlegroups.com

I am using OVS ver 1.9.0 and that does not seem to support the “protocols” options (ie ovs-vsctl set bridge br0 protocols=OpenFlow10), there is a long thread on this at http://openvswitch.org/pipermail/discuss/2013-January/008812.html but it seems that OVS 1.9.0 only support Openflow 1.0 (plus some bits), however you do not seem to be able to turn off the “plus bits”. It looks like this may be possible with OVS 1.10, but from the previous post it seems you got this to work with an earlier version of OVS. I would be really interested in the design that you got to work. Over time I have tried a load of configurations, but would welcome feedback on either what is wrong, or what has worked in the past.

I have however retested and provided some more data hopefully this explains what I am trying to do a bit better, also may be worth a quick look at the diagram in the original post.

Started all the guest systems (ldn,ny and roma) and sorted out all the links, so the correct OSPF hello packets are seen on each link. Dump ldn-ovs config and routing table (hight lighting that the controller is online, and the HW MAC address that is set).

[root@ldn ~]# ovs-vsctl show;ovs-vsctl list bridge;route -n

eec8da21-e833-401f-901a-4a7c4dc72d7a

Bridge ldn-ovs

Controller "tcp:192.168.100.150:6633"

is_connected: true

Port "eth1"

Interface "eth1"

Port "eth3"

Interface "eth3"

Port "eth2"

Interface "eth2"

Port ldn-ovs

Interface ldn-ovs

type: internal

ovs_version: "1.9.0"

_uuid : c25948f2-1689-4dbc-a267-f51e70e1e67f

controller : [512383d1-1591-4ba8-8d10-238bdd5549d0]

datapath_id : "0000020000000199"

datapath_type : ""

external_ids : {}

fail_mode : []

flood_vlans : []

flow_tables : {}

mirrors : []

name : ldn-ovs

netflow : []

other_config : {hwaddr="02:00:00:00:01:99"}

ports : [480ee2d5-d860-4950-bd11-50d43760626f, 512fcffe-f610-4df6-8060-5bdd40d61e7f, a5141eb6-05a7-4c7f-a74e-8b014adaf63c, b53c1e31-2427-4b79-bb8e-a174cff7833e]

sflow : []

status : {}

stp_enable : false

Kernel IP routing table

Destination Gateway Genmask Flags Metric Ref Use Iface

0.0.0.0 10.101.1.100 0.0.0.0 UG 0 0 0 ldn-ovs

10.101.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ldn-ovs

10.101.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth3

192.168.100.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0

[root@ldn ~]#

Ping address 10.102.1.10 form ldn-ovs bridge (command “ping –I ldn-ovs 10.102.1.10”) – result no responce;

Trace from the ldn-ovs bridge (“tcpdump –i ldn-ovs -ne”) shows ICMP echo requests, NOTE scource MAC address this is the MAC address of the bridge (highlight the source mac address, this is the address of the bridge as defined above).

11:52:09.886638 02:00:00:00:01:99 > 02:00:00:00:02:01, ethertype IPv4 (0x0800), length 98: 10.101.1.100 > 10.102.1.10: ICMP echo request, id 4723, seq 16, length 64

11:52:10.886586 02:00:00:00:01:99 > 02:00:00:00:02:01, ethertype IPv4 (0x0800), length 98: 10.101.1.100 > 10.102.1.10: ICMP echo request, id 4723, seq 17, length 64

11:52:11.886603 02:00:00:00:01:99 > 02:00:00:00:02:01, ethertype IPv4 (0x0800), length 98: 10.101.1.100 > 10.102.1.10: ICMP echo request, id 4723, seq 18, length 64

11:52:12.886570 02:00:00:00:01:99 > 02:00:00:00:02:01, ethertype IPv4 (0x0800), length 98: 10.101.1.100 > 10.102.1.10: ICMP echo request, id 4723, seq 19, length 64

Tcpdump for the same time on the link between ldn and ny nodes, note NO ICMP packets on OSPF hello messages.

11:52:08.123674 02:00:00:00:02:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

11:52:08.125886 02:00:00:00:01:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

11:52:09.136996 02:00:00:00:02:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

11:52:09.138210 02:00:00:00:01:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

11:52:09.918452 02:00:00:00:01:02 > 01:23:20:00:00:01, ethertype LLDP (0x88cc), length 61:

11:52:10.134482 02:00:00:00:02:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

11:52:10.136709 02:00:00:00:01:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

11:52:11.118280 02:00:00:00:02:01 > 01:23:20:00:00:01, ethertype LLDP (0x88cc), length 61:

11:52:11.118981 02:00:00:00:02:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

11:52:11.123334 02:00:00:00:01:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

11:52:12.132257 02:00:00:00:02:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

11:52:12.133389 02:00:00:00:01:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

11:52:13.106840 02:00:00:00:02:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

11:52:13.107728 02:00:00:00:01:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

Dump of flows from ldn-ovs (high lighting nearest match, however the source MAC address does not match).

[root@ldn ~]# ovs-ofctl dump-flows ldn-ovs

NXST_FLOW reply (xid=0x4):

cookie=0x0, duration=1272.101s, table=0, n_packets=15, n_bytes=630, idle_age=595, arp actions=CONTROLLER:65535

cookie=0x0, duration=1272.122s, table=0, n_packets=0, n_bytes=0, idle_age=1272, tcp,tp_dst=179 actions=CONTROLLER:65535

cookie=0x0, duration=1272.114s, table=0, n_packets=0, n_bytes=0, idle_age=1272, udp,nw_dst=224.0.0.9 actions=CONTROLLER:65535

cookie=0x0, duration=1237.303s, table=0, n_packets=0, n_bytes=0, idle_age=1237, priority=32792,ip,dl_dst=02:00:00:00:01:01,nw_dst=10.10.102.0/24 actions=mod_dl_src:02:00:00:00:01:01,mod_dl_dst:02:00:00:00:03:01,output:1

cookie=0x0, duration=1242.233s, table=0, n_packets=0, n_bytes=0, idle_age=1242, priority=32792,ip,dl_dst=02:00:00:00:01:01,nw_dst=10.102.1.0/24 actions=mod_dl_src:02:00:00:00:01:01,mod_dl_dst:02:00:00:00:02:01,output:2

cookie=0x0, duration=1237.302s, table=0, n_packets=0, n_bytes=0, idle_age=1237, priority=32792,ip,dl_dst=02:00:00:00:01:01,nw_dst=10.103.1.0/24 actions=mod_dl_src:02:00:00:00:01:01,mod_dl_dst:02:00:00:00:03:01,output:1

cookie=0x0, duration=1272.084s, table=0, n_packets=561, n_bytes=54978, idle_age=136, icmp actions=CONTROLLER:65535

cookie=0x0, duration=1272.131s, table=0, n_packets=2514, n_bytes=206552, idle_age=0, ip,nw_proto=89 actions=CONTROLLER:65535

[root@ldn ~]#

Repeat test, but ping same address form the RF controller, from within the LXC container for ldn.

root@ldnSdnVM:~#

root@ldnSdnVM:~# ping 10.102.1.10

PING 10.102.1.10 (10.102.1.10): 56 data bytes

64 bytes from 10.102.1.10: icmp_seq=0 ttl=64 time=19.033 ms

64 bytes from 10.102.1.10: icmp_seq=1 ttl=64 time=48.518 ms

64 bytes from 10.102.1.10: icmp_seq=2 ttl=64 time=46.245 ms

64 bytes from 10.102.1.10: icmp_seq=3 ttl=64 time=29.828 ms

Trace from the link between ldn-ny (note ICMP packtes both request and reply)

[root@mandeville ~]# tcpdump -i tap.l-n.ny -ne

tcpdump: WARNING: tap.l-n.ny: no IPv4 address assigned

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on tap.l-n.ny, link-type EN10MB (Ethernet), capture size 65535 bytes

12:10:01.061009 02:00:00:00:01:01 > 02:00:00:00:02:01, ethertype IPv4 (0x0800), length 98: 10.10.100.2 > 10.102.1.10: ICMP echo request, id 60161, seq 768, length 64

12:10:01.063218 02:00:00:00:02:01 > 02:00:00:00:01:01, ethertype IPv4 (0x0800), length 98: 10.102.1.10 > 10.10.100.2: ICMP echo reply, id 60161, seq 768, length 64

12:10:01.847623 02:00:00:00:02:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

12:10:01.848819 02:00:00:00:01:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

12:10:02.046730 02:00:00:00:01:01 > 02:00:00:00:02:01, ethertype IPv4 (0x0800), length 98: 10.10.100.2 > 10.102.1.10: ICMP echo request, id 60161, seq 1024, length 64

12:10:02.049330 02:00:00:00:02:01 > 02:00:00:00:01:01, ethertype IPv4 (0x0800), length 98: 10.102.1.10 > 10.10.100.2: ICMP echo reply, id 60161, seq 1024, length 64

12:10:02.165419 02:00:00:00:02:01 > 01:23:20:00:00:01, ethertype LLDP (0x88cc), length 61:

12:10:02.864274 02:00:00:00:02:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

12:10:02.865021 02:00:00:00:01:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

12:10:03.082415 02:00:00:00:01:01 > 02:00:00:00:02:01, ethertype IPv4 (0x0800), length 98: 10.10.100.2 > 10.102.1.10: ICMP echo request, id 60161, seq 1280, length 64

12:10:03.083303 02:00:00:00:02:01 > 02:00:00:00:01:01, ethertype ARP (0x0806), length 42: Request who-has 10.10.100.2 tell 10.10.100.1, length 28

12:10:03.086725 02:00:00:00:02:01 > 02:00:00:00:01:01, ethertype IPv4 (0x0800), length 98: 10.102.1.10 > 10.10.100.2: ICMP echo reply, id 60161, seq 1280, length 64

12:10:03.087149 02:00:00:00:01:01 > 02:00:00:00:02:01, ethertype ARP (0x0806), length 42: Reply 10.10.100.2 is-at 02:00:00:00:01:01, length 28

12:10:03.843994 02:00:00:00:02:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

12:10:03.884069 02:00:00:00:01:01 > 01:00:5e:00:00:05, ethertype IPv4 (0x0800), length 82: 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

Trace from ldn-ovs – no traffic because packets get passed back to RF controller before hitting bridge (I guess).

[root@ldn ~]# tcpdump -i ldn-ovs -ne

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on ldn-ovs, link-type EN10MB (Ethernet), capture size 65535 bytes

^C

0 packets captured

0 packets received by filter

0 packets dropped by kernel

[root@ldn ~]#

<font color="#000000" size="3" face="Time...

ian collett

unread,

May 30, 2013, 1:12:43 PM5/30/13

to routeflo...@googlegroups.com

Another day of tests and I am still stuck … any feedback or suggestions would be really welcome … PLEASE J.

I have now worked out how to set the datapath-id, and the OVS bridge MAC address to the same as first ethernet port. See below

[root@ldn ~]#

[root@ldn ~]# cat /etc/sysconfig/network-scripts/ifcfg-ldn-ovs

DEVICE=ldn-ovs

DEVICETYPE=ovs

TYPE=OVSBridge

IPV6INIT=no

ONBOOT=yes

OVS_OPTIONS="-- set-controller ldn-ovs tcp:192.168.100.150:6633 -- set Bridge ldn-ovs other-config:datapath-id=0000000000000101 -- set bridge ldn-ovs other-config:hwaddr=02:0:0:0:1:1"

IPADDR=10.101.1.100

NETMASK=255.255.255.0

GATEWAY=10.101.1.100

DELAY=0

NM_CONTOLLED=no

[root@ldn ~]#

[root@ldn ~]# ovs-vsctl list bridge

_uuid : 1c06daaa-e6ea-469c-a8e3-36aa65f5a2d2

controller : [7d7b1a10-79d6-4b20-bfdb-9ec76bc3925c]

datapath_id : "0000000000000101"

datapath_type : ""

external_ids : {}

fail_mode : []

flood_vlans : []

flow_tables : {}

mirrors : []

name : ldn-ovs

netflow : []

other_config : {datapath-id="0000000000000101", hwaddr="02:0:0:0:1:1"}

ports : [385effaa-0c65-44de-b57c-6f8e43a44edc, 4d9d58ac-91e3-40fe-bad2-f60a65348a57, 81f10137-fa3e-4bbf-a22d-3f20fcc1c9d1, b6d56ee6-8ade-435d-82c2-24dfd01ebcfb]

sflow : []

status : {}

stp_enable : false

[root@ldn ~]#

[root@ldn ~]# ifconfig ldn-ovs

ldn-ovs: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 10.101.1.100 netmask 255.255.255.0 broadcast 10.101.1.255

inet6 fe80::ff:fe00:101 prefixlen 64 scopeid 0x20<link>

ether 02:00:00:00:01:01 txqueuelen 0 (Ethernet)

RX packets 10 bytes 420 (420.0 B)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 155 bytes 15974 (15.5 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

[root@ldn ~]#

However now with the same MAC address of the ETH1 and bridge the OSPF in the LXC dose is not working correctly, see the tcpdump below.

18:57:04.256570

18:57:04.261667 IP 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

18:57:04.307825 IP 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

18:57:05.170289 ARP, Request who-has 10.10.100.1 tell 10.10.100.2, length 28

18:57:05.173555 ARP, Reply 10.10.100.1 is-at 02:00:00:00:02:01, length 28

18:57:05.268654 IP 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

18:57:05.303718 IP 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

18:57:06.294346 IP 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

18:57:06.299801 IP 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

18:57:07.223796 ARP, Reply 10.10.101.2 is-at 02:00:00:00:03:01, length 28

18:57:07.288398 IP 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

18:57:07.306603 IP 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

18:57:08.177143 IP 10.10.100.1 > 10.10.100.2: OSPFv2, Database Description, length 32

18:57:08.177686 ARP, Request who-has 10.10.100.1 tell 10.10.100.2, length 28

18:57:08.180234 ARP, Reply 10.10.100.1 is-at 02:00:00:00:02:01, length 28

18:57:08.246934 ARP, Reply 10.10.101.2 is-at 02:00:00:00:03:01, length 28

18:57:08.264098 IP 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

18:57:08.305887 IP 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

18:57:08.472713

Note the ARP messages, and the OSPF database description messages that keep requiring.

Change the OVS bridge mac address and the OSPF works but now the OVS bridge address dose NOT match any of the flows provided by route flow.

_uuid : 3bd858ee-c7e4-4f2f-89b0-9a67737ee964

controller : [8cfa6411-78e2-4e41-bb13-133c1a50ac7d]

datapath_id : "0000000000000101"

datapath_type : ""

external_ids : {}

fail_mode : []

flood_vlans : []

flow_tables : {}

mirrors : []

name : ldn-ovs

netflow : []

other_config : {datapath-id="0000000000000101", hwaddr="02:0:0:0:1:99"}

ports : [0b58d9fe-68df-4592-a8a5-1789d68a6282, 3a63abf9-ff03-46a9-a70c-0cb709355268, 3b57e324-928d-4d39-87fc-dc69dd87694e, 43706628-7df9-472e-8042-bc8203a05c43]

sflow : []

status : {}

stp_enable : false

NXST_FLOW reply (xid=0x4):

cookie=0x0, duration=3.084s, table=0, n_packets=0, n_bytes=0, idle_age=3, priority=32792,ip,dl_dst=02:00:00:00:01:01,nw_dst=10.10.102.0/24 actions=mod_dl_src:02:00:00:00:01:01,mod_dl_dst:43:43:43:43:43:43,output:1

cookie=0x0, duration=3.044s, table=0, n_packets=0, n_bytes=0, idle_age=3, priority=32792,ip,dl_dst=02:00:00:00:01:01,nw_dst=10.102.1.0/24 actions=mod_dl_src:02:00:00:00:01:01,mod_dl_dst:43:43:43:43:43:43,output:2

cookie=0x0, duration=3.044s, table=0, n_packets=0, n_bytes=0, idle_age=3, priority=32792,ip,dl_dst=02:00:00:00:01:01,nw_dst=10.103.1.0/24 actions=mod_dl_src:02:00:00:00:01:01,mod_dl_dst:43:43:43:43:43:43,output:1

cookie=0x0, duration=9.614s, table=0, n_packets=2, n_bytes=90, idle_age=1, dl_dst=01:23:20:00:00:01,dl_type=0x88cc actions=CONTROLLER:65535

[root@ldn ~]# ping 10.102.1.10

PING 10.102.1.10 (10.102.1.10) 56(84) bytes of data.

^C

--- 10.102.1.10 ping statistics ---

14 packets transmitted, 0 received, 100% packet loss, time 13001ms

OSPF works

19:08:50.759138

19:08:51.673907 IP 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

19:08:51.674694 IP 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

19:08:52.715143 IP 10.10.100.1 > 224.0.0.5: OSPFv2, Hello, length 48

19:08:52.756849 IP 10.10.100.2 > 224.0.0.5: OSPFv2, Hello, length 48

19:08:53.198082

Tcpdump of ldn-ovs shows that mac address dose not macth any of the flows!

[root@ldn ~]# tcpdump -ne -i ldn-ovs

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on ldn-ovs, link-type EN10MB (Ethernet), capture size 65535 bytes

19:06:46.776210 02:00:00:00:01:99 > Broadcast, ethertype ARP (0x0806), length 42: Request who-has 10.102.1.10 tell 10.101.1.100, length 28

19:06:46.827068 02:00:00:00:02:01 > 02:00:00:00:01:99, ethertype ARP (0x0806), length 42: Reply 10.102.1.10 is-at 02:00:00:00:02:01, length 28

19:06:46.827089 02:00:00:00:01:99 > 02:00:00:00:02:01, ethertype IPv4 (0x0800), length 98: 10.101.1.100 > 10.102.1.10: ICMP echo request, id 18936, seq 1, length 64

19:06:47.777905 02:00:00:00:01:99 > 02:00:00:00:02:01, ethertype IPv4 (0x0800), length 98: 10.101.1.100 > 10.102.1.10: ICMP echo request, id 18936, seq 2, length 64

19:06:48.777909 02:00:00:00:01:99 > 02:00:00:00:02:01, ethertype IPv4 (0x0800), length 98: 10.101.1.100 > 10.102.1.10: ICMP echo request, id 18936, seq 3, length 64

19:06:49.777888 02:00:00:00:01:99 > 02:00:00:00:02:01, ethertype IPv4 (0x0800), length 98: 10.101.1.100 > 10.102.1.10: ICMP echo request, id 18936, seq 4, length 64

On Wednesday, 8 May 2013 11:47:41 UTC+2, ian collett wrote:

<font color="#000000" size="3" face="Time...

Allan Vidal

unread,

Jun 3, 2013, 9:05:59 AM6/3/13

to routeflo...@googlegroups.com

Hi Ian,

I think I finally understood your scenario :) Basically, you want to run OVS instances attached to interfaces in hosts, to make them act as routers. These router will be connected by bridges, that are OVS instances also. All of this will be simulated inside a single machine. Correct me if I made any mistake :)

So, what I did before was run OVS instances directly on the same machine, connected through virtual links. I don't have the script I used to do this with me right now, I'll send tomorrow. If you can't get your scenario to work, you could first try this simpler approach, and then divide it into different virtual machines.

I can see that some MAC adresses are 43:43:43:43:43:43 in the latest test, which means failure in resolving them. This is probably due to a faulty assumption somewhere. I had this issue with OVS before, it was caused by misconfiguration, but in the end it can be caused by many things, so I just don't know what to say :D

Try to take a look at /var/log/rfclient.log in each of the LXC containers, they should show that some MAC addresses are failing to resolve. From there, we'll have to investigate why OVS is not forwarding the packets appropriately.

In the previous message, you said ldn-ovs was not getting any traffic. I don't know if this should be the behavior, but you should look into it. There's a nice link with information on how to debug OVS, it might help you: http://openvswitch.org/slides/OVS-Debugging-110414.pdf

I'll try to better understand what's happening in the next few days, I'm overwhelmed by the logs and outputs :D

Allan

--

ian collett

unread,

Jun 23, 2013, 12:10:53 PM6/23/13

to routeflo...@googlegroups.com

Below you said you could send me more details of the tests you completed with OVS, and maybe a "script", I know you are a busy guy but any more info would be really welcome.

when you say "you want to run OVS instances attached to interfaces in hosts, to make them act as routers" you are 100% correct, and I am looking to use route flow to provide the network routing, but with no luck :-(.

I have done some more testing, got some more logs/traces - but based on your last comments I guess you my be over loaded with logs :-).

As always thanks in advance,

IAN

On Wednesday, May 8, 2013 11:47:41 AM UTC+2, ian collett wrote:

<font color="#000000" size="3" face="Time...

Christian Esteve Rothenberg

unread,

Jun 25, 2013, 8:19:27 AM6/25/13

to routeflo...@googlegroups.com

Hi Ian,

did you manage to advance with your setup?

are you still using the RouteFlow version in the VM?

one suggestion would be to try to update to the latest code version in github.

Cheers,

Christian

--

You received this message because you are subscribed to the Google Groups "RouteFlow" group.

To unsubscribe from this group and stop receiving emails from it, send an email to routeflow-disc...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Christian Esteve Rothenberg, Ph.D.
Converged Networks Business Unit
CPqD - Center for Research and Development in Telecommunications
Tel. (+55 19) 3705 4479 / Cel. (+55 19) 8193-7087

Allan Vidal

unread,

Jun 25, 2013, 10:41:16 AM6/25/13

to routeflo...@googlegroups.com

Hi Ian,

The script for the setup I mentioned is attached. It features a virtual router (OVS), connected to virtual switches, that in turn are connected to virtual hosts. Those were the commands I used to simulate an inter-VLAN routing setup.

I really don't know what could be wrong with your setup... Like I said before, try simplify and start with fewer devices, and then incrementally add the other parts. Keep us posted on this, I'll try to help you more closely.