Issue on trema-edge with CPqD Software Switch

[Muxi Yan]

Hello All,

We are recently working on a very interesting but confusing problem of trema-edge. 
We could not figure out solution to this problem, but we hope someone here may
have seen this problem before or have some idea of why it happens.

The platforms we are working on are:
1) OpenWRT switch box with CPqD Software Switch (OF v1.3). The OpenFlow ports
   are one Ethernet port and the WiFi interface.
2) Trema-edge running on Ubuntu 12.04 LTS, which is running on VirtualBox on Mac
   OS X. The network interface of Ubuntu is bridged to Ethernet port of Mac.

Now our problem is this:
We run multi_learning_switch example in trema-edge, and in SOME cases
switch-controller connection will break at the time after controller receives SOME
number of packet_in messages. The problem exists for certain switch box but not
for others with exactly the same platform and the same software.

We probed the problem using tcpdump and 'debug' logging level. It seems that
when some packet_in messgae is received and ACKed by the controller, the
controller sent out a tcp FIN message, breaking the connection. The log of
trema-edge shows that the recv() function in src/lib/messenger.c:on_recv()
returned 0 at the time of the problem, suggesting that connection is already
closed.  However we are not sure why because, by tcpdump trace, obviously it is
the controller that initiated the disconnection, not the other end.

I am attaching one of our tcpdump trace here if it could provide any helpful
information. Ignoring all previous messages, simply search for my note "Problem
is here" at line 197; you'll see the controller initiated a tcp FIN here as a
response to the packet_in message above at line 178.

We appreciate any help or information!

Thanks and best,

Muxi Yan 

[Denis Ovsienko]

>information. Ignoring all previous messages, simply search for my note "Problem
>is here" at line 197; you'll see the controller initiated a tcp FIN here as a
>response to the packet_in message above at line 178.

Hello.

This could be, for example, due to a hello timeout, I had a somewhat similar issue with OF 1.0 (Trema non-edge) when the message exchange became too busy for the switch to respond promptly. Also the controller can close the connection when the switch violates encoding in a message it sends, this usually can be seen in Trema log files.

--
Denis Ovsienko

[Muxi Yan]

Thank you Denis for these information. I also noticed the hello timeout problem 

that you metioned. In this case the switch and trema cannot establish connection 

after a number of attempts. I did not find a solution to that problem as well. 

Maybe the same problem exists for packet_in. Did you have a solution to this problem?

Also I do not quite understand why it will timeout since it is only 1 switch-1 

controller configuration, and disconnection seems to happen very fast (within 1s).

About your second suggestion, we checked the frame of the packet_in messages

but we did not find anything abnormal there. The trema-edge log also shows that

the packet_in messages seem to have been processed normally. 

Best,

Muxi Yan

[Denis Ovsienko]

---- On Fri, 16 Jan 2015 22:50:22 +0000 Muxi Yan wrote ----

>Thank you Denis for these information. I also noticed the hello timeout problem that you metioned. In this case the switch and trema cannot establish connection
>after a number of attempts. I did not find a solution to that problem as well.
>Maybe the same problem exists for packet_in. Did you have a solution to this problem?

Greetings.

I have not had any issues with packet-in messages, however, my work is limited to regular Trema and thus OF1.0. It may help to work out exact steps to reproduce and file an issue for trema-edge.

--
Denis Ovsienko