OpenFlow theory and implementation in ONOS

[Sergio Vera Martínez]

Hello everyone,

I am developing my custom forward application and I wanted to install all the flow rules (bidirectional) right from the source Device. As far as I know, with the first incoming packet, the switch will ask the controller what to do with this packet. This is the moment if I am not wrong when the packet is processed using the process method (existent in ReactiveForwarding). In my case, I have developed a method that installs all the necessary rules given a path. What goes beyond my understanding is why another switch in this path asks again to the controller. As can be seen in the screenshot, the difference between installing the flows again is 7ms. I suppose if this switch has the flows already installed, this should not happen. I only call the packetOut method when all flows are installed, to avoid problems. What could I do? Thanks in advance.

Have everyone a good Christmas,

Best regards

PS: Maybe the time it takes the switch to change the state of a flow entry from PENDING_ADD to ADDED is greater than this 5ms-ish (I attach another image). What could I do in that case?

[Eder Ollora]

Hi,

Without seeing the code or testing the system this is difficult so I might be wrong on some of my assumptions. If you install all two-way rules in the path (let's say when the first switch in the path sends a PacketIn to the controller) then you have to decide when to call PacketOut. If you call PacketOut right away (or even after some time) and another switch sends a PacketIn that means no rules have been matched (yet). So FlowMod messages have not been processed or installed, etc. You will always encounter this situation with the first packet(s) because if you don't wait until all rules have been installed you will experience this situation in some of the switches in the path. I am not completely sure about this but I guess that the reason for the default ReactiveForwarding doing flow installations per switch (I think that they installed one rule for the current PacketIn switch) is that you don't run this "race condition". If you do it per switch you get one PacketIn per switch and you don't have to wait for rules for being installed and assume you will get one PacketIn request per switch on each way. This simplifies the procedure significantly. You could, of course, get several PacketIn messages from the subsequent packets being sent from the sender until the rules have been installed. I don't remember how ReactiveForwarding dealt with this case. Maybe they still send more FlowMod messages to prevent tracking what they have done in the past.

In your case, if you want to avoid the case of getting several PacketIn messages from subsequent switches, you might need to wait until the controller can send the packet to the next hop (wait until all rules are installed). This might also imply significant issues, for instance, especially if the flow sends packets continuously, so you might experience several packets sent as PacketIn requests from the first switch to the controller. You might need to buffer packets at the control plane until rules are sent to the next hop (because you need to wait for rules to be installed). This might be a problem in your case if data rates are high. One solution I can come up with is tracking flow installations, but I am not sure if this is a better procedure than the one offered by default ReactiveForwarding. What I mean is that you can install all rules when the first switch sends the first PacketIn and send the packet out right away to the proper output port (to send it to the next switch). You could, in principle, track that you have sent all the flow installations to the switches for this particular packet and just send the packet out in any next PacketIn request you get (to prevent multiple FlowMod messages). This might be interesting to prevent buffering every single packet that does not match a rule and also to prevent delaying the packets. When you detect that the rules have been installed then remove it from your own "tracker". 

Not sure if my reply covers your question. 

Happy Christmas.

[Sergio Vera Martínez]

Hi Eder,

Thank you so much for your time answering my question. As you have said, if I wait for all flow rules to be installed it wouldn't be efficient for big topologies (when dozens or even hundreds of flows are installed). Basically, I was asking this question for this problem: I have the topology in the attached screenshot, where S1 has two paths of equal weight to go to S7, and the default algorithm is Dijkstra. If I install the bidirectional flows only per switch and call packetOut, I will encounter that when S7 calculates the path using Dijkstra, sometimes it will use the path S7-S2-S1 and others S7-S3-S1. This is why I wanted to install all the flow rules in the path straight, to avoid this problem. So far, using this technique I haven't had any problem except that rules are installed twice (although duplicated ones are deleted in 5s). I don't know if there is any solution to the problem mentioned. 

Merry Christmas!

Best regards

El dia dijous, 24 de desembre de 2020 a les 21:39:34 UTC+1, Eder Ollora va escriure:

[Eder Ollora]

Hi,

1) I understand the issues about installing rules at each switch, which can make the system use different paths when the cost is the same. Before going deeper into your multiple flow installations, a simple solution you might take to solve the criteria to always pick the same path is to always take the same path based on the same criteria. For instance, with S7-2-1 and S7-3-1, you might decide to pick always the path that holds the set of switches with the "lowest" device id number. For instance, you can pick S7-2-1 because 2 is lower than 3, etc. If you have more hops like (imagine) S7-6-2-1 and S7-5-3-1, then you might see that at hop 2 (S6 compared to S5) the device id is lower of S5 compared to S6, so you pick the path S7-5-3-1. This criterion can still be an issue if you plan to load balance the traffic at some point. 

Unless I am mistaken, your main problem is that "rules are installed twice" as you said, is that correct? I don't understand though why only twice and not maybe 10 or 100 times. Imagine you get the same packet every 1 ms in the controller because of PacketIn from a switch, and every time you process it you try to install the same rules of course. Unless ONOS detects that you already tried the same flow installations then you install the same rule every millisecond. To solve this you need to wait (imagine) 30 ms until you detect that flows are installed. In this case, you will get 30 packets as PacketIn and you might try 30 flow installations. To detect that you already tried to install the same flow rule you might try to store the flow installation information in a shared variable (say a Map) and check the map before trying to install the rules. Then remove the information from the map when an async function of your program detects that it has been installed.

2) About the issue we talked about in https://groups.google.com/u/2/a/onosproject.org/g/onos-dev/c/wjTHBI5WfBk, check if different versions of the controller present the same issues. Make sure GUI2 is running and check if the problems are the same. As we talked use different browsers and see if any of them has issues checking the console. Unfortunately, that is as far as I can go. I am not totally sure what is wrong, or which is the main issue. I can still try to contact on that thread later on if you ask any other question.

3) Finally, about monitoring networks, the question is what do you want to monitor? Some things can already be monitored using the ONOS Web GUI (port utilization, device up/down, etc.). You can also use the REST API from ONOS like checking flows from devices, check if a device is up or down, etc. You might be able to use this information in tools like Grafana or Prometheus.

Cheers,

[Sergio Vera Martínez]

Hi Eder,

1) As you've said, I've created a HashMap containing the installed ForwardingObjectives to avoid multiple installations. When I receive a RULE_ADDED eventI check if the rule added exists in the shared HashMap. If so, I remove it. Thanks to this, I've solved the problem of multiple additions but another problem has raised. While the ping is running it seems that those duplicate flows that weren't installed are stuck in the ADD_REQUESTED event (see the attached screenshot). Why could this be possible?

2) I've answered this in this thread: https://groups.google.com/u/2/a/onosproject.org/g/onos-dev/c/wjTHBI5WfBk

3) I just wanted to monitor the transmitted and received traffic in the ports' switches but in a graphical way. I have installed the sFlow-RT tool but the ways it prints the traffic is quite overwhelming (it overlaps the traffic) as shown in the screenshot. It's a shame that I can't use tools like Cacti in mininet SDN networks. 

Thank you so much for your time Eder,

Best regards

El dia dissabte, 26 de desembre de 2020 a les 20:50:33 UTC+1, Eder Ollora va escriure:

[Sergio Vera Martínez]

Hi Eder,

First of all happy new year!

Regarding the three questions covered in this thread, I've been able to solve two of them:

1) I've modified the HashMap and now everything works properly, but I'd like to point out something about PACKET_INs. As you've maybe seen in the "flow-rule-addition-requested" screenshot every five seconds or so I receive Flow Rule Addition requests despite the path being already established. Taking a look at Wireshark I see lots of PACKET_IN coming from the different interfaces of the switches. To discard that I wasn't something wrong about my app, I've tried to use the same FlowRuleListener in the "fwd" app and it happens the same. Is this something related to ONOS version 2.4.0? As far as I know, there shouldn't be PACKET_INs when the path is established, am I missing something?

2) I haven't been able to solve this problem yet, nor with version 2.5.0. Honestly, I don't know why when a device is placed using latitude and longitude traffic monitoring labels don't work. Taking a look at past posts and at the ONOS Wiki, it seems that it worked with versions prior to 2.0.0.

3) This one is also solved. I've used Grafana with InfluxDB as a time-series database.

Thank you so much for your help!

Best regards

El dia diumenge, 27 de desembre de 2020 a les 18:31:57 UTC+1, Sergio Vera Martínez va escriure:

[Eder Ollora]

Hi Sergio,

Sorry for the delay, here are some of my comments:

1) The way I understand it, if you receive PACKET_IN messages at the controller then the path is not be established (I mean rules are not installed yet), else you'd not get packets at the controller. Can you explain what you mean by: "I see lots of PACKET_IN coming from the different interfaces of the switches"? We might need to look closer to this case.

2) In your case I would discard anything related to longitude and latitude positioning if you really need to make it work properly. If you need to make a location for switches, you might as well do it in a separate figure using Visio or a similar tool., And the use the default mode in ONOS without using maps and the like.

3) Nice to see that.

Please let me know the issues you have with 1) and 2), maybe I can help. 

Cheers!

[Sergio Vera Martínez]

Hi Eder,

Don't worry at all for the delay! Answering the questions:

1) Exactly, what you've said is what concerns me because it would mean that the path hasn't been established yet. That said, all the flow necessary flow rules are installed. What I've mentioned in my previous message is depicted in the attached screenshot "flow-rule-addition-requested" and in the attached Wireshark capture in zip format so it's supported by Google Groups (I performed a basic ping between two hosts). You'll see how PACKET_INs are sent despite having apparently established the path. What's more, I realized that this also happens when doing a ping using the default forwarding app (fwd), which got me even more confused.

2) I'll need the latitude and longitude to represent the GEANT network in the end (or any other network but somewhat similar). By Visio do you mean Microsoft Visio (is the only software that comes to my mind)? Let's hope if this thread or the other is seen by many developers so it can be solved in the future (I initially thought of changing the core code but it was very difficult for me). 

Best regards

PS: The Wireshark capture is done in the "any" interface with filter "openflow_v4" and from the very beginning the ping was running

El dia dijous, 7 de gener de 2021 a les 19:05:11 UTC+1, Eder Ollora va escriure:

[Eder]

Hi,

1) Can you share your project as of the code you developed as of today I am not sure if I checked it already but maybe I can take a look again. I see that you get a series of packet in requests. This is weird, to be honest.

2) I meant that you can use Visio, Lucidchart, draw.io, or any other software to represent the network and place it in your report/paper but then stick to the default layout in ONOS so you have no problems. Sometimes you have to make tradeoffs because to solve the problems you see you might need to debug quite some code, likely javascript if the issue is on the frontend. You might still be able to do so but maybe you prefer to first solve properly the issue on 1) instead of this.

Cheers,

Virus-free. www.avast.com

On Thu, 7 Jan 2021 at 20:30, Sergio Vera Martínez <sergio.ver...@estudiantat.upc.edu> wrote:

Hi Eder,

Don't worry at all for the delay! Answering the questions:

1) Exactly, what you've said is what concerns me because it would mean that the path hasn't been established yet. That said, all the flow necessary flow rules are installed. What I've mentioned in my previous message is depicted in the attached screenshot "flow-rule-addition-requested" and in the attached Wireshark capture (I performed a basic ping between two hosts). You'll see how PACKET_INs are sent despite having apparently established the path. What's more, I realized that this also happens when doing a ping using the default forwarding app (fwd), which got me even more confused.

2) I'll need the latitude and longitude to represent the GEANT network in the end (or any other network but somewhat similar). By Visio do you mean Microsoft Visio (is the only software that comes to my mind)? Let's hope if this thread or the other is seen by many developers so it can be solved in the future (I initially thought of changing the core code but it was very difficult for me). 

Best regards

PS: The Wireshark capture is done in the "any" interface with filter "openflow_v4" and from the very beginning the ping was running

Virus-free. www.avast.com

[Sergio Vera Martínez]

Hi Eder,

1) I attach to this thread the link to my repo (the app is routing-app): https://github.com/SergiVera/onos. Apart from this, I don't know if you've already tried it but using the already-existent fwd app does happen exactly the same. It might be my mininet topology when configuring the remote controller. It's quite weird honestly.

2) I understand what you say, I take note. Furthermore, I think it would be great if the developers fix the error since it's present from the very beginning of GUI2.

Best regards

El dia dijous, 7 de gener de 2021 a les 23:08:36 UTC+1, Eder Ollora va escriure: