Flow add problem

2,243 views
Skip to first unread message

Carl Klatsky

unread,
Feb 22, 2017, 1:51:13 PM2/22/17
to ONOS Discuss
Hi,

Continuing on from the items discussed in topic "Segment routing application questions" using the same configuration, I am trying to 'hard code' flows by inserting them with the web interface to the REST API.  When I add the flow, the web UI reports back a 200 OK to the API call, but when checked on the ONOS CLI the flow stays in the PENDING ADD state.  An error appears in the ONOS logs, but its not clear to me what the error means.

I have attached the *.json used in the flow add, the flow listing before & after the change, and the ONOS log snippet.  The full curl command is shown below.  Any help is appreciated.

Thanks & Regards,
Carl Klatsky

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{ \ 
   "flows": [ \ 
     { \ 
       "priority": 1, \ 
       "timeout": 0, \ 
       "isPermanent": true, \ 
       "deviceId": "of:0000000000000001", \ 
        "treatment": { \ 
           "instructions": [ \ 
           { \ 
             "type": "OUTPUT", \ 
             "port": "9" \ 
           } \ 
         ] \ 
       }, \ 
         "selector": { \ 
           "criteria": [ \ 
           { \ 
           "type": "ETH_TYPE", \ 
           "ethType": "0x800" \ 
           }, \ 
           { \ 
            "type": "IN_PORT", \ 
            "port": "1" \ 
           }, \ 
           { \ 
             "type": "IPV4_SRC", \ 
             "ip": "10.255.227.0/24" \ 
           } \ 
         ] \ 
       } \ 
     } \ 
   ] \ 
 }' 'http://10.255.14.167:8181/onos/v1/flows?appId=org.onosproject.segmentrouting'
onos_ flows_after2.txt
onos_ flows_before2.txt
onos_error.txt
flow_vsg_v3.txt

Saurav Das

unread,
Feb 23, 2017, 1:33:46 AM2/23/17
to Carl Klatsky, ONOS Discuss
Carl

I'll take a closer look tomorrow but at a minimum the flow needs to specify a table id of 60 which is the acl table. By default it will go to table 0 which won't work.

Also you need to point the flow out of a group (not a port) and there are specific rules for groups 

I'll give you a list of flows and groups to try - we have not tried the rest API to do this but I'm glad you are trying - so we can get some feedback

Thanks
Saurav

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.
To post to this group, send email to onos-d...@onosproject.org.
Visit this group at https://groups.google.com/a/onosproject.org/group/onos-discuss/.
To view this discussion on the web visit https://groups.google.com/a/onosproject.org/d/msgid/onos-discuss/f6d55e30-919d-453c-9dd6-b0cf9d8f9fc3%40onosproject.org.
--
The will MUST be stronger than the skill -- Ali

Klatsky, Carl

unread,
Feb 23, 2017, 11:26:39 AM2/23/17
to Saurav Das, ONOS Discuss

Saurav,

 

To clarify, would the sequence be:

 

1)      An entry in table 0 to match source IP subnet and sends the packet to ACL table 60; if no match then continue onto the Bridge table for destination MAC forwarding

2)      An entry in the ACL table that matches source IP subnet and sends the packet to the Group table

3)      A Group table entry with an Action to send the packet out Physical Port X

 

Just trying to correlate which flow sequence this would be when looking at the reference flows here:

 

https://wiki.opencord.org/display/CORD/Fabric+Design+Note

 

Thanks,

Carl Klatsky

Saurav Das

unread,
Feb 23, 2017, 10:28:44 PM2/23/17
to Klatsky, Carl, ONOS Discuss
Hi Carl,

Your sequence would be
1) Add groups for pushing packet out of port 9 with the right internal vlan 4094 popped off. This would actually require two groups -- one called an L3_Unicast which sets the vlan to 4094, and the other called L2_Interface which pops off this internal vlan tag and sends the packet out of port 9. (Side note: the packets don't actually "go" to a group-table, but they do go out of the groups. The group-table is not a forwarding table -- not in the packet-path. It's just a store for all the groups programmed in the switch)
1a) The order in which the groups are sent to the switch matters. You need to send the L2_interface group first, and then the L3_Unicast group that points to the L2_Interface group

2) Then you add the ACL table entry to match on the src IP subnet and point it to the L3 Unicast group. There is nothing needed in table 0.

L2Interface group: 

   id=0xffe0009, type=INDIRECT, a single bucket with actions=[VLAN_POP, OUTPUT:9]

L3Unicast group:

  id=0x20001000, type=INDIRECT, a single bucket with actions=[ETH_DST:<use eth of dst-router on port 5 - I think 34:DB:FD:03:06:41>, ETH_SRC:<use eth of src-router on port 1-  I think 00:21:A0:CE:2B:24>, VLAN_ID:4094, GROUP:0xffe0009]
 
ACL Table Entry:

priority=65535, tableId=60,  selector=[In_Port=1, ETH_TYPE:0x800, IPV4_DST:10.255.227.0/24], treatment=DefaultTrafficTreatment{immediate=[], deferred=[GROUP=0x20001000], transition=None, meter=None, cleared=false, metadata=null}


A few things to point out:
  • You haven't sent us the output of "groups" cli command, so I am not completely sure what groups you have in the switch right now. The L2Interface group may already be there for example. But it does not hurt to try and add it again
  • I have chosen the L3Unicast group id to be something that is likely not there in the switch. Note that the mac addresses need to be set when using this group. But we can set it to the router mac addresses which we know, so in the end the mac addresses on the packet will remain unchanged.
  • The L3Unicast group changes the incoming (internally) assigned vlan on port 1 (4092) to the outgoing assigned vlan on port 9 (4094). The L2 Interface group will pop this off before egress.
  • Give the ACL table entry the highest priority 65535. BTW on Broadcom chips, the ACL table is the only table that packet/byte counters go up when packets match on rules. Flows in all other tables will show 0 for pkt/byte counters even though packets are matching them
  • The ACL table entry has two kinds of "treatments" - immediate and deferred -- you need to use the deferred treatment as this is what will overwrite the deferred treatment set in the bridging table entry that these packets would have normally been forwarded with. 
For reference the bridging table entry (table 50) is this one:

id=400000aad71fed, state=ADDED, bytes=0, packets=0, duration=7397, priority=100, tableId=50, appId=org.onosproject.segmentrouting, payLoad=null, selector=[ETH_DST:34:DB:FD:03:06:41, EXTENSION:of:0000000000000001/OfdpaMatchVlanVid{vlanId=4092}], treatment=DefaultTrafficTreatment{immediate=[], deferred=[GROUP:0xffc0005], transition=TABLE:60, meter=None, cleared=false, metadata=null}

Now the problems:
  • I have no idea if you can actually program these flows and groups through the rest api. Some functionality may be missing in the rest api that may not allow you to do this - for example the deferred treatments.
  • Even if you are able to get these into the switch (by rest or some other means), I have no idea if this will actually work, since we haven't tried this case. We have used ACL table entries to overrule/override routing table entries, but not bridging table entries. The complication comes in your case from the fact that we have to change vlans going from port 1 to port 9, in order to avoid port 9 being in the same bridging domain as port 1. It should work but we'll never know till we try. If it doesn't work there are other tricks we can play, but we can get to them later :)

Thanks
Saurav

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--

The will MUST be stronger than the skill -- Ali

Klatsky, Carl

unread,
Feb 24, 2017, 4:31:18 PM2/24/17
to Saurav Das, ONOS Discuss

Thanks for the details Saurav.  A lot of good information here that I need to let sink in, and then try out next week.  Will share the results back to the mailing list next week.

 

Regards,

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--

The will MUST be stronger than the skill -- Ali

Klatsky, Carl

unread,
Mar 1, 2017, 10:44:24 AM3/1/17
to Klatsky, Carl, Saurav Das, ONOS Discuss

Saurav,

 

To rephrase your instructions back to ensure I understood them, after applying the groups & flows noted below, then the packet flow would be:

 

1)      Incoming packet is processed first by the bridging table (table 50) and moves through the pipeline to the ACL table (not sure how this is linked in?)

2)      ACL table has a flow rule that matches the packet based on the selector criteria and the deferred treatment sends it to the L3 Unicast group 0x20001000

3)      L3 Unicast group 0x20001000 gives the action to adjust the source & destination MAC addresses (really to what they already are), add internal VLAN tag 4094, and sends it on the L2 Interface group 0xffe0009

4)      L2 Interface group 0xffe0009 gives the action to pop off the internal VLAN tag and send it out port 9

 

Hopefully I understood that correctly.  I’ve attached the group listing prior to attempting the change, and it does not look like either group 0x20001000 or 0xffe0009 is in use and should be OK to add them.  Also attached are the JSON files for L2 Interface group, L3 Unicast group, and ACL flow entry.  The JSON is properly formatted, and I think the syntax is correct.  But when I go to first add the L2 Interface group using the web-based API page, I get this error back:

 

{

  "code": 500,

  "message": "org.onosproject.rest.resources.FlowsWebResource.createFlows(FlowsWebResource.java:121)"

}

 

I’m not sure what triggered this error.  Any thoughts?

 

Regards,

Carl Klatsky

groups_before.txt
l3_unicast_group_v1.txt
l2_interface_group_v1.txt
acl_entry_v1.txt

Jonathan Hart

unread,
Mar 1, 2017, 11:18:43 AM3/1/17
to Klatsky, Carl, Saurav Das, ONOS Discuss
Hi Carl,

That error is coming from the Flows REST API handler - is it possible that you are sending the l2_interface_group_v1 JSON to the flows REST API instead of the groups REST API?

Thanks,

To view this discussion on the web visit https://groups.google.com/a/onosproject.org/d/msgid/onos-discuss/703687dae3b84d1c9901cf8755904170%40PACDCEX43.cable.comcast.com.

Klatsky, Carl

unread,
Mar 1, 2017, 3:45:22 PM3/1/17
to Jonathan Hart, Saurav Das, ONOS Discuss

Jono,

 

Thanks, yes, I was trying the group modification under the flows REST API.  I changed to the POST for the groups REST API.  The web interface showed a 201 response instead of a 200.  I see this error in the log.  I had also updated the JSON over what I sent in the original mail.  Revised JSON attached.

 

2017-03-01 14:42:18,727 | ERROR | ew I/O worker #1 | OFChannelHandler                 | 162 - org.onosproject.onos-of-ctl - 1.7.0 | OFGroupModFailedErrorMsgVer13(xid=70, code=INVALID_GROUP, data=OFGroupAddVer13(xid=70, groupType=INDIRECT, group=0, buckets=[OFBucketVer13(weight=0, watchPort=any, watchGroup=any, actions=[OFActionOutputVer13(port=9, maxLen=0), OFActionPopVlanVer13()])])) from switch org.onosproject.driver.handshaker.DefaultSwitchHandshaker [/10.255.14.164:37356 DPID[00:00:00:00:00:00:00:01]] in state ACTIVE

2017-03-01 14:42:18,729 | WARN  | vent-error-msg-0 | OpenFlowRuleProvider             | 165 - org.onosproject.onos-of-provider-flow - 1.7.0 | Received error message OFGroupModFailedErrorMsgVer13(xid=70, code=INVALID_GROUP, data=OFGroupAddVer13(xid=70, groupType=INDIRECT, group=0, buckets=[OFBucketVer13(weight=0, watchPort=any, watchGroup=any, actions=[OFActionOutputVer13(port=9, maxLen=0), OFActionPopVlanVer13()])])) from 00:00:00:00:00:00:00:01

2017-03-01 14:42:18,731 | WARN  | vent-error-msg-0 | OpenFlowGroupProvider            | 166 - org.onosproject.onos-of-provider-group - 1.7.0 | Received a group mod error OFGroupModFailedErrorMsgVer13(xid=70, code=INVALID_GROUP, data=OFGroupAddVer13(xid=70, groupType=INDIRECT, group=0, buckets=[OFBucketVer13(weight=0, watchPort=any, watchGroup=any, actions=[OFActionOutputVer13(port=9, maxLen=0), OFActionPopVlanVer13()])]))

2017-03-01 14:42:18,731 | WARN  | vent-error-msg-0 | DistributedGroupStore            | 78 - org.onosproject.onos-core-dist - 1.7.0 | groupOperationFailed: group operation ADD failedfor group DefaultGroupId{id=0x0} in device of:0000000000000001 with code INVALID_GROUP

2017-03-01 14:42:18,732 | WARN  | vent-error-msg-0 | DistributedGroupStore            | 78 - org.onosproject.onos-core-dist - 1.7.0 | groupOperationFailed: cleaningup group DefaultGroupId{id=0x0} from store in device of:0000000000000001....

Also say this error on the switch CLI which I happened to have open

 

03-26 23:23:14.470750 [indigo_ofdpa_driver] Error in adding Group, rv = -22

 

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{ \

   "type": "INDIRECT", \

   "appCookie": "0x1234abcd", \

   "groupId": "0xffe0009", \

   "buckets": [ \

    { \

      "treatment": { \

        "instructions": [ \

           { \

             "type": "OUTPUT", \

             "port": "9" \

           }, \

           { \

             "type":"L2MODIFICATION", \

             "subtype":"VLAN_POP", \

             "vlanId": "4094" \

           } \

         ] \

       } \

     } \

   ] \

 } \

 ' 'http://10.255.14.167:8181/onos/v1/groups/of%3A0000000000000001'

 

 

Regards,

Carl Klatsky

l2_interface_group_v2.txt

Saurav Das

unread,
Mar 1, 2017, 4:28:42 PM3/1/17
to Klatsky, Carl, Jonathan Hart, ONOS Discuss
It seems to me that even though you are sending the groupid 0xffe0009 in your rest call, the ONOS group subsystem is pushing groupId = 0 to the switch.

2017-03-01 14:42:18,731 | WARN  | vent-error-msg-0 | DistributedGroupStore            | 78 - org.onosproject.onos-core-dist - 1.7.0 | groupOperationFailed: group operation ADD failedfor group DefaultGroupId{id=0x0} in device of:0000000000000001 with code INVALID_GROUP

which of-course the switch rejects.

Not sure why the groupid does not go thru ...


To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

Saurav Das

unread,
Mar 1, 2017, 4:37:43 PM3/1/17
to Klatsky, Carl, Jonathan Hart, ONOS Discuss
Try passing in the groupid as decimal == > 0xffe0009 == 268304393

Klatsky, Carl

unread,
Mar 2, 2017, 12:07:11 PM3/2/17
to Saurav Das, Jonathan Hart, ONOS Discuss

By changing the group ID to decimal, I was able to add the L2 Interface group.  When I went to add the L3 Unicast group using a unique appCookie value and also setting the group ID to decimal, I get the errors below on the ONOS logs and switch CLI.  Is my syntax for the L3 Unicast group correct?

 

+++ONOS Logs++++

2017-03-02 10:44:12,887 | ERROR | ew I/O worker #1 | OFChannelHandler                 | 162 - org.onosproject.onos-of-ctl - 1.7.0 | OFGroupModFailedErrorMsgVer13(xid=177, code=INVALID_GROUP, data=OFGroupAddVer13(xid=177, groupType=INDIRECT, group=536875008, buckets=[OFBucketVer13(weight=0, watchPort=any, watchGroup=any, actions=[OFActionSetFieldVer13(field=OFOxmEthDstVer13(value=34:db:fd:03:06:41)), OFActionSetFieldVer13(field=OFOxmEthSrcVer13(value=00:21:a0:ce:2b:24)), OFActionPushVlanVer13(ethertype=0x8100)])])) from switch org.onosproject.driver.handshaker.DefaultSwitchHandshaker [/10.255.14.164:37370 DPID[00:00:00:00:00:00:00:01]] in state ACTIVE

2017-03-02 10:44:12,889 | WARN  | vent-error-msg-0 | OpenFlowRuleProvider             | 165 - org.onosproject.onos-of-provider-flow - 1.7.0 | Received error message OFGroupModFailedErrorMsgVer13(xid=177, code=INVALID_GROUP, data=OFGroupAddVer13(xid=177, groupType=INDIRECT, group=536875008, buckets=[OFBucketVer13(weight=0, watchPort=any, watchGroup=any, actions=[OFActionSetFieldVer13(field=OFOxmEthDstVer13(value=34:db:fd:03:06:41)), OFActionSetFieldVer13(field=OFOxmEthSrcVer13(value=00:21:a0:ce:2b:24)), OFActionPushVlanVer13(ethertype=0x8100)])])) from 00:00:00:00:00:00:00:01

2017-03-02 10:44:12,891 | WARN  | vent-error-msg-0 | OpenFlowGroupProvider            | 166 - org.onosproject.onos-of-provider-group - 1.7.0 | Received a group mod error OFGroupModFailedErrorMsgVer13(xid=177, code=INVALID_GROUP, data=OFGroupAddVer13(xid=177, groupType=INDIRECT, group=536875008, buckets=[OFBucketVer13(weight=0, watchPort=any, watchGroup=any, actions=[OFActionSetFieldVer13(field=OFOxmEthDstVer13(value=34:db:fd:03:06:41)), OFActionSetFieldVer13(field=OFOxmEthSrcVer13(value=00:21:a0:ce:2b:24)), OFActionPushVlanVer13(ethertype=0x8100)])]))

2017-03-02 10:44:12,891 | WARN  | vent-error-msg-0 | DistributedGroupStore            | 78 - org.onosproject.onos-core-dist - 1.7.0 | groupOperationFailed: group operation ADD failedfor group DefaultGroupId{id=0x20001000} in device of:0000000000000001 with code INVALID_GROUP

2017-03-02 10:44:12,894 | WARN  | vent-error-msg-0 | DistributedGroupStore            | 78 - org.onosproject.onos-core-dist - 1.7.0 | groupOperationFailed: cleaningup group DefaultGroupId{id=0x20001000} from store in device of:0000000000000001....

 

+++Switch CLI+++

03-27 19:25:09.308985 [indigo_ofdpa_driver] Incompatible fields for Group Type

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--

You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

groups_after_l2_interface_group_added_v1.txt
l3_unicast_group_v2.txt
l2_interface_group_v2.txt

Charles Chan

unread,
Mar 2, 2017, 9:33:39 PM3/2/17
to Klatsky, Carl, Saurav Das, Jonathan Hart, ONOS Discuss
On Thu, Mar 2, 2017 at 9:07 AM Klatsky, Carl <Carl_K...@comcast.com> wrote:

By changing the group ID to decimal, I was able to add the L2 Interface group.  When I went to add the L3 Unicast group using a unique appCookie value and also setting the group ID to decimal, I get the errors below on the ONOS logs and switch CLI.  Is my syntax for the L3 Unicast group correct?

 

+++ONOS Logs++++

2017-03-02 10:44:12,887 | ERROR | ew I/O worker #1 | OFChannelHandler                 | 162 - org.onosproject.onos-of-ctl - 1.7.0 | OFGroupModFailedErrorMsgVer13(xid=177, code=INVALID_GROUP, data=OFGroupAddVer13(xid=177, groupType=INDIRECT, group=536875008, buckets=[OFBucketVer13(weight=0, watchPort=any, watchGroup=any, actions=[OFActionSetFieldVer13(field=OFOxmEthDstVer13(value=34:db:fd:03:06:41)), OFActionSetFieldVer13(field=OFOxmEthSrcVer13(value=00:21:a0:ce:2b:24)), OFActionPushVlanVer13(ethertype=0x8100)])])) from switch org.onosproject.driver.handshaker.DefaultSwitchHandshaker [/10.255.14.164:37370 DPID[00:00:00:00:00:00:00:01]] in state ACTIVE


Please change push vlan to set vlan 4094 and see if it works.
 
--
Charles Chan
Member of Technical Staff, ON.Lab

Saurav Das

unread,
Mar 2, 2017, 9:55:01 PM3/2/17
to Charles Chan, Klatsky, Carl, Jonathan Hart, ONOS Discuss
Also add one more action -- a group action to point to the L2 Interface group

id=0x20001000, type=INDIRECT, a single bucket with actions=[ETH_DST:<use eth of dst-router on port 5 - I think 34:DB:FD:03:06:41>, ETH_SRC:<use eth of src-router on port 1-  I think 00:21:A0:CE:2B:24>, VLAN_ID:4094, GROUP:0xffe0009]

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

To post to this group, send email to onos-d...@onosproject.org.
Visit this group at https://groups.google.com/a/onosproject.org/group/onos-discuss/.
--
Charles Chan
Member of Technical Staff, ON.Lab

Klatsky, Carl

unread,
Mar 3, 2017, 3:46:13 PM3/3/17
to Saurav Das, Charles Chan, Jonathan Hart, ONOS Discuss

I included the changes noted by both Charles & Saurav.  In my first try I had set the group value in L3 Unicast group using the hex value and got an error back.  I changed that to a decimal value, and the web UI for the REST API reported back a 201 response.  There were also no errors seen on the switch CLI.  But in the ONOS console, the new group is in state PENDING ADD and an exception error was seen in the ONOS logs.

 

2017-03-03 14:41:50,679 | ERROR | f-event-stats-20 | OpenFlowControllerImpl           | 162 - org.onosproject.onos-of-ctl - 1.7.0 | Uncaught exception on onos-of-event-stats-20

java.lang.IllegalArgumentException: value exceeds allowed maximum VLAN ID value (4095)

        at org.onlab.packet.VlanId.vlanId(VlanId.java:78)[72:org.onosproject.onlab-misc:1.7.0]

        at org.onosproject.driver.extensions.OfdpaExtensionTreatmentInterpreter.mapAction(OfdpaExtensionTreatmentInterpreter.java:68)[159:org.onosproject.onos-drivers:1.7.0]

        at org.onosproject.provider.of.flow.util.FlowEntryBuilder.handleSetField(FlowEntryBuilder.java:452)[165:org.onosproject.onos-of-provider-flow:1.7.0]

        at org.onosproject.provider.of.flow.util.FlowEntryBuilder.configureTreatmentBuilder(FlowEntryBuilder.java:364)[165:org.onosproject.onos-of-provider-flow:1.7.0]

        at org.onosproject.provider.of.group.impl.GroupBucketEntryBuilder.buildTreatment(GroupBucketEntryBuilder.java:127)[166:org.onosproject.onos-of-provider-group:1.7.0]

        at org.onosproject.provider.of.group.impl.GroupBucketEntryBuilder.build(GroupBucketEntryBuilder.java:82)[166:org.onosproject.onos-of-provider-group:1.7.0]

        at org.onosproject.provider.of.group.impl.OpenFlowGroupProvider.buildGroupMetrics(OpenFlowGroupProvider.java:251)[166:org.onosproject.onos-of-provider-group:1.7.0]

        at org.onosproject.provider.of.group.impl.OpenFlowGroupProvider.pushGroupMetrics(OpenFlowGroupProvider.java:229)[166:org.onosproject.onos-of-provider-group:1.7.0]

        at org.onosproject.provider.of.group.impl.OpenFlowGroupProvider.access$100(OpenFlowGroupProvider.java:86)[166:org.onosproject.onos-of-provider-group:1.7.0]

        at org.onosproject.provider.of.group.impl.OpenFlowGroupProvider$InternalGroupProvider.handleMessage(OpenFlowGroupProvider.java:328)[166:org.onosproject.onos-of-provider-group:1.7.0]

        at org.onosproject.openflow.controller.impl.OpenFlowControllerImpl$OFMessageHandler.run(OpenFlowControllerImpl.java:757)[162:org.onosproject.onos-of-ctl:1.7.0]

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_102]

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_102]

        at java.lang.Thread.run(Thread.java:745)[:1.8.0_102]

 

onos> groups

deviceId=of:0000000000000001, groupCount=19

   id=0xffb0006, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0xffb0006, bucket=1, bytes=0, packets=0, actions=[VLAN_POP:unknown, OUTPUT:6]

   id=0xffb0007, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0xffb0007, bucket=1, bytes=0, packets=0, actions=[VLAN_POP:unknown, OUTPUT:7]

   id=0xffc0002, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0xffc0002, bucket=1, bytes=0, packets=0, actions=[VLAN_POP:unknown, OUTPUT:2]

   id=0xffc0003, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0xffc0003, bucket=1, bytes=0, packets=0, actions=[VLAN_POP:unknown, OUTPUT:3]

   id=0xffc0004, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0xffc0004, bucket=1, bytes=0, packets=0, actions=[VLAN_POP:unknown, OUTPUT:4]

   id=0xffd0001, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0xffd0001, bucket=1, bytes=0, packets=0, actions=[VLAN_POP:unknown, OUTPUT:1]

   id=0xffd0005, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0xffd0005, bucket=1, bytes=0, packets=0, actions=[VLAN_POP:unknown, OUTPUT:5]

   id=0xffe0009, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.rest

   id=0xffe0009, bucket=1, bytes=0, packets=0, actions=[OUTPUT:9, VLAN_POP:unknown]

   id=0x20000001, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0x20000001, bucket=1, bytes=0, packets=0, actions=[ETH_DST:5C:B9:01:9A:16:69, ETH_SRC:10:00:00:00:00:02, EXTENSION:of:0000000000000001/OfdpaSetVlanVid{vlanId=4092}, GROUP:0xffc0003]

   id=0x20000002, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0x20000002, bucket=1, bytes=0, packets=0, actions=[ETH_DST:0C:4D:E9:D1:4D:6D, ETH_SRC:10:00:00:00:00:02, EXTENSION:of:0000000000000001/OfdpaSetVlanVid{vlanId=4091}, GROUP:0xffb0006]

   id=0x20000003, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0x20000003, bucket=1, bytes=0, packets=0, actions=[ETH_DST:34:DB:FD:03:06:41, ETH_SRC:10:00:00:00:00:02, EXTENSION:of:0000000000000001/OfdpaSetVlanVid{vlanId=4093}, GROUP:0xffd0005]

   id=0x20000004, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0x20000004, bucket=1, bytes=0, packets=0, actions=[ETH_DST:D4:6D:50:33:67:BD, ETH_SRC:10:00:00:00:00:02, EXTENSION:of:0000000000000001/OfdpaSetVlanVid{vlanId=4092}, GROUP:0xffc0002]

   id=0x20000005, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0x20000005, bucket=1, bytes=0, packets=0, actions=[ETH_DST:5C:B9:01:9A:0F:AD, ETH_SRC:10:00:00:00:00:02, EXTENSION:of:0000000000000001/OfdpaSetVlanVid{vlanId=4092}, GROUP:0xffc0004]

   id=0x20000006, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0x20000006, bucket=1, bytes=0, packets=0, actions=[ETH_DST:D4:BE:D9:2B:76:2E, ETH_SRC:10:00:00:00:00:02, EXTENSION:of:0000000000000001/OfdpaSetVlanVid{vlanId=4091}, GROUP:0xffb0007]

   id=0x20000007, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0x20000007, bucket=1, bytes=0, packets=0, actions=[ETH_DST:00:21:A0:CE:2B:24, ETH_SRC:10:00:00:00:00:02, EXTENSION:of:0000000000000001/OfdpaSetVlanVid{vlanId=4093}, GROUP:0xffd0001]

   id=0x20001000, state=PENDING_ADD, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.rest

   id=0x20001000, bucket=1, bytes=0, packets=0, actions=[ETH_DST:34:DB:FD:03:06:41, ETH_SRC:00:21:A0:CE:2B:24, VLAN_ID:4094, GROUP:0xffe0009]

   id=0x4ffb0000, state=ADDED, type=ALL, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0x4ffb0000, bucket=1, bytes=0, packets=0, actions=[GROUP:0xffb0006]

   id=0x4ffb0000, bucket=2, bytes=0, packets=0, actions=[GROUP:0xffb0007]

   id=0x4ffc0000, state=ADDED, type=ALL, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0x4ffc0000, bucket=1, bytes=0, packets=0, actions=[GROUP:0xffc0002]

   id=0x4ffc0000, bucket=2, bytes=0, packets=0, actions=[GROUP:0xffc0003]

   id=0x4ffc0000, bucket=3, bytes=0, packets=0, actions=[GROUP:0xffc0004]

   id=0x4ffd0000, state=ADDED, type=ALL, bytes=0, packets=0, appId=org.onosproject.segmentrouting

   id=0x4ffd0000, bucket=1, bytes=0, packets=0, actions=[GROUP:0xffd0001]

   id=0x4ffd0000, bucket=2, bytes=0, packets=0, actions=[GROUP:0xffd0005]

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--

Charles Chan
Member of Technical Staff, ON.Lab

l3_unicast_group_v4.txt

Saurav Das

unread,
Mar 5, 2017, 12:56:52 AM3/5/17
to Klatsky, Carl, Charles Chan, Jonathan Hart, ONOS Discuss
Carl,

It looks like the l3 unicast group is in the switch. ONOS seeks confirmation from the switch by asking for groups and tries to reconcile it with the info it has in it's group store. This is where the error is happening - there is probably a small difference between the way the rest-call is setting a vlan in the group and the way our app/driver does it. The reconciliation is taking into account one way but not the other. Charles will know this better.

The logs will show these reconciliation errors, and ONOS may believe that the group is not in the switch (as the reconciliation failed -- thus PENDING_ADD) and will keep trying. But it does not matter - the group is in the switch. You are ready to try the ACL table rule.

Thanks
Saurav



To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--

Charles Chan
Member of Technical Staff, ON.Lab



 

--

The will MUST be stronger than the skill -- Ali

Klatsky, Carl

unread,
Mar 6, 2017, 11:19:05 AM3/6/17
to Saurav Das, Charles Chan, Jonathan Hart, ONOS Discuss

Thanks Saurav.  I added the rule in the ACL table, but it seems to be defined with treatment “immediate” instead of “deferred”.  I checked the Flow Rules definition wiki page and could not find the syntax to specify the treatment as “deferred”.  Do you know the JSON format for that syntax?

 

I am also specifying the group in the flow rule using decimal instead of hex.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--

Charles Chan
Member of Technical Staff, ON.Lab



 

--

The will MUST be stronger than the skill -- Ali

flows_after_acl_added_v1.txt
flows_before_acl_added_v1.txt
acl_entry_v1.txt

Saurav Das

unread,
Mar 6, 2017, 1:10:08 PM3/6/17
to Klatsky, Carl, Charles Chan, Jonathan Hart, ONOS Discuss
Yes I was afraid of this - the rest treatment decoder has no concept of deferred - I'll play around with it and get back to you

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--

Charles Chan
Member of Technical Staff, ON.Lab



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

Saurav Das

unread,
Mar 6, 2017, 4:50:36 PM3/6/17
to Klatsky, Carl, Charles Chan, Jonathan Hart, ONOS Discuss
Carl,

There is no way to get around this other than changing source code.
Fortunately it is a simple change. In the file highlighted add the lines which have the + sign

diff --git a/core/common/src/main/java/org/onosproject/codec/impl/TrafficTreatmentCodec.java b/core/common/src/main/java/org/onosproject/codec/impl/TrafficTreatmentCodec.java
index 6ec5a67..1378c3e 100644
--- a/core/common/src/main/java/org/onosproject/codec/impl/TrafficTreatmentCodec.java
+++ b/core/common/src/main/java/org/onosproject/codec/impl/TrafficTreatmentCodec.java
@@ -34,6 +34,7 @@ import static com.google.common.base.Preconditions.checkNotNull;
  */
 public final class TrafficTreatmentCodec extends JsonCodec<TrafficTreatment> {
     private static final String INSTRUCTIONS = "instructions";
+    private static final String DEFERRED = "deferred";
 
     @Override
     public ObjectNode encode(TrafficTreatment treatment, CodecContext context) {
@@ -78,6 +79,15 @@ public final class TrafficTreatmentCodec extends JsonCodec<TrafficTreatment> {
                             instructionsCodec.decode(get(instructionsJson, i),
                                     context)));
         }
+
+        JsonNode deferredJson = json.get(DEFERRED);
+        if (deferredJson != null) {
+            IntStream.range(0, deferredJson.size())
+            .forEach(i -> builder.deferred().add(
+                    instructionsCodec.decode(get(deferredJson, i),
+                            context)));
+        }
+
         return builder.build();
     }
 }

You would need to recompile and launch ONOS again.
Then you should be able to send the following json:

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{ \ 
   "flows": [ \ 
     { \ 
       "priority": 65535, \ 
       "timeout": 0, \ 
       "isPermanent": true, \ 
       "deviceId": "of:0000000000000226", \ 
       "tableId" : 60, \ 
       "treatment": { \ 
           "deferred": [ \ 
           { \ 
             "type": "GROUP", \ 
             "groupId": "536870957" \ 
           } \ 
         ] \ 
       }, \ 
       "selector": { \ 
         "criteria": [ \ 
         { \ 
           "type": "ETH_TYPE", \ 
           "ethType": "0x800" \ 
         }, \ 
         { \ 
            "type": "IN_PORT", \ 
            "port": "1" \ 
         }, \ 
         { \ 
           "type": "IPV4_SRC", \ 
           "ip": "10.255.227.0/24" \ 
          } \ 
         ] \ 
       } \ 
     } \ 
   ] \ 
 }' 'http://10.128.0.216:8181/onos/v1/flows?appId=org.onosproject.segmentrouting%20'


Note the "deferred" syntax is the same as the "instructions" syntax other than of-course the keyword. If you just send "instructions" it is assumed to be "immediate" instructions. You need to send "deferred" to make it deferred instructions. You can also send both "instructions" and "deferred" - although you don't need to do it for the acl example

Let me know if you have any issues

Thanks
Saurav


Charles Chan

unread,
Mar 7, 2017, 3:49:50 PM3/7/17
to Klatsky, Carl, Saurav Das, Jonathan Hart, ONOS Discuss
On Sat, Mar 4, 2017 at 9:56 PM Saurav Das <wildc...@gmail.com> wrote:
Carl,

It looks like the l3 unicast group is in the switch. ONOS seeks confirmation from the switch by asking for groups and tries to reconcile it with the info it has in it's group store. This is where the error is happening - there is probably a small difference between the way the rest-call is setting a vlan in the group and the way our app/driver does it. The reconciliation is taking into account one way but not the other. Charles will know this better.

Carl,

According to the log message, it seems your are using ONOS 1.7.0. I suspect that you didn't specify the driver in the network config. The default OFDPA driver in ONOS 1.7.0 is OFDPA 2, in which set vlan vid expects isPresent bit set to zero. (This does not compliant with OF spec).

If your switch is running OFDPA 3, it will report a set vlan vid with isPresent bit set to one. (This is OF compliant)
However, if the driver is not set, ONOS will try to interpret that directly and results in invalid (>4095) VLAN ID.

Charles
 

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--

Charles Chan
Member of Technical Staff, ON.Lab



 

--

The will MUST be stronger than the skill -- Ali




--
The will MUST be stronger than the skill -- Ali

Klatsky, Carl

unread,
Mar 8, 2017, 11:19:44 AM3/8/17
to Charles Chan, Saurav Das, Jonathan Hart, ONOS Discuss

Charles,

 

Thanks for the suggestion.  I do not specify the driver in the network.json configuration file, but when I check the ONOS CLI it seems to indicate that the switch OFPDA is 2.

 

onos> devices

id=of:0000000000000001, available=true, role=MASTER, type=SWITCH, mfr=Broadcom Corp., hw=OF-DPA 2.0, sw=OF-DPA 2.0, serial=, driver=ofdpa, channelId=10.255.14.164:37435, managementAddress=10.255.14.164, protocol=OF_13

 

The switch is an Accton 5712 and the OFDPA packet loaded on it is ofdpa-i.12.1.1_12.1.1+accton1.7-1_amd64.deb, which I think is OFDPA 2.

 

Maybe the isPresent bit item is not the issue>

 

Regards,

Carl Klatsky

Klatsky, Carl

unread,
Mar 8, 2017, 4:20:38 PM3/8/17
to Saurav Das, Charles Chan, Jonathan Hart, ONOS Discuss

Hi Saurav,

 

I made a backup copy of the file in the same directory then was able to edit the file as needed.  But I do not think my re-build & install steps are correct as the change did not result in the desired outcome when adding that ACL Flow Rule.  After the edit, I just was $ONOS_ROOT/mvn clean install, which I think kicks off the compile followed by the install.  Is that the correct action to re-compile the source?

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--

Charles Chan
Member of Technical Staff, ON.Lab



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

Saurav Das

unread,
Mar 8, 2017, 4:42:23 PM3/8/17
to Klatsky, Carl, Charles Chan, Jonathan Hart, ONOS Discuss
Carl,

Not sure which version you are using but I highly recommend using ONOS 1.8.3  or 1.9 - we use buck to compile


Also if you are running onos locally with a single-instance buck can be used to launch..

If you are running an ONOS cluster on a collection of remote servers, we use STC - I can send directions for this if you are interested

Thanks
Saurav





To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--

Charles Chan
Member of Technical Staff, ON.Lab



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

Klatsky, Carl

unread,
Mar 8, 2017, 4:50:57 PM3/8/17
to Saurav Das, Charles Chan, Jonathan Hart, ONOS Discuss

Thanks Saurav.  Let me check with the other folks here using ONOS about moving to the later release then work the change using buck.

 

I am out the next two days and will send on the results next week.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--

Charles Chan
Member of Technical Staff, ON.Lab



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

Saurav Das

unread,
Mar 13, 2017, 2:59:07 PM3/13/17
to Klatsky, Carl, Charles Chan, Jonathan Hart, ONOS Discuss
Carl,

If  you follow directions here:

you would not need to edit code yourself as my fix to the rest call has merged.
But you would need to change config on your ports 1-9 as per

Let me know if you want to go down this path and run into issues

Thanks
Saurav


To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--

Charles Chan
Member of Technical Staff, ON.Lab



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

Klatsky, Carl

unread,
Mar 13, 2017, 3:05:21 PM3/13/17
to Saurav Das, Charles Chan, Jonathan Hart, ONOS Discuss

Thanks & timely e-mail Saurav.  I just completed the upgrade to the current 1.10-SNAPSHOT which should have picked up your rest call change.  I am working now on the updates to network.json configuration file to be compatible with the latest syntax.  I working that first before the other ACL & Group entries in order to re-confirm my baseline in 1.10 is still OK.  Once that is confirmed, I will try the ACL & Group entries again.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

--

Charles Chan
Member of Technical Staff, ON.Lab



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--

You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

Klatsky, Carl

unread,
Mar 15, 2017, 11:57:49 AM3/15/17
to Klatsky, Carl, Saurav Das, Charles Chan, Jonathan Hart, ONOS Discuss

Saurav and all,

 

After the upgrade to ONOS-1.10-SNAPSHOT the other day, I made what I think are the needed adjustments to my network configuration file.  I change the ‘ports’ section to use the new VLAN definition and adjusted the ‘devices’ section per info I learned in other mail threads.  When I apply the configuration, I get errors in the ONOS CLI (attached) and the error below is shown on the switch CLI.

 

04-09 18:35:31.823583 [ofstatemanager] Error from Forwarding while inserting flow: Unknown error

 

I have also attached the old ONOS-1.7 working configuration (FINAL_ONOS9.txt) and the modified one for current ONOS support (FINAL_ONOS10.txt).  Any idea on what is triggering the error?  Thanks.

onos_log_errors.txt
FINAL_ONOS10.txt
FINAL_ONOS9.txt

Saurav Das

unread,
Mar 15, 2017, 1:39:25 PM3/15/17
to Klatsky, Carl, Charles Chan, Jonathan Hart, ONOS Discuss
Hi Carl,

A few things:

1) Did you upgrade the switch software? It is necessary to do that with the latest ONOS code/

2) Your config looks ok. I would recommend adding 

               "ipv6NodeSid" : 237,

                "ipv6Loopback" : "2000::c0a8:00e3",

to the device config, even though you don't need v6,  just because I am not sure what happens without it. In principle if config is not there, default values should be used, but I haven't tested it - so just to be safe.

3) It is still necessary to follow a certain order of bringing things up right now. We will remove this constraint in the near future, but for now

 - bring up the controller

 - configure it with netcfg

 - and then connect the switch to the controller

4) Ensure that the following apps are running for your case

ONOS_APPS=drivers,openflow,segmentrouting,netcfghostprovider

It can be verified on the ONOS cli with "apps -a -s"

5) Sometimes we need to see the logs from the beginning. Using the ONOS cli, and "log:tail" only shows the most recent logs. To get the full log, ssh into your controller machine and check

/opt/onos/log/karaf.log


In the meantime I will try out config similar to yours in my local setup


Thanks

Saurav








To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--

Charles Chan
Member of Technical Staff, ON.Lab



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

Pier Luigi Ventre

unread,
Mar 15, 2017, 3:08:03 PM3/15/17
to ONOS Discuss, Carl_K...@comcast.com, cha...@onlab.us, jo...@onlab.us
Hi Saurav,
the ipv6NodeSid and ipv6Loopback are not necessary, I usually run the fabric without.

Thanks
Pier

        at <a href="http:

Saurav Das

unread,
Mar 15, 2017, 4:08:29 PM3/15/17
to Pier Luigi Ventre, ONOS Discuss, Klatsky, Carl, Charles Min-Cheng Chan, Jonathan Hart
Great - thanks for the clarification Pier

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.
To post to this group, send email to onos-d...@onosproject.org.
Visit this group at https://groups.google.com/a/onosproject.org/group/onos-discuss/.
To view this discussion on the web visit https://groups.google.com/a/onosproject.org/d/msgid/onos-discuss/f6e82508-3ca5-4757-8f1c-09afc4e2c6ed%40onosproject.org.

Saurav Das

unread,
Mar 15, 2017, 9:06:45 PM3/15/17
to Pier Luigi Ventre, ONOS Discuss, Klatsky, Carl, Charles Min-Cheng Chan, Jonathan Hart
Hi Carl,

I tried your case with a very setup similar to yours. With the right ONOS, ONL and OF-DPA, and the correct order for bringing things up, everything works.
I saw no issues in the initial flow/group programming.
I was able to ping between the two hosts I had configured (in vlan 10). Specifically I was pinging from 10.0.1.10 (port 12) to 10.0.1.20 (port 16).
Next, I was able to program the L2_Interface and L3_Unicast groups through the rest interface.
Note that with the newer switch software, there were no errors in the L3_Unicast group programming or reconciliation in ONOS.
Finally, with the ping running, I was able to program the ACL table rule, and notice that the ping fails, as the packets from port 12 are redirected to port 24. I was able to verify with tcpdump at a host connected to port 24 that the packets are indeed coming out with the mac-addresses intact.

Details in the attachment

Thanks
Saurav

acl-test.rtf

Klatsky, Carl

unread,
Mar 16, 2017, 6:40:05 AM3/16/17
to Saurav Das, Pier Luigi Ventre, ONOS Discuss, Charles Min-Cheng Chan, Jonathan Hart

Hi Saurav,

 

That’s great news and thank you so much for checking that.  At the end of the day yesterday, I was still upgrading the ONL & OF-DPA on our leaf switches.  Hopefully, I will be able to finish that today and move onto adding the group & ACL entries.  Will let you know.

 

Regards,

Carl Klatsky

 

From: Saurav Das [mailto:wildc...@gmail.com]
Sent: Wednesday, March 15, 2017 9:07 PM
To: Pier Luigi Ventre <pl.v...@gmail.com>
Cc: ONOS Discuss <onos-d...@onosproject.org>; Klatsky, Carl <Carl_K...@cable.comcast.com>; Charles Min-Cheng Chan <cha...@onlab.us>; Jonathan Hart <jo...@onlab.us>
Subject: Re: [onos-discuss] Flow add problem

 

Hi Carl,

Klatsky, Carl

unread,
Mar 16, 2017, 2:01:07 PM3/16/17
to Saurav Das, Pier Luigi Ventre, ONOS Discuss, Charles Min-Cheng Chan, Jonathan Hart

Hi Saurav,

 

I completed the ONL & OFDPA upgrades to our leaf switches.  I was then able to successfully load the updated network configuration file including the change noted in your last mail (attached as FINAL_ONOS11.txt).  I only observed two warnings in the ONOS logs:

…

2017-03-16 12:33:28,798 | WARN  | ew I/O worker #1 | OFChannelHandler                 | 166 - org.onosproject.onos-protocols-openflow-ctl - 1.10.0.SNAPSHOT | Config Reply from switch [/10.255.14.164:60496 DPID[00:0:00:00:00:00:00:01]] has miss length set to 0

2017-03-16 12:33:28,807 | WARN  | ew I/O worker #1 | OFChannelHandler                 | 166 - org.onosproject.onos-protocols-openflow-ctl - 1.10.0.SNAPSHOT | Received OFError BAD_REQUEST. It seems of:000000000000001 does not support Meter.

…

2017-03-16 12:33:29,025 | WARN  | tive-installer-0 | Ofdpa3Pipeline                   | 159 - org.onosproject.onos-drivers-default - 1.10.0.SNAPSHOT | ICMPv6 Type and ICMPv6 Code are not supported

 

This got my baseline configuration working as before J

 

I then tried to add the flows and groups in the order you previously noted: L2_interface group first, then the L3_Unicast group, then ACL flow.

 

The L2_interface group & L3_Unicast group settings loaded fine with no messages in the log file, but when I added the ACL entry in the flows, this error appeared in the log.

 

….

2017-03-16 12:53:25,421 | ERROR | ew I/O worker #1 | OFChannelHandler                 | 166 - org.onosproject.onos-protocols-openflow-ctl - 1.10.0.SNAPSHOT | OFBadActionErrorMsgVer13(xid=100, code=BAD_TYPE, data=OFFlowAddVer13(xid=100, cookie=0x00380000fc43deee, cookieMask=0x0000000000000000, tableId=0x3c, idleTimeout=0, hardTimeout=0, priority=65535, bufferId=4294967295, outPort=any, outGroup=any, flags=[SEND_FLOW_REM], match=OFMatchV3Ver13(in_port=1, eth_type=0x800, ipv4_src=10.255.227.0/255.255.255.0), instructions=[OFInstructionApplyActionsVer13(actions=[OFActionGroupVer13(group=536875008)])])) from switch org.onosproject.driver.handshaker.DefaultSwitchHandshaker [/10.255.14.164:60496 DPID[00:00:00:00:00:00:00:01]] in state ACTIVE

2017-03-16 12:53:25,423 | WARN  | vent-error-msg-0 | OpenFlowRuleProvider             | 169 - org.onosproject.onos-providers-openflow-flow - 1.10.0.SNAPSHOT | Received error message OFBadActionErrorMsgVer13(xid=100, code=BAD_TYPE, data=OFFlowAddVer13(xid=100, cookie=0x00380000fc43deee, cookieMask=0x0000000000000000, tableId=0x3c, idleTimeout=0, hardTimeout=0, priority=65535, bufferId=4294967295, outPort=any, outGroup=any, flags=[SEND_FLOW_REM], match=OFMatchV3Ver13(in_port=1, eth_type=0x800, ipv4_src=10.255.227.0/255.255.255.0), instructions=[OFInstructionApplyActionsVer13(actions=[OFActionGroupVer13(group=536875008)])])) from 00:00:00:00:00:00:00:01

2017-03-16 12:53:29,959 | ERROR | ew I/O worker #1 | OFChannelHandler                 | 166 - org.onosproject.onos-protocols-openflow-ctl - 1.10.0.SNAPSHOT | OFBadActionErrorMsgVer13(xid=0, code=BAD_TYPE, data=OFFlowAddVer13(xid=0, cookie=0x00380000fc43deee, cookieMask=0x0000000000000000, tableId=0x3c, idleTimeout=0, hardTimeout=0, priority=65535, bufferId=4294967295, outPort=any, outGroup=any, flags=[SEND_FLOW_REM], match=OFMatchV3Ver13(in_port=1, eth_type=0x800, ipv4_src=10.255.227.0/255.255.255.0), instructions=[OFInstructionApplyActionsVer13(actions=[OFActionGroupVer13(group=536875008)])])) from switch org.onosproject.driver.handshaker.DefaultSwitchHandshaker [/10.255.14.164:60496 DPID[00:00:00:00:00:00:00:01]] in state ACTIVE

2017-03-16 12:53:29,961 | WARN  | vent-error-msg-0 | OpenFlowRuleProvider             | 169 - org.onosproject.onos-providers-openflow-flow - 1.10.0.SNAPSHOT | Received error message OFBadActionErrorMsgVer13(xid=0, code=BAD_TYPE, data=OFFlowAddVer13(xid=0, cookie=0x00380000fc43deee, cookieMask=0x0000000000000000, tableId=0x3c, idleTimeout=0, hardTimeout=0, priority=65535, bufferId=4294967295, outPort=any, outGroup=any, flags=[SEND_FLOW_REM], match=OFMatchV3Ver13(in_port=1, eth_type=0x800, ipv4_src=10.255.227.0/255.255.255.0), instructions=[OFInstructionApplyActionsVer13(actions=[OFActionGroupVer13(group=536875008)])])) from 00:00:00:00:00:00:00:01

 

And this error in the switch:

 

04-10 20:38:07.726111 [indigo_ofdpa_driver] unsupported action of_action_group

04-10 20:38:07.726178 [indigo_ofdpa_driver] Error processing Apply actions for flow table 60.

 

Do I need a change in the ACL flow entry when using the newer ONOS version?

 

Regards,

Carl Klatsky

 

From: Klatsky, Carl
Sent: Thursday, March 16, 2017 6:40 AM
To: 'Saurav Das' <wildc...@gmail.com>; Pier Luigi Ventre <pl.v...@gmail.com>
Cc: ONOS Discuss <onos-d...@onosproject.org>; Charles Min-Cheng Chan <cha...@onlab.us>; Jonathan Hart <jo...@onlab.us>
Subject: RE: [onos-discuss] Flow add problem

 

Hi Saurav,

 

That’s great news and thank you so much for checking that.  At the end of the day yesterday, I was still upgrading the ONL & OF-DPA on our leaf switches.  Hopefully, I will be able to finish that today and move onto adding the group & ACL entries.  Will let you know.

 

Regards,

Carl Klatsky

 

From: Saurav Das [mailto:wildc...@gmail.com]

Sent: Wednesday, March 15, 2017 9:07 PM
To: Pier Luigi Ventre <pl.v...@gmail.com>

Cc: ONOS Discuss <onos-d...@onosproject.org>; Klatsky, Carl <Carl_K...@cable.comcast.com>; Charles Min-Cheng Chan <cha...@onlab.us>; Jonathan Hart <jo...@onlab.us>
Subject: Re: [onos-discuss] Flow add problem

 

Hi Carl,

FINAL_ONOS11.txt
acl_entry_v1.txt
l3_unicast_group_v4.txt
l2_interface_group_v2.txt
karaf.log

Saurav Das

unread,
Mar 16, 2017, 2:20:48 PM3/16/17
to Klatsky, Carl, Pier Luigi Ventre, ONOS Discuss, Charles Min-Cheng Chan, Jonathan Hart
It has to be deferred

{
  "flows": [
    {
      "priority": 65535,
      "timeout": 0,
      "isPermanent": true,
      "deviceId": "of:0000000000000001",
      "tableId" : 60,
       "treatment": {
          "deferred": [
          {
            "type": "GROUP",
            "groupId": "536875008"
          }
        ]
      },
      "selector": {
        "criteria": [
        {
          "type": "ETH_TYPE",
          "ethType": "0x800"
        },
        {
           "type": "IN_PORT",
           "port": "1"
        },
        {
          "type": "IPV4_SRC",
          "ip": "10.255.227.0/24"
         }
        ]
      }
    }
  ]
}

Klatsky, Carl

unread,
Mar 16, 2017, 3:47:09 PM3/16/17
to Saurav Das, Pier Luigi Ventre, ONOS Discuss, Charles Min-Cheng Chan, Jonathan Hart

Thanks for spotting my error.  Once I made the change and applied the ACL, it seems to have worked but I need to take additional steps to fully confirm.  In my setup, since the VNF was not yet ready, I had port 9 connected with an external fiber loop to port 10.  I also made a test network configuration that included ports 9 & 10 also untagged into VLAN 10 which is the VLAN that would service this subnet when the ACL is not applied.

 

After applying the ACL, I did see the ping I had going stop but I had also thought with the packets being looped back in port 10, that the incoming packets to port 10 would be forwarded on using standard bridge forwarding.

 

In any case, this is a big step forward for us here.  I will find some equipment I can use to connect to port 9 and confirm the packets are transmitted it out using the expected  source & destination MAC addresses. 

 

Thank you so much for supporting this effort!

 

Regards,

Carl Klatsky

 

From: Saurav Das [mailto:wildc...@gmail.com]

Sent: Thursday, March 16, 2017 2:21 PM
To: Klatsky, Carl <Carl_K...@cable.comcast.com>

--

You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

Saurav Das

unread,
Mar 16, 2017, 7:06:27 PM3/16/17
to Klatsky, Carl, Pier Luigi Ventre, ONOS Discuss, Charles Min-Cheng Chan, Jonathan Hart
On Thu, Mar 16, 2017 at 12:47 PM, Klatsky, Carl <Carl_K...@comcast.com> wrote:

Thanks for spotting my error.  Once I made the change and applied the ACL, it seems to have worked but I need to take additional steps to fully confirm.  In my setup, since the VNF was not yet ready, I had port 9 connected with an external fiber loop to port 10.  I also made a test network configuration that included ports 9 & 10 also untagged into VLAN 10 which is the VLAN that would service this subnet when the ACL is not applied.

 

After applying the ACL, I did see the ping I had going stop but I had also thought with the packets being looped back in port 10, that the incoming packets to port 10 would be forwarded on using standard bridge forwarding.


This should have worked. One thing to take care of here -- unlike hosts or routes, interface config cannot be loaded or changed dynamically -- this is technical debt on our side. So if you added your test  config later (ports 9 and 10 in vlan 10), then it may not have been picked up. If you do it in the beginning, and it still does not work through the fiber loop, send us the flows/groups


 

--

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

Klatsky, Carl

unread,
Mar 17, 2017, 3:13:28 PM3/17/17
to Saurav Das, Pier Luigi Ventre, ONOS Discuss, Charles Min-Cheng Chan, Jonathan Hart

I re-ran the test and got the same results.  I confirmed that interfaces 9 & 10 were added to VLAN to 10 in the initial network configuration prior to adding the L2 Interface group, L3 Unicast group, and the ACL entry.  Attached the flows & groups output following adding of the network configuration (before files) and after L2 Interface group, L3 Unicast group, and the ACL entry (after files).

 

Since port 10 was added to VLAN 10, my assumption was that the incoming return traffic from the VNF (in this case just the fiber loop) would use standard bridge forward destination MAC lookup for that VLAN to be forwarded onto it destination (Termination MAC Table?).  Is my assumption correct, or is a flow rule needed for this incoming?

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

flows_after_acl_added_v2.txt
groups_before_v2.txt
flows_before_acl_added_v2.txt
groups_after_v2.txt

Saurav Das

unread,
Mar 17, 2017, 5:24:40 PM3/17/17
to Klatsky, Carl, Pier Luigi Ventre, ONOS Discuss, Charles Min-Cheng Chan, Jonathan Hart
Honestly I don't see anything wrong with the flows/groups.

After doing the rest programming and sending a bunch of traffic, can you try to see the portstats to see where packets are getting dropped

Try:

onos> portstats -nz

and also send  a few (say 5 taken 5 secs apart) 
 
onos> portstats -nz -d

Thanks
Saurav



To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.

To post to this group, send email to onos-d...@onosproject.org.
Visit this group at https://groups.google.com/a/onosproject.org/group/onos-discuss/.
To view this discussion on the web visit https://groups.google.com/a/onosproject.org/d/msgid/onos-discuss/90ad8a36f8d140aa97d9afca7fe98a20%40PACDCEX43.cable.comcast.com.

Klatsky, Carl

unread,
Mar 17, 2017, 8:21:56 PM3/17/17
to Saurav Das, Pier Luigi Ventre, ONOS Discuss, Charles Min-Cheng Chan, Jonathan Hart

Understood.  I will try those checks on Monday.

 

One thought I had – we are running the lldpprovider app in addition to segmentrouting and a few others.  Is it possible that when the external fiber loop is in place between ports 9 & 10, that lldpprovider app when it detects the loop takes one or both of the ports out of service?

 

onos> apps -s -a

*   4 org.onosproject.drivers              1.10.0.SNAPSHOT Default device drivers

*   7 org.onosproject.optical-model        1.10.0.SNAPSHOT Optical information model

*  14 org.onosproject.lldpprovider         1.10.0.SNAPSHOT LLDP Link Provider

*  15 org.onosproject.openflow-base        1.10.0.SNAPSHOT OpenFlow Provider

*  31 org.onosproject.netcfghostprovider   1.10.0.SNAPSHOT Network Config Host Provider

*  60 org.onosproject.segmentrouting       1.10.0.SNAPSHOT Segment Routing App

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.

Saurav Das

unread,
Mar 17, 2017, 9:31:51 PM3/17/17
to Klatsky, Carl, Pier Luigi Ventre, ONOS Discuss, Charles Min-Cheng Chan, Jonathan Hart
It's a good thought but lldpprovider app is not designed to take out loops.

But a loop is an issue we are seeing. I tried to do what you are doing in my setup. I put all four ports in the same vlan, and then put an external cable-loop between the redirect ports. I saw that the ACL table rule sent packets out of the redirect port (eg 9 in your setup) and so it comes back in through the other port (eg. 10 in your setup) and dies. It does not get bridged back out of the original dst-port (port 5) as we hoped.

In portstats you will see the RxDrp counters go up

port=32, pktRx=26, pktTx=6, bytesRx=3072, bytesTx=1032, rateRx=Infinity, rateTx=Infinity, pktRxDrp=26, pktTxDrp=0, interval=0.000

I am not sure why this is happening.

But the external loop does cause other problems. In particular broadcast packets get flooded through these ports, loop back in and get amplified and eventually the LEDs on all these ports start flashing funny and the counters go beserk. It is safe to say that these ports should not all be on the same broadcast domain given the external loop.

So I put the original ports in a vlan  and importantly left the two redirect ports (9 and 10 in your setup) un-configured.

Then using the groups and the ACL table rules as before I was able to send packets out of the 1st redirect port (port 9 in your setup). With the external cable loop these come back in through the other port (port 10). But because port 10 is un-configured there is no broadcast domain associated with it, although it does get a different (internal) vlan tag.

At this point you need another flow and group to direct these packets back out of the original dst - port (port 5)

1) Create an L2 Interface group with group id 0xffe0005 - assuming you want to go back out of port 5. Please use the decimal version of the groupId in the rest call. And make sure you use a different "appCookie" than the ones you used before. The group treatment->instructions should do VLAN_POP and OUTPUT out of port 5.

2) You don't need an L3_Unicast group in this direction. You will need another ACL rule that will match on the incoming port 10, and the dst_mac of the router on port 5, and send it out of the group you just created to send out of port 5.

In my setup, it looks like this:


curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{ \ 
   "priority": 65535, \ 
   "timeout": 0, \ 
   "isPermanent": true, \ 
   "deviceId": "of:0000000000000204", \ 
   "tableId": 60, \ 
   "treatment": { \ 
     "deferred": [ \ 
       { \ 
         "type": "GROUP", \ 
         "groupId": "268304400" \ 
       } \ 
     ] \ 
   }, \ 
   "selector": { \ 
     "criteria": [ \ 
       { \ 
         "type": "IN_PORT", \ 
         "port": "32" \ 
       }, \ 
       { \ 
        "type": "ETH_DST", \ 
       "mac": "B2:A4:E2:72:D1:91" \ 
       } \ 
     ] \ 
   } \ 
 }' 'http://10.128.0.216:8181/onos/v1/flows/of%3A0000000000000204?appId=org.onosproject.rest'

According to the scenarios you had sent earlier:


Scenario 1

1)  Leaf switch ports 1 & 2 are defined together in a bridge domain with router-1 and router-2 connected to those ports respectively

2)  The router interfaces connecting to those ports are on the same IP subnet.  They can communicate and exchange routing information about the additional subnets they connect.  The routing protocol packets in this case get standard L2 bridged based on destination MAC or flooded based on destination MAC of broadcast / multicast.

3)  Packets coming into port 1 from router-1’s connected IP subnet(s) will have: source MAC of router-1; source IP from a connected IP subnet on router-1; destination MAC of router-2; destination IP to a connected subnet on router-2

4)  Can a flow rule be added for packets coming in port-1, and match for the source IP subnet(s) connected to router-1 (but not the router-1 / router-2 IP subnet), and if match direct the packet out port 3, and  if not matched then fall back to forwarding on destination MAC using standard L2 bringing?  Basically, the goal is to move the traffic from the connected subnets out a specific port but exclude any traffic originating from router-1’s own IP address.

5)  Similar question as 4) but in reverse.  Can a flow rule be added for packets coming in port-2, and match for the destination IP subnet(s) connected to router-1 (but not the router-1 / router-2 IP subnet), and if match direct the packet out port 4, and  if not matched then fall back to forwarding on destination MAC using standard L2 bringing?

 

Scenario 2

1)  Assuming scenario 1 is possible and continuing from there, packets directed out port 3, after transiting an intermediate node, are sent back in port 4 but retaining their original source MAC / source IP / destination MAC / destination IP values

2)  As the packets come into port 4, can a flow rule be added that matches for the destination MAC of router-2, and if matched send those packets out port 2 towards router-2?  With the objective of NOT including port 4 in the same bridge domain as port 1 / port 2.

3)  Similar question as 2) but in reverse.  As the packets come into port 3, can a flow rule be added that matches for the destination MAC of router-1, and if matched send those packets out port 1 towards router-1?  With the objective of NOT including port 3 in the same bridge domain as port 1 / port 2.



I believe we have  Scenario 1 (1 to 4) and Scenario 2 (1 to 2)  working in our lab, but you need to try the above and confirm


Thanks to Charles for his help in debugging

Cheers
Saurav









To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss+unsubscribe@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali

Klatsky, Carl

unread,
Mar 20, 2017, 12:04:13 PM3/20/17
to Saurav Das, Pier Luigi Ventre, ONOS Discuss, Charles Min-Cheng Chan, Jonathan Hart

Hi Saurav,

 

I created the additional L2_Interface group & ACL to forward the looped back traffic received on port 10 with matching destination MAC of the router connected to port 5, and send it out port 5.  This worked!!!  I confirmed the ping works, and if I pull out the external fiber loop, the ping stops.  This gets Scenario 1 (1 to 4) and Scenario 2 (1 to 2)  working.

 

I then tried to configure the groups & flow rules for the reverse path [Scenario 1 (5) and Scenario 2 (3)].  The sequence I followed: create an L2 Interface group; L3 Unicast group; and ACL rule to match the reverse traffic and send it out port 10; create another L2 Interface group & ACL rule to match the return looped traffic coming in port 9 matching the destination MAC of the router connect to port 1, and send it out port 1.

 

When I went to add the first L2 Interface group from the new set, I got an INVALID_GROUP error in the ONOS logs and this error on the switch CLI: [indigo_ofdpa_driver] Error in adding Group bucket, rv = -22

 

I used unique group IDs and appCookie for this second set of groups & flow rule, which I believe are correct, but can you please double check me?  Thanks.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.



 

--

The will MUST be stronger than the skill -- Ali

acl-3_entry_v1.txt
acl-2_entry_v1.txt
acl-4_entry_v1.txt
l2_interface_group_v2.txt
l2-2_interface_group_v1.txt
l2-3_interface_group_v1.txt
l2-4_interface_group_v1.txt
l3_unicast_group_v4.txt
l3-3_unicast_group_v1.txt
acl_entry_v3.txt

Saurav Das

unread,
Mar 20, 2017, 1:49:29 PM3/20/17
to Klatsky, Carl, Charles Min-Cheng Chan, Jonathan Hart, ONOS Discuss, Pier Luigi Ventre
Carl 

For app cookie u can put anything u want as long as it is unique. There are some rules for group ids. 

For l2 interface groups the format is 

0x12bit-vlan16bitportid

So to send out of port 10 on vlan 4094 the group id is 0xffe000a

For l3unicast just start with 0x2 followed by 28 bit index which should be unique

Also in l2interface you do not need to specify a vlanid for the pop vlan instruction 


Thanks
Saurav

Klatsky, Carl

unread,
Mar 20, 2017, 2:00:32 PM3/20/17
to Saurav Das, Charles Min-Cheng Chan, Jonathan Hart, ONOS Discuss, Pier Luigi Ventre

Thanks Saurav.  I will follow the rule syntax you note below.  For all the group ID, after I arrive at the hex value per the syntax, should I still continue to enter the value as decimal in the JSON structure?

Charles Chan

unread,
Mar 20, 2017, 2:05:38 PM3/20/17
to Klatsky, Carl, Saurav Das, Jonathan Hart, ONOS Discuss, Pier Luigi Ventre
Carl,

Yes, you still need to convert the group id to its decimal value for the REST request.

Charles
--

Charles Chan
Member of Technical Staff, ON.Lab

Member of Ambassador Steering Team, ONOS/CORD Community

Klatsky, Carl

unread,
Mar 20, 2017, 3:48:35 PM3/20/17
to Saurav Das, Charles Min-Cheng Chan, Jonathan Hart, ONOS Discuss, Pier Luigi Ventre

I made the changes to the groupID including specifying the ID in decimal in the REST API call, and it all seems to be working now.  Just a few follow on questions:

1)      At one point in this scenario, you had noted “that the mac addresses need to be set when using this group. But we can set it to the router mac addresses which we know, so in the end the mac addresses on the packet will remain unchanged.”  I was not clear on why that was needed?  I had thought the original source & destination MAC on the incoming packet would be retained.

2)      In the recent part of this discussion, after the packets are sent out port 9 and looped back in port 10 you had noted “It does not get bridged back out of the original dst-port (port 5) as we hoped.”  Any further thoughts on why they did not get bridged out once the port was added to the VLAN?  Just trying to confirm if the additional rules to forward the looped packets is really needed.

3)      I am glad we were able to complete out this scenario.  Its gives me a good base understanding to work with.  The team here wants to try this revise this scenario to be VLAN based and to move all the traffic through to the VNF, instead of trying to let the router control plane traffic bypass the VNF.  I think I can develop the rules based on what we did here.  If I need to raise a question for this revised scenario, I assume I should start a new discussion topic, yes?

 

Regards,

Carl Klatsky

 

From: Saurav Das [mailto:wildc...@gmail.com]

Sent: Monday, March 20, 2017 1:49 PM
To: Klatsky, Carl <Carl_K...@cable.comcast.com>

Cc: Charles Min-Cheng Chan <cha...@onlab.us>; Jonathan Hart <jo...@onlab.us>; ONOS Discuss <onos-d...@onosproject.org>; Pier Luigi Ventre <pl.v...@gmail.com>

ggi...@palcnetworks.com

unread,
Jul 10, 2018, 6:08:33 AM7/10/18
to ONOS Discuss, jo...@onlab.us
Hi Saurav,

I'm also facing the same problem while creating the flow rule from REST API.
I have created the L2 interface group, L3 unicast group followed by flow group.

These are the outputs of group tables.

onos> devices
id=of:000000000000da7a, available=true, role=MASTER, type=SWITCH, mfr=Broadcom Corp., hw=OF-DPA 2.0, sw=OF-DPA 2.0, serial=, managementAddress=10.10.26.122, protocol=OF_13, channelId=10.10.26.122:57059
onos> groups
deviceId=of:000000000000da7a, groupCount=2
   id=0xffe0009, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.rest
   id=0xffe0009, bucket=1, bytes=0, packets=0, actions=[OUTPUT:9, VLAN_POP]
   id=0x20001000, state=ADDED, type=INDIRECT, bytes=0, packets=0, appId=org.onosproject.rest
   id=0x20001000, bucket=1, bytes=0, packets=0, actions=[ETH_DST:34:DB:FD:03:06:41, ETH_SRC:00:21:A0:CE:2B:24, VLAN_ID:4094, GROUP:0xffe0009]
onos>

REST:
=====
curl -X POST --header "Content-Type: application/json" --header "Accept: application/json" -d "{
  \"flows\": [
    {
      \"priority\": 1,
      \"timeout\": 0,
      \"isPermanent\": true,
      \"deviceId\": \"of:000000000000da7a\",
      \"tableId\" : 60,
       \"treatment\": {
          \"deferred\": [
          {
            \"type\": \"GROUP\",
            \"groupId\": \"536875008\"
          }
        ]
      },
      \"selector\": {
        \"criteria\": [
        {
          \"type\": \"ETH_TYPE\",
          \"ethType\": \"0x800\"
        },
        {
           \"type\": \"IN_PORT\",
           \"port\": \"1\"
        },
        {
          \"type\": \"IPV4_SRC\",
          \"ip\": \"10.255.227.0/24\"
         }
        ]
      }
    }
  ]
}" "http://10.12.22.105:8181/onos/v1/flows/of%3A000000000000da7a"


When i'm trying add the flow rule, I'm getting below error.
{ "code": 400, "message": "priority member is required in FlowRule" }

{
  "flows": [
    {
      "priority": 1,
      "timeout": 0,
      "isPermanent": true,
      "deviceId": "of:000000000000da7a",
      "tableId" : 60,
       "treatment": {
          "deferred": [
          {
            "type": "GROUP",
            "groupId": "536875008"
          }
        ]
      },
      "selector": {
        "criteria": [
        {
          "type": "ETH_TYPE",
          "ethType": "0x800"
        },
        {
           "type": "IN_PORT",
           "port": "1"
        },
        {
          "type": "IPV4_SRC",
          "ip": "10.255.227.0/24"
         }
        ]
      }
    }
  ]
}

Regards,
Girish

On Thursday, February 23, 2017 at 12:21:13 AM UTC+5:30, Carl Klatsky wrote:
Hi,

Continuing on from the items discussed in topic "Segment routing application questions" using the same configuration, I am trying to 'hard code' flows by inserting them with the web interface to the REST API.  When I add the flow, the web UI reports back a 200 OK to the API call, but when checked on the ONOS CLI the flow stays in the PENDING ADD state.  An error appears in the ONOS logs, but its not clear to me what the error means.

I have attached the *.json used in the flow add, the flow listing before & after the change, and the ONOS log snippet.  The full curl command is shown below.  Any help is appreciated.

Thanks & Regards,
Carl Klatsky

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{ \ 
   "flows": [ \ 
     { \ 
       "priority": 1, \ 
       "timeout": 0, \ 
       "isPermanent": true, \ 
       "deviceId": "of:0000000000000001", \ 
        "treatment": { \ 
           "instructions": [ \ 
           { \ 
             "type": "OUTPUT", \ 
             "port": "9" \ 
           } \ 
         ] \ 
       }, \ 
         "selector": { \ 
           "criteria": [ \ 
           { \ 
           "type": "ETH_TYPE", \ 
           "ethType": "0x800" \ 
           }, \ 
           { \ 
            "type": "IN_PORT", \ 
            "port": "1" \ 
           }, \ 
           { \ 
             "type": "IPV4_SRC", \ 
             "ip": "10.255.227.0/24" \ 
           } \ 
         ] \ 
       } \ 
     } \ 
   ] \ 
 }' 'http://10.255.14.167:8181/onos/v1/flows?appId=org.onosproject.segmentrouting'

Saurav Das

unread,
Jul 10, 2018, 4:11:14 PM7/10/18
to ggi...@palcnetworks.com, r...@opennetworking.org, onos-d...@onosproject.org, Jonathan Hart
I honestly don't see anything wrong with the flow rule, but for some reason the decoder is reading null for the priority field when it is obviously there - perhaps Ray can spot something

Thanks
Saurav

--
You received this message because you are subscribed to the Google Groups "ONOS Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to onos-discuss...@onosproject.org.
To post to this group, send email to onos-d...@onosproject.org.
Visit this group at https://groups.google.com/a/onosproject.org/group/onos-discuss/.
To view this discussion on the web visit https://groups.google.com/a/onosproject.org/d/msgid/onos-discuss/946b63e4-cf86-4080-986e-5f740af31183%40onosproject.org.

Hadjer MEDJHOUM

unread,
Jun 2, 2022, 9:00:53 AM6/2/22
to ONOS Discuss, Carl Klatsky
Hi Carl, 
I want to add flows to my topologie mininet with ONOS controller, but i don't know how to add them in the REST API, or in any other interface.
Could you or anyone else help me to acheive that,( ie, what are the steps that i have to do, to have interface to manage flows on ONOS controller)?

Hadjer

Klatsky, Carl

unread,
Jun 3, 2022, 9:20:01 AM6/3/22
to Hadjer MEDJHOUM, ONOS Discuss

Hi Hadjer,

 

Unfortunately, I have not been working on ONOS related efforts since late 2018, and I do not recall what the solution to get around this issue was.  I could not find the resolution in my notes either.

 

Regards,

Carl Klatsky

0ice...@gmail.com

unread,
Jun 8, 2022, 9:25:52 AM6/8/22
to ONOS Discuss, Hadjer MEDJHOUM
Hi Hadjer,

You can access the onos REST via  http://localhost:8181/onos/v1/docs then you have to looking for the tab flows or flowrules and then you can create your  flow rule json, here the tips for the syntax https://wiki.onosproject.org/display/ONOS/Flow+Rules#:~:text=Flow%20Rule%20Criteria,-JSON%20Format&text=During%20a%20flow%20POST%20request,work%20in%20a%20POST%20request.

Best
Davide

Reply all
Reply to author
Forward
0 new messages